Skip to main content

Every model, every provider, one endpoint

A unified access layer for LLMs. Apps and agents from any vendor call commercial and open-weight models through one endpoint, routed by cost or fit and metered per team, while provider keys stay in the gateway.

Who calls it

  • Your apps
  • Agents
  • People

AI Gateway

Every request, in this order:

  1. 01AccessEvery model behind one endpoint and one key.
  2. 02ControlBudgets, rate limits and allowed models per team.
  3. 03CostRouted to the cheapest or best-fit host, with fallback.
  4. 04VisibilityEvery request metered by model, team and consumer.

Commercial models

  • Claude
  • GPT-4o

Open-weight models

  • Llama
  • Qwen
  • Mistral

Hosts

  • AWS
  • Alibaba Cloud
  • AI factories

Every model, host and rate behind one key.

  1. Commercial models and open-weight models such as Llama and Qwen sit behind a single endpoint. Switching models means changing a model name, not an integration.

    • Commercial and open-weight models
    • Many hosting providers
    • Change the name, keep the code

Usage metered per model, team and consumer

Every request is metered in tokens and cost, then attributed to the model, the team and the consumer that made it, so spend can be billed back to whoever used it.

Total tokens
4.82M
Input tokens
3.55M
Output tokens
1.27M
  • qwen-2.5-72b1.92M
  • claude-sonnet1.24M
  • gpt-4o0.93M
  • llama-3.1-70b0.73M
  • Tokens in and out, and cost
  • By model, team or consumer
  • Billing per team or consumer

Cap spend before it happens.

Budgets and rate limits

Cap spend and requests per team, project or app key, and choose which models each team is cleared to use.

  • Spend cap and request limit
  • Models enabled per project

Spend caps with a circuit breaker

Get an alert at 80% of a budget. At the cap, the gateway throttles, blocks, or falls back to a cheaper model, whichever you chose.

Guardrails

Mask personal data and stop prompt injection before a request reaches a model.

A log of every request

Who sent it, with which key, to which model, how long it took and what it cost.

Route to the cheapest or best-fit endpoint

Set a routing rule per model or per consumer: lowest cost, a preferred host, or one fixed endpoint. If an endpoint is down or rate limited, the request falls back to the next one.

  • Lowest cost, preferred host or fixed
  • Fallback on outage or rate limit
  • The log shows which endpoint served it

More ways to cut the bill.

Load balancing

Spread traffic for one model across hosts, regions and keys, so no single one becomes the bottleneck.

Routing by difficulty

Simple requests go to a cheaper model and hard ones to a stronger model, without a code change.

Caching

Answer repeated and near-identical requests from a cache instead of paying a model for them again.

Two sides of the same catalog.

For model consumers

A catalog of every model available to you: where each one is hosted, what each endpoint costs, and which ones your team is cleared to use.

  • Where each model is hosted
  • What each endpoint costs
  • What your team can use

For model providers

Data centers and AI factories onboard and host models, set their own pricing, and track usage and revenue per model.

  • Onboard and host models
  • Set your own pricing
  • Usage and revenue per model