An AI gateway is a single API endpoint that sits in front of many model providers. Instead of integrating Anthropic, Google and OpenAI separately — three SDKs, three keys, three invoices — your app sends one request shape to one base URL with one key, and the gateway routes it to the right provider. It is also called an LLM gateway when the focus is text models.
What an AI gateway does
A gateway is the control plane between your application and the model providers. The core responsibilities:
| Job | What it means |
|---|---|
| Unified API | One request/response shape (usually OpenAI-compatible) for every model, so you integrate once. |
| Provider routing | Pick the model/provider per request; swap models by changing a string, not rewriting client code. |
| Failover | Retry on a healthy upstream when one provider errors or rate-limits — resilience your app doesn't have to code. |
| Caching | Reuse identical or stable prompts (e.g. Anthropic prompt caching) to cut repeat input cost. |
| Spend & rate control | Per-key budgets and rate limits so one leaked key or runaway loop can't drain the account. |
| Observability & billing | Per-call logging, token/cost usage, and — for inference gateways — a single wallet and invoice. |
When you need one
Reach for an AI gateway as soon as any of these is true:
- You call more than one model or provider. A gateway turns “add a provider” into a config change.
- You want failover. Providers degrade; a gateway routes around it without a redeploy.
- You need cost control and visibility. Per-key spend caps and usage logs beat reading three provider dashboards.
- You want one bill. One wallet across text, image, video and audio instead of an account per provider.
For a throwaway single-model prototype you can call a provider directly — but starting behind a gateway keeps every later change a one-line edit.
Two kinds of AI gateway
The term covers two architectures that solve different halves of the problem:
| Type | You bring | Examples | Best for |
|---|---|---|---|
| BYO-key gateway | Your own provider keys + billing | Portkey, Helicone, LiteLLM | Routing, guardrails and observability over accounts you already have |
| Inference gateway | Nothing — the gateway resells access | Kunavo, OpenRouter | One wallet and one key across providers, often below list price |
BYO-key gateways add a control/observability layer on top of your existing Anthropic, Google and OpenAI accounts — you still hold those keys and pay each provider. Inference gateways front the providers themselves: you top up one balance and never touch a provider console. See Kunavo vs Portkey, Helicone and LiteLLM for the BYO-key side, Kunavo vs Cloudflare AI Gateway for the edge-proxy case, Kunavo vs Kong AI Gateway for the enterprise API-management case, and Kunavo vs OpenRouter for the router comparison.
Kunavo as an AI gateway
Kunavo is an inference AI gateway. It is OpenAI-wire-compatible at https://api.kunavo.com/v1, so if your code can call the OpenAI API it can call Kunavo by changing only the base URL. One sk-kunavo- bearer token reaches Claude, Gemini, GPT and 200+ models — priced roughly 60% under the providers' list rates, pay-as-you-go from a $5 top-up, one Stripe-billed wallet.
from openai import OpenAI
# One gateway endpoint + one key reaches every provider.
client = OpenAI(
api_key="sk-kunavo-...",
base_url="https://api.kunavo.com/v1", # the AI gateway
)
# Same request shape, different providers — just change the model.
for model in ["claude-sonnet-4-6", "gemini-2-5-flash", "gpt-5-4"]:
r = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "One word: ready?"}],
)
print(model, "→", r.choices[0].message.content)The same key also does the native Anthropic Messages API, image, video and audio generation — see the OpenAI-compatible API guide for how the wire format lets you move between local (Ollama) and hosted models without a code change, and the quickstart for your first call.
FAQ
What is an AI gateway?
A single API endpoint in front of many model providers. Your app uses one base URL and one key; the gateway handles routing, authentication, failover, caching, spend limits and usage logging — so you integrate once instead of wiring each provider separately.
What is the difference between an AI gateway and an LLM gateway?
They are the same pattern. “LLM gateway” stresses text/chat models; “AI gateway” is the broader term covering image, video and audio behind the same endpoint. Kunavo is an AI gateway across all four modalities.
Do I need an AI gateway?
Yes, once you call more than one model or provider, want failover, need per-key spend control, or want one invoice. A single-model prototype can skip it, but a gateway makes adding the second provider a one-line change.
Is an AI gateway the same as a router like OpenRouter?
A router is one kind of gateway focused on picking a provider for a text request. A full AI gateway adds billing, multimodal endpoints, caching and spend controls. Kunavo is an inference gateway with all of these.