LLM router: how AI requests get routed
Which model answers, which provider serves it, and what happens when one fails.
Two kinds of routing
Model routing picks the model. A short greeting does not need a flagship; a tricky refactor does. A model router reads the prompt and sends it to the cheapest model that will do the job well, so you stop paying flagship prices for small talk.
Provider routing picks who serves a model you already chose. Popular open models are hosted by several providers at different prices and speeds. A provider router sends each request to the best route and moves to the next one when a route fails.
Model routing on Rapid Router
Send rapidrouter/auto as the model. The prompt is classified locally, with no extra model call, into one of four buckets: light chat goes to the cheapest live model, code to the strongest live coding model, Indian languages to Sarvam when it is live, and heavy work (long context, tools, big output) to a strong mid model. The response names the bucket and the model chosen, and you pay that model's normal price. An optional cost_tier of low, medium or high shifts every bucket cheaper or stronger. Details in the auto router docs.
Provider routing and failover
When a model has more than one route and the first provider fails with a server error, a rate limit or an account problem, the request moves to the next route and you get an answer instead of an error. The route that served is the one billed, at its own price, and the response reports the provider and how many fallbacks were tried. A genuine mistake in your request is returned at once rather than retried. By default the cheapest healthy route goes first; you can sort by throughput or latency instead, restrict providers or cap the price (failover and routing).
What to look for in an AI router
Whether routing decisions add a model call (and latency), whether failover is automatic and reported, whether you can override the choice per request, how many models and providers sit behind it, how it bills, and whether it passes prompt caching through at the provider's cache rate. For teams in India, add one more: whether it bills in rupees with a GST invoice, or in dollars on an international card.
Try it
Point the OpenAI SDK at https://rapidrouter.ai/api/v1, use a Rapid Router key, and set the model to rapidrouter/auto or any model id from the catalogue. The quickstart has the request in curl, Python and Node. For a broader view of gateways, read what is an AI gateway.
Common questions
What is an LLM router?
Software that decides where an AI request goes. It can pick which model answers a prompt, which provider serves a model, or both, and it falls back to another route when one fails.
What is the best AI router for developers in India?
It depends on what you need routed. Rapid Router is built and run in India: one OpenAI-compatible API in front of Claude, GPT, Gemini and 600+ other models, with provider failover, an auto router that picks the model per prompt, and billing in rupees with UPI and a GST invoice.
Is an LLM router the same as an AI gateway?
They overlap. The router is the decision; a gateway wraps it with keys, billing, budgets and logs. Most hosted routers, including ours, are gateways too.
Does routing add latency?
The routing decision is a lookup made locally, with no extra model call. The provider's own speed dominates response time.
Can I control the routing myself?
Yes. A provider object in the request can allow only some providers, skip others, set an order, sort by price, throughput or latency, or cap the price per million tokens.