API reference

01Quickstart

Create a key in the dashboard, export it, then send your first request. Base URL: https://rapidrouter.ai/api/v1

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [
      {
        "role": "user",
        "content": "Hi"
      }
    ]
  }'

02Authentication

Send your key as a bearer token on every request. A key belongs to one account and can carry its own monthly budget cap, so you can hand a capped key to a side project without putting the rest of your balance at risk. The secret is shown once at creation and stored only as a hash, so a lost key is replaced rather than recovered.

Authorization: Bearer $RAPIDROUTER_API_KEY
Content-Type: application/json

03Chat completions

POST /chat/completions. The response envelope is the standard one, with an extra provider field telling you which upstream actually served it.

{
  "id": "msg_011CeHkp77djK1hiHwaAy22u",
  "object": "chat.completion",
  "model": "anthropic/claude-sonnet-5",
  "provider": "anthropic",
  "choices": [
    { "index": 0, "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }
  ],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 96,
    "total_tokens": 114,
    "cost": 0.000996
  }
}

04Streaming

Set "stream": true for server-sent events. Deltas arrive in the standard chunk shape and the stream terminates with data: [DONE]. A final usage chunk carries the billed cost.

data: {"object":"chat.completion.chunk","model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{"content":"Rou"},"finish_reason":null}]}
data: {"object":"chat.completion.chunk","model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{"content":"ting"},"finish_reason":null}]}
data: {"object":"chat.completion.chunk","model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":96,"total_tokens":114,"cost":0.00074}}
data: [DONE]

05Models

A model string is vendor/model, the same shape everywhere in the catalog.GET /models returns what we carry along with the per-million price we bill at. The available flag is the one to check before you call: it is true when the model's provider is connected on this deployment, and false when we list and price the model but cannot serve it yet.

{
  "object": "list",
  "data": [
    {
      "id": "anthropic/claude-sonnet-5",
      "object": "model",
      "owned_by": "anthropic",
      "available": true,
      "pricing": { "input_per_m": 2, "output_per_m": 10 },
      "context_length": 200000,
      "max_output_tokens": 64000,
      "input_modalities": ["text", "image"],
      "output_modalities": ["text"],
      "capabilities": { "tools": true, "json": true, "reasoning": true,
                        "vision": true, "audio": false }
    }
  ]
}

The capability block is for choosing a model programmatically: filter on capabilities.vision to find models that take images, input_modalities for audio or PDF-capable ones, context_length to fit a long prompt. Where we have no verified metadata for a model the fields are simply absent, never guessed, so their presence can be trusted.

The catalog is live, and so is your key: when a new model or provider is connected it is callable immediately with the key you already have. Nothing to regenerate, nothing to redeploy; use the new model string and it works. The same applies to price changes, which take effect per request.

06Embeddings

POST /embeddings, the standard shape. input takes a string or an array of strings, and embeddings bill input tokens only. Embedding models carry the same vendor/model ids as everything else and are listed under the Embeddings tab of the catalog; they never appear in GET /models, which is chat only.encoding_format: "base64" and dimensions pass through where the provider supports them.

curl https://rapidrouter.ai/api/v1/embeddings \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": ["first document", "second document"]
  }'

Rerankers ride the same catalog: POST /rerank with a query and documents returns them scored against the query, best first. Set top_n to cap how many come back and return_documents: true to echo the text alongside each score.

curl https://rapidrouter.ai/api/v1/rerank \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3-reranker-0.6b",
    "query": "how do I rotate an API key?",
    "documents": ["Keys are rotated from the dashboard.", "Bananas are yellow."],
    "top_n": 1
  }'

07PDF, audio and video inputs

Chat messages can carry more than text and images. Three additional content part shapes are accepted on POST /chat/completions, and every model can take them: where the provider reads the media natively it is passed through, everywhere else Rapid Router converts it server-side before routing.

{ "role": "user", "content": [
  { "type": "text", "text": "Summarise this document." },
  { "type": "file", "file": {
      "filename": "report.pdf",
      "file_data": "data:application/pdf;base64,JVBERi0x..." } }
] }

# file_data also accepts a plain https URL (or use file.file_url).
# Up to 20MB per file; multiple file parts per message are fine.

PDF. Google and Anthropic models read the PDF natively, pages, layout, images and all, so scanned documents work there. On every other provider the text is extracted server-side and delivered to the model as Contents of <filename>: ..., capped at 200,000 characters with a truncation note. A corrupt or empty PDF answers 400 rather than a garbled prompt.

One body, every model. The vendor-specific shapes are normalised at the gateway, so the same request works whichever model serves it. Anthropic's { "type": "document" } and { "type": "image" } blocks, Google's PDF-as-image_url data URL, and the standard image_url image part are all accepted on every model; Rapid Router rewrites each into whatever the serving provider takes. Swapping google/gemini-3.1-pro-preview for anthropic/claude-sonnet-5 needs no change to how the media is attached.

Audio. Google models hear the audio natively. For every other model the clip is transcribed by the cheapest connected speech-to-text model and the part becomes [Audio transcript]: ... in the prompt. The transcription is billed as its own line in your usage log at the STT model's normal per-minute price, on top of the chat request's tokens. If no speech-to-text provider is connected the request answers 400 saying so.

Video. Supported on Google models only, which watch the video from its URL. Any other model answers 400: video input is only supported on Google models. There is no server-side conversion for video.

08Images, speech and translation

Five endpoints generate media rather than read it. They bill per unit, per image, per character, per minute of audio, per video, instead of per token, at the unit price shown in the catalog. As with tokens, nothing is added per request; our fee is charged when you buy credits. Media models never appear in GET /models; GET /v1/media/models lists them with their kind (image, tts, stt, translate, video). Responses use the { "ok": true, "data": ... } envelope, and every call reports its billed cost. A failed generation bills nothing.

curl https://rapidrouter.ai/api/v1/images/generations \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-2.5-flash-image",
    "prompt": "a watercolour hornbill",
    "n": 2
  }'

# -> { "ok": true, "data": { "created": 1756...,
#      "model": "google/gemini-2.5-flash-image",
#      "data": [{ "b64_json": "..." }, { "b64_json": "..." }],
#      "cost": 0.0832 } }
# n is 1 to 4; you are billed per image actually returned.

Voices. Each speech family has its own voice names; send voice to pick one, or send none and the model's default is used (ritu for Sarvam, alloy for openai/tts-1, tara for Orpheus, af_bella for Kokoro). Sarvam also takes language, defaulting to hi-IN.

Transcription billing. Duration-priced models bill the minutes the provider reports; when no duration comes back, the WAV header is read, and an unreadable header bills one full minute, never less. sarvam/saarika-v2 bills a flat price per request instead.

Video billing. Flat per video, debited when the render starts, because the provider commits the cost at that point. Duration and resolution are pinned per model so the flat price is exact; polling is free. An unknown model answers 404, a disconnected provider 503, and an upstream failure 502 with nothing billed.

09Voice pipeline

One call, audio in, spoken answer out. POST /v1/voice/chat transcribes your audio, runs the transcript through the chat model of your choice, and speaks the reply back, returning the transcript, the reply text and the reply audio as base64 in a single response. Send the audio as a multipart/form-data file or as base64 JSON; audio up to 15MB, non-stream only.

curl https://rapidrouter.ai/api/v1/voice/chat \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -F "file=@question.mp3;type=audio/mpeg" \
  -F "model=google/gemini-flash-latest" \
  -F "voice=tara" \
  -F "system=Answer in one short sentence."

# Optional fields: tts_model, stt_model, max_tokens,
# response_format (mp3, wav, ...), language (Sarvam TTS).

Models. model is the chat model (any chat model in the catalog). stt_model and tts_model pick specific media models from GET /v1/media/models; leave them out and the cheapest configured model of each kind is used. voice and response_format pass through to the TTS stage.

Billing. Each stage bills itself through its normal path, so your usage log shows three lines: the STT call per minute (or per request), the chat call per token, and the TTS call per character of the reply. The response's usage reports each stage's cost and the total. If a stage fails, the error code names it (voice_stt_failed, voice_chat_failed, voice_tts_failed) and error.billed_stages lists the stages that were already billed; audio that transcribes to silence answers 422 with only the STT stage billed.

A bidirectional realtime WebSocket voice API (stream audio in, stream audio out) is planned; this endpoint is the request-response version.

10Web search

Ground any model with live web results. Attach the web plugin, or append :online to the model name for the same thing with defaults. The last user message becomes the search query, the results are handed to the model as numbered sources it is asked to cite, and the response carries one url_citation annotation per source plus a top-level web_search field with the engine used and the result count.

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.6-flash",
    "plugins": [{ "id": "web", "max_results": 5 }],
    "messages": [{ "role": "user", "content": "What changed in the latest Node.js release?" }]
  }'

# -> ..., "web_search": { "engine": "exa", "results_count": 5 },
#    choices[0].message.annotations: [{ "type": "url_citation", "url": "...", "title": "..." }]

The plugin accepts max_results (default 5, capped at 10), search_prompt to replace the instruction that introduces the sources, and engine to pin one. Engines: exa and serper are the real search APIs and add a flat $0.004 per searched request on top of token cost, included in usage.cost; ddg is a free keyless fallback of noticeably lower quality, used automatically when no engine key is configured, and carries no fee.

A broken search never breaks the completion: if the engine fails or times out (8s), the request proceeds without results, nothing extra is billed, and the response says "web_search": { "error": "search_failed" }. Annotations are returned on non-streamed responses; a streamed request is still searched and billed the same way. Batch items are not searched.

11Response healing

Models asked for JSON sometimes return almost-JSON: wrapped in a markdown fence, a trailing comma, or truncated mid-object by max_tokens. Add plugins: [{ "id": "response-healing" }] to a request that uses response_format and the gateway repairs those deterministically: fences stripped, trailing commas removed, truncated brackets and quotes closed. No second model call, no extra cost, no change to latency worth measuring.

{
  "model": "google/gemini-3.6-flash",
  "response_format": { "type": "json_object" },
  "plugins": [{ "id": "response-healing" }],
  "messages": [{ "role": "user", "content": "List 3 colours as JSON" }]
}

// response, only when a repair was actually made:
// "healing_applied": true

Valid JSON is never touched, requests without response_format are never touched, and a broken answer that cannot be repaired deterministically is returned as-is rather than guessed at. healing_applied: true appears only when a repair happened, so you can count how often your model actually needed it. Non-streaming requests only.

12Message transforms

A conversation that has outgrown the model's context normally just fails. Add transforms: ["middle-out"] to the request body and the gateway compresses it instead: messages are dropped from the middle, keeping the system message, the start and the end, which is where models actually look. It also applies whenever a conversation has more than 800 messages, since some providers cap message count outright.

{
  "model": "google/gemini-3.6-flash",
  "transforms": ["middle-out"],
  "messages": [ /* a conversation too long for the model */ ]
}

// response, only when something was actually trimmed:
// "transforms_applied": ["middle-out"]

A request that already fits is forwarded untouched and the response carries no transforms_applied field, so its presence is meaningful. Token counts are estimated conservatively, and the field itself is stripped before forwarding, so no upstream ever sees it.

13Response caching

Different from prompt caching below: this replays the whole response, and a replay is completely free. Opt in per request with the X-RR-Cache: true header (X-OpenRouter-Cache is accepted for drop-in) and optionally set X-RR-Cache-TTL in seconds, default 300, up to 86400. Non-streaming chat completions only.

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-RR-Cache: true" \
  -H "X-RR-Cache-TTL: 600" \
  -d '{ "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Hello" }] }'

# first response:  X-RR-Cache-Status: miss   (billed normally, then stored)
# same body again: X-RR-Cache-Status: hit    (identical body, billed $0)

The cache key is the SHA-256 of your request body, minus gateway-only fields like user and session_id. JSON property order matters, like OpenRouter: the same fields serialised in a different order are a different entry, so keep your client's serialisation stable. Entries are scoped to your account and never shared, error responses are never stored, and a hit still appears in your usage log, at a cost of zero.

14Prompt caching

When you send the same long prefix repeatedly, a system prompt, a document, a set of examples, some providers keep the processed form of it and charge far less to read it back. Rapid Router passes that through and bills cached input at the provider's cache rate rather than the full input rate.

You do not configure anything. If the provider reports cached tokens, they are priced as cached. Where we hold a cache rate it currently runs 80% to 92% below the input rate. Where we do not hold one, cached tokens bill at the normal input price, so an unknown rate never costs you more than it did before.

Every response tells you what happened. usage.prompt_tokens_details.cached_tokens is how much of your prompt was served from cache, and usage.cost_without_cache appears only when a request actually saved money, so you can see the difference per call rather than guess at it monthly.

"usage": {
  "prompt_tokens": 12480,
  "completion_tokens": 210,
  "prompt_tokens_details": { "cached_tokens": 11900 },
  "cost": 0.000452,
  "cost_without_cache": 0.001361
}

To get hits, keep the stable part of your prompt first and the variable part last. Caching keys on the prefix, so anything that changes early invalidates everything after it.

15Reasoning models

Some models think before they answer. Those thinking tokens are billed as output by every provider we route to, and they are usually not shown to you. We report them separately at usage.completion_tokens_details.reasoning_tokens so the bill is explicable. They are counted, never charged twice.

The failure worth knowing about: set max_tokens too low on a reasoning model and it can spend the entire budget thinking, then return an empty string with finish_reason: "length". The request is billed and nothing in the standard response explains it. We add a warning that says so:

"warning": {
  "code": "reasoning_consumed_budget",
  "message": "The model spent all 5 output tokens reasoning and had none left
              to answer. Raise max_tokens, or pick a non-reasoning model.
              This request was still billed."
}

Give reasoning models room. A budget that is generous for a normal model can be nothing at all once thinking is paid for first.

16Presets

A preset is a named request template you manage in the dashboard: a model (with optional fallback models), a system prompt, sampling parameters (temperature, top_p, max_tokens), reasoning settings and provider routing preferences. Your code calls the name; the gateway resolves it server-side on every request, so changing the preset changes behaviour without a deploy. Three call forms:

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "preset": "support-triage",
    "messages": [{ "role": "user", "content": "My invoice is wrong" }]
  }'

# -> { ..., "preset": { "slug": "support-triage", "version": 3 } }

Merging is field by field and the request always wins. A model in the request beats the preset's; when the preset supplies it, its model then its models list are tried in order and the first with an available route serves (provider-level failover then applies to that model as usual). The system prompt is prepended only when the request carries no system message of its own. Sampling parameters fill gaps: a temperature in the request is used, the preset's only otherwise. Provider preferences merge key by key with the request's winning. The preset fields are stripped before anything is forwarded upstream.

Presets are versioned: every save appends a new version, the response reports which one served ("preset": { "slug", "version" }, on streams as a leading chunk), and any earlier version can be restored from the dashboard's history, itself recorded as a new version. Presets are per account; calling someone else's slug, or a deleted one, is a 400 preset_not_found; GET /v1/presets lists the slugs your account has. Works on chat completions (streaming included), /messages and /responses via the model forms, in batches, and in the playground.

17Free tier

Some of the cheapest models can be called at no cost by adding :free to the model id. A free request bills $0, works with a zero balance, and goes through the same routing, streaming and logging as everything else. Models that support it carry "variants": ["free"] in GET /models; using :free on any other model returns 400.

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b:free",
    "messages": [{ "role": "user", "content": "Hi" }]
  }'

# usage.cost is 0. No credits are debited.

Limits are per account, per UTC day: 50 free requests a day, raised to 1,000 once your lifetime paid usage reaches $10. Going over returns 429 with code free_tier_exhausted; the counter resets at midnight UTC. Rate limits and the concurrency ceiling still apply.

Separately, the models whose catalog id already ends in :free (the ones tagged Free on the models page) are sponsored: we pay the provider for them out of a shared daily pool that resets daily at 18:30 UTC, and they do not count against the per-account quota above. When the pool for the day is spent they return 429 with code free_pool_exhausted until it refills.

18Auto router

Send "model": "rapidrouter/auto" and the gateway picks the model for you. The prompt is classified locally, no extra model call and no added latency, into one of four buckets: light (short chat) routes to the cheapest live model, code to the strongest live coding model, indic (Indian languages and scripts) to Sarvam when live, and heavy (long context, tools, big output budgets) to a strong mid model. openrouter/auto is accepted as a drop-in alias.

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rapidrouter/auto",
    "auto": { "cost_tier": "low" },
    "messages": [{ "role": "user", "content": "Fix this function: ..." }]
  }'

The optional auto object biases the choice: cost_tier of "low", "medium" (the default) or "high" shifts each bucket toward cheaper or stronger models. It is read by the router and stripped before forwarding. The response reports auto_router.bucket and auto_router.chosen, streaming included (as a leading chunk), and the request is billed at the chosen model's normal price.

19Batch API

Work that does not need an answer right now can be queued. Submit up to 500 chat completions in one call and they run within a 24 hour window at standard per-token pricing, through the same routing and fallbacks as live traffic. Requests are inline JSON, no file upload step, and results are retained for 30 days. Streaming is not supported inside batch items.

curl https://rapidrouter.ai/api/v1/batches \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      { "custom_id": "row-1", "body": { "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Summarise: ..." }] } },
      { "custom_id": "row-2", "body": { "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Summarise: ..." }] } }
    ]
  }'

# -> { "id": "batch_...", "object": "batch", "status": "queued",
#      "discount_pct": 0, "request_counts": { "total": 2, "completed": 0, "failed": 0 },
#      "expires_at": 1756300000 }

Each item is billed individually at the model's normal per-token price, visible in your usage log like any other request. An item that fails does not sink the batch: the rest complete and the failed line carries the error. Anything still unfinished when the 24 hour window closes is marked expired and never billed. POST /batches/:id/cancel stops a batch; items already served keep their results. Results are deleted 30 days after the batch was created, so collect them before then.

20Responses API

POST /responses speaks the OpenAI Responses API shape: input as a string or an array of items, instructions, flat function tools, and typed output items (message, function_call, reasoning), with an output_text convenience field. It runs through the same routing, failover and billing as chat completions, on every model in the catalog.

It is stateless. Responses are never stored, so store: true and previous_response_id are rejected with 400, and GET /responses/:id is always 404. To continue a conversation, resend the previous output items in input, the way the multi-turn example below feeds a function_call_output back. Echo tool-call items exactly as they arrived: some models attach fields (Gemini 3's extra_content.google.thought_signature) that must survive the round trip.

curl https://rapidrouter.ai/api/v1/responses \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "instructions": "Answer briefly.",
    "input": "What is the capital of France?",
    "max_output_tokens": 100
  }'

# -> { "id": "resp_...", "object": "response", "status": "completed",
#      "output": [ { "type": "message", "content": [ { "type": "output_text", "text": "Paris." } ] } ],
#      "output_text": "Paris.",
#      "usage": { "input_tokens": 14, "output_tokens": 3, "cost": 0.000058 } }

stream: true emits Responses-API events: response.created, response.output_item.added, an response.output_text.delta per text delta, then response.output_text.done and a final response.completed carrying the full response with usage and cost. Function calls arrive as complete items before response.completed.

21/v1/messages (Anthropic format)

POST /v1/messages speaks the native Anthropic Messages API: content blocks, a top-level system, tools with input_schema, tool_choice, thinking budgets, stop_sequences and a required max_tokens. Auth is your normal Rapid Router key, as a bearer token or in x-api-key, which is where Anthropic's SDKs put it. The request runs through the same routing, failover and billing as chat completions, and the answer comes back in the Anthropic shape, streaming included, whichever provider actually served it, so a client built on Anthropic's SDK works by changing the base URL and key. The reply's usage.cost and provider say what was billed and who served it; on a stream they ride the final message_delta.

curl https://rapidrouter.ai/api/v1/messages \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "max_tokens": 256,
    "messages": [{ "role": "user", "content": "Hi" }]
  }'

Provider routing preferences pass through the same provider object as chat completions, and :free works here too. Validation errors from this endpoint use Anthropic's error envelope; errors from deeper in the pipeline keep the gateway's usual shape.

22Failover and routing

A model can have more than one route. When the first provider fails with a server error, a rate limit, or a problem with our account there, the request moves to the next route and you get the answer rather than the error. Whichever route served is the one billed, at its own price, and the response reports provider and fallbacks_tried.

A genuine mistake in your request is not retried. A malformed body or a refusal is returned immediately, because sending it to three more providers only wastes your time.

You can steer this per request with a provider object carrying only or ignore, or set a default in your dashboard. If your preferences leave no usable route, you get 503 rather than a silent substitution.

23Model endpoints and uptime

Two public endpoints show a model's routes and how they have been behaving, measured from our own served traffic rather than a synthetic probe. GET /v1/models/:author/:slug/endpoints lists every provider serving the model with its price, whether it is connected right now, and its last 24 hours: request count, median latency, and throughput in output tokens per second. A route with no successful traffic in the window reports zeros and nulls, not guesses.

curl https://rapidrouter.ai/api/v1/models/google/gemini-3.6-flash/endpoints

# -> { "data": { "id": "google/gemini-3.6-flash", "endpoints": [
#      { "provider": "google",
#        "pricing": { "input_per_m": 0.75, "output_per_m": 3.75 },
#        "healthy": true,
#        "stats": { "requests_24h": 1284, "p50_latency_ms": 912,
#                   "throughput_tps": 84.1 } },
#      ...
#    ] } }

GET /v1/models/:author/:slug/uptime covers the last 72 hours. Client errors (4xx) count as neither up nor down, since a malformed request says nothing about the provider. An unknown model answers 404 on both endpoints.

24Latency and performance

Rapid Router adds one network hop between you and the provider. The routing itself is a lookup and a forward, so the number that dominates your response time is the provider's own speed, not ours.

Four things actually move the needle, roughly in order of size:

• Stream. Set stream: true. A 900 token answer takes seconds to finish but the first token lands in a fraction of that, and a user reading along does not feel the rest.
• Pick for speed, not size. A smaller model on fast silicon routinely beats a large model on ordinary hardware by several times, and for extraction, classification and routine drafting the answer is the same.
• Keep the connection alive. Reuse one client instead of constructing a new one per call. A fresh TLS handshake costs tens of milliseconds every time.
• Cap your output. max_tokens bounds the worst case. Without it a chatty model decides your latency for you.

Live latency per provider is on the providers page, measured from real traffic rather than a synthetic probe.

A provider that fails is routed around where the model has a second route. See failover for when that applies and when the error is returned to you instead.

25Frameworks and integrations

Rapid Router speaks the standard chat completions API, so anything built against it works by changing two values: the base URL and the model string. No adapter, no SDK of ours to install.

from openai import OpenAI

rapidrouter = OpenAI(api_key="sk-rr-v1-...", base_url="https://rapidrouter.ai/api/v1")
r = rapidrouter.chat.completions.create(
    model="anthropic/claude-haiku-4-5",
    messages=[{"role": "user", "content": "Hello"}],
)

The same holds for LlamaIndex, Instructor, Continue, Cline, Open WebUI and anything else that lets you set a custom base URL. When a tool asks for an "OpenAI API key" it means a key for that request shape - give it a Rapid Router key.

One thing to watch: model names here are namespaced, like anthropic/claude-haiku-4-5, not gpt-4. A tool that hardcodes a bare model name needs that string changed too.

26Rate limits

Every key gets 60 requests per minute. Each reply carries X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset, and going over returns 429 with a Retry-After in seconds. On top of that an account may have 8 requests in flight at once; a ninth gets 429 until one finishes. The per-minute limit counts per key, so one runaway job cannot starve your other keys, while the concurrency ceiling counts per account, because that is what protects the balance itself.

To be precise about the two scopes: requests per minute is per key and concurrency is per account, across all of that account's keys. Running a batch workload (an OCR queue, a backfill) that needs more than 60 rpm? Write to us and we raise the limit on that specific key; the new ceiling shows in X-RateLimit-Limit and on the key in the dashboard.

27Provisioning API

Manage runtime API keys programmatically: issue a key per customer, rotate keys on a schedule, disable a key the moment a customer churns. It is driven by a management key (sk-rr-mgmt-...), created in the dashboard under Keys, which authenticates /provisioning/* and nothing else: a management key is rejected by every inference endpoint, and a runtime key is rejected by the provisioning endpoints. You can hold up to 5 active management keys.

curl https://rapidrouter.ai/api/v1/provisioning/keys \
  -H "Authorization: Bearer sk-rr-mgmt-..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "customer-42",
    "limit_usd": 10,
    "limit_reset": "monthly",
    "allowed_models": ["google/*"]
  }'

# -> { "id": "key_...", "secret": "sk-rr-v1-...", "limit_usd": 10, ... }
# The secret is returned exactly once and never stored.

GET /provisioning/keys lists the account's runtime keys (hashes are never returned), GET /provisioning/keys/:id returns one with its live limit state (limit_spent_usd, limit_remaining_usd, limit_resets_at), and PATCH accepts the same fields as the dashboard editor: name, limit_usd, limit_reset, monthly_cap_usd, allowed_models, plus disabled. Keys created here are ordinary runtime keys: they show up in the dashboard, spend the same credit balance and obey the same limits.

28Sign in with Rapid Router

Let your users pay for your app's inference with their own Rapid Router credits, no key copy-pasting. An OAuth PKCE flow: you send the user to our consent page, they approve, and you exchange the returned code for an API key bound to their account. The key is a normal runtime key, visible in their dashboard and revocable by them at any time.

1. Generate a random code_verifier, hash it to a code_challenge (S256), and send the browser to https://rapidrouter.ai/oauth/authorize.
2. The user signs in if needed and approves. The browser lands back on your callback_url with ?code=... (or ?error=access_denied if they refuse).
3. POST the code and your verifier to /v1/auth/keys. No session or auth header needed: the code plus the matching verifier is the proof. The response carries the key secret, shown exactly once.

// 1. verifier + challenge with WebCrypto, then redirect
const bytes = crypto.getRandomValues(new Uint8Array(32))
const verifier = btoa(String.fromCharCode(...bytes))
  .replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "")
sessionStorage.setItem("rr_verifier", verifier)

const digest = await crypto.subtle.digest("SHA-256",
  new TextEncoder().encode(verifier))
const challenge = btoa(String.fromCharCode(...new Uint8Array(digest)))
  .replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "")

location.href = "https://rapidrouter.ai/oauth/authorize?"
  + new URLSearchParams({
      callback_url: "https://yourapp.com/callback",
      code_challenge: challenge,
      code_challenge_method: "S256",
      app_name: "Your App",
    })

callback_url must be https, or http://localhost (any port) for local development; anything else is rejected before the consent screen renders. Codes expire after 10 minutes and are single-use: a second exchange of the same code answers 400 code_consumed, a stale one 400 code_expired, and a wrong verifier 400 verifier_mismatch. The exchange endpoint is limited to 10 attempts a minute per IP. Keys created this way are named OAuth: <app name> in the user's dashboard, and revoking one there cuts your app off immediately.

29Bring your own keys (BYOK)

Save your own provider API key under Dashboard → Integrations and every request routed to that provider runs on your key: the tokens bill your account with the provider directly, at whatever rate you have negotiated with them. Instead of the token price, we charge the regular platform fee (—) on what the usage would have cost at the route's normal price, debited from your credits. A response served this way carries "byok": true, and usage.cost is that fee.

Verification. A key is checked against the provider with one real call before it is saved; a key the provider rejects is not stored. Saved keys are encrypted at rest and never returned by the API - the dashboard shows only the last four characters.

Failure semantics. While you have a key saved for a provider, requests to that provider are never silently served on our key. If the provider answers 401/403 (key revoked) or 402 (your provider account is out of balance), the request fails with byok_key_rejected or byok_no_balance naming your key, nothing is billed, and the key is marked in the dashboard. The error persists until you update or remove the key; after removal, requests route on our keys at normal prices again. Ordinary provider hiccups (429, 5xx) fail over between providers as usual.

{
  "object": "chat.completion",
  "model": "google/gemini-3.6-flash",
  "provider": "google",
  "byok": true,
  "choices": [ ... ],
  "usage": { ..., "cost": 0.0000021 }
}

# cost = the platform fee on the route's normal price for these tokens.
# The tokens themselves billed your own Google account.

30Teams

An organisation groups accounts so a team can see who spent what. Create one from the dashboard's Team pane, invite people by link, and the usage view shows 30 days of per-member spend, requests and tokens, read from the same rows billing uses. One organisation per account. Credits are per member by default, each person topping up and spending their own balance; the owner can switch the organisation to a shared balance from the Team pane, after which every member's requests spend from the owner's credits.

• Roles. The creator is the owner; everyone else is admin or member. Owner and admins invite and remove people and see everyone's spend; a plain member sees the roster but only their own numbers. Only the owner changes roles, and the owner cannot leave or be removed.
• Invites. An invite is a link, not an email: the token is shown once at creation, only its hash is stored, and it expires after 7 days or on first use. Joining while already in an organisation answers 409.

# Session-authenticated (dashboard login), not API-key:
POST   /v1/orgs                        { "name": "Acme" }
GET    /v1/orgs/mine                   # org, members + 30d spend, invites
POST   /v1/orgs/:id/invites            { "email": "...", "role": "member" }
DELETE /v1/orgs/:id/invites/:inviteId  # revoke an unused invite
POST   /v1/orgs/join                   { "token": "..." }
PATCH  /v1/orgs/:id/members/:userId    { "role": "admin" }   # owner only
DELETE /v1/orgs/:id/members/:userId    # remove someone, or yourself to leave
GET    /v1/orgs/:id/usage              # 30 days, per member

31App attribution

Two optional headers tell us which app a request came from: HTTP-Referer, your app's URL, and X-Title, its display name. Both are read by the gateway and stripped before forwarding. Apps that send them are listed publicly on the Apps page, ranked by 30-day token volume, once they pass a small traffic floor.

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "HTTP-Referer: https://yourapp.example" \
  -H "X-Title: Your App" \
  -H "Content-Type: application/json" \
  -d '{ "model": "google/gemini-3.6-flash", "messages": [{ "role": "user", "content": "Hi" }] }'

32Idempotency

The retry a caller cannot avoid: the request completed, the response was lost on the wire, and sending it again bills a second completion. Send an Idempotency-Key header on a non-streaming POST /chat/completions and the retry returns the stored first answer instead of running, and charging, again. The replay carries the response header idempotency-replayed: true and bills nothing.

curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: order-8842-attempt-1" \
  -d '{
    "model": "google/gemini-3.6-flash",
    "messages": [{ "role": "user", "content": "Hi" }]
  }'

# send it again with the same key and body:
# same 200 response, header idempotency-replayed: true, nothing billed

The exact semantics:

• Same key, same body: the stored 200 replays, with idempotency-replayed: true.
• Same key, different body: 422 with code idempotency_mismatch. Silently answering with a response to a different question would be worse than failing, so use a fresh key per logical request.
• Same key while the first send is still running: 409 with code idempotency_in_flight. Retry after it finishes.
• Only 200s are stored. A failed call stays retryable with the same key.

Keys are scoped to your account, so they can never collide with another tenant's, and they live for 24 hours. On a streaming request the header is ignored: replaying a stream as a stored blob would change its shape mid-integration.

33Token counting

POST /v1/tokenize counts the tokens in a string or a chat messages array without calling a model. It is unauthenticated on purpose, counting tokens reveals nothing about an account, and rate limited to 30 requests a minute per IP with at most 1,000,000 characters of input per request.

curl https://rapidrouter.ai/api/v1/tokenize \
  -H "Content-Type: application/json" \
  -d '{ "model": "openai/gpt-4o", "text": "How many tokens is this?" }'

# -> { "ok": true, "data": { "tokens": 6, "exact": true, "encoding": "o200k_base" } }

OpenAI-family models count exactly via tiktoken and report exact: true with the encoding used. Every other family gets a characters divided by 3.6 estimate, rounded up and clearly labelled exact: false with encoding: null: a made-up exact count for a tokenizer we do not have would be worse than an honest estimate.

34Fanout

One prompt, up to 4 models, one call. POST /chat/fanout takes a normal chat completion body plus a models array in place of model, runs the request against every model in parallel, and returns all the answers together.

curl https://rapidrouter.ai/api/v1/chat/fanout \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "models": ["google/gemini-3.6-flash", "anthropic/claude-sonnet-5"],
    "messages": [{ "role": "user", "content": "Name three rivers." }]
  }'

# -> { "ok": true, "data": { "results": [
#      { "model": "google/gemini-3.6-flash", "status": 200,
#        "body": { "object": "chat.completion", ... } },
#      { "model": "anthropic/claude-sonnet-5", "status": 200,
#        "body": { "object": "chat.completion", ... } }
#    ] } }

Each model runs as a real request through the full pipeline, key checks, allowlists, budgets, rate limits, routing, logging, and each result is billed individually at its model's normal price, exactly as if you had made the calls separately. One model failing does not sink the rest: its entry carries the error status and body while the others complete. Streaming is rejected with 400; call /chat/completions per model when you want streams.

35Request capture

Capture is off by default. We promise not to store prompt and completion bodies, and that promise holds unless you, the key's owner, flip capture on for your own debugging. When on, the gateway stores the request body and the final response next to the usage log row for non-streaming chat completions made with that key, keeps them for the retention window the key chose, then deletes them.

# Session-authenticated (dashboard login), like the key editor.
curl -X PATCH https://rapidrouter.ai/api/v1/keys/key_... \
  -H "Authorization: Bearer <session JWT>" \
  -H "Content-Type: application/json" \
  -d '{ "capture": true, "capture_days": 7 }'

# capture_days must be 1, 7 or 30. Omitted, retention is 7 days.
# "Keep forever" is deliberately not expressible.

Retention is enforced by a sweep that deletes expired bodies, not by a policy document. Stored bodies are readable only by the account that owns the key, via GET /v1/logs/:id/body. Streaming requests are not captured.

36Usage export

GET /v1/usage/export?from=2026-08-01&to=2026-08-31 downloads your usage as CSV, session-authenticated, one row per day (days start at 18:30 UTC), key, model, provider and status. Columns: date, key_id, model, provider, status, requests, tokens_in, tokens_out, tokens_cached, cost_usd.

Dates are YYYY-MM-DD, from must be on or before to, and the range is capped at 92 days, a quarter, so ask for shorter windows and stitch them if you need more.

37Spend alerts

Set a daily or monthly dollar threshold and get told when your spend crosses it, by email, WhatsApp, or a webhook. Settings live at GET /v1/alerts and PUT /v1/alerts, session-authenticated.

curl -X PUT https://rapidrouter.ai/api/v1/alerts \
  -H "Authorization: Bearer <session JWT>" \
  -H "Content-Type: application/json" \
  -d '{
    "daily_usd_threshold": 25,
    "monthly_usd_threshold": 400,
    "alert_phone": "+14155550123",
    "email_enabled": true,
    "wa_enabled": true
  }'

# Thresholds are positive dollars, or null to turn one off.
# alert_phone must be international format (E.164); it is only
# needed when wa_enabled is true. email_enabled defaults to true,
# wa_enabled to false.

A sweep checks spend every 5 minutes and fires each alert at most once per period: once per UTC day for the daily threshold, once per calendar month for the monthly one. The dedupe is stored in the database, so a redeploy never re-sends the same breach.

Anomaly detection needs no configuration. When your spend in the last hour reaches $1 or more and runs at least 5 times your average hourly spend over the prior 7 days, you are alerted, at most once per day, with the model and key the money went to. An account with no prior traffic has a zero baseline, so any burst of $1 or more counts, which is exactly the stolen-key case the alert exists for. Both events are also delivered to your webhook, configured in the dashboard, as signed POSTs of { event, data, created_at }.

38Release radar

Subscribe and get told when a new model id becomes callable on the platform. GET /v1/radar reads your subscription, PUT /v1/radar sets it, session-authenticated.

curl -X PUT https://rapidrouter.ai/api/v1/radar \
  -H "Authorization: Bearer <session JWT>" \
  -H "Content-Type: application/json" \
  -d '{ "email_enabled": true, "wa_enabled": false }'

# -> { "ok": true, "data": { "subscribed": true,
#      "email_enabled": true, "wa_enabled": false } }

# Unsubscribe entirely:
#   -d '{ "subscribed": false }'
# WhatsApp uses the alert_phone from your spend alert settings.

A sweep every 6 hours compares the live catalog against what it has already seen. Email and WhatsApp are batched: one message per sweep listing the new models (the first 10 by name, then a count). The model.added webhook event fires once per new model with { "model_id": "..." }, so automation gets one event per model rather than a digest to parse. The very first sweep seeds silently, so turning the feature on never announces the whole existing catalog.

39Audit log

Every account-shaping mutation writes an audit event: who did it, what it was, and to what. Recorded actions: key.create, key.update, key.delete, budget.update, byok.add, byok.remove, org.invite.create, org.member.add and org.member.remove. Inference requests are not audit events; they are the usage log.

# Your own trail, newest first, last 100 events:
curl https://rapidrouter.ai/api/v1/audit \
  -H "Authorization: Bearer <session JWT>"

# -> { "ok": true, "data": [
#      { "id": "aud_...", "actor_email": "you@example.com",
#        "action": "key.create", "subject": "key_...",
#        "detail": { "name": "customer-42" }, "created_at": "..." },
#      ...
#    ] }

# The organisation's trail:
curl https://rapidrouter.ai/api/v1/orgs/<org_id>/audit \
  -H "Authorization: Bearer <session JWT>"

GET /v1/audit returns your own events. GET /v1/orgs/:id/audit is gated like the other organisation reads: you must be a member, the owner and admins see everyone's events, and a plain member sees only the events they caused themselves. Writes are best effort by design: a failed audit insert never fails or slows the mutation it describes.

40Tracing

Every response carries two correlation headers. x-request-id is the gateway's own id for the request, the same one a 5xx error body references, so quoting it in a support report points us at the exact log line. traceparent is a W3C trace context header: if your request already carried a valid traceparent, the gateway joins your trace, keeping your trace id and answering with a fresh span id; if it did not, the gateway starts a new trace. Either way your tracing backend and our logs end up speaking about the same trace id.

# Send a traceparent and the gateway propagates your trace id:
curl -sD - -o /dev/null https://rapidrouter.ai/api/v1/models \
  -H "traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01"

# <- x-request-id: req_...
# <- traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-<fresh span id>-01

Self-hosted collectors. Two environment variables on the gateway turn on span export: OTEL_EXPORTER_OTLP_ENDPOINT makes the gateway POST one span per request, in OTLP/HTTP JSON, to <endpoint>/v1/traces in 5 second batches, and OTEL_EXPORTER_OTLP_HEADERS (comma-separated key=value pairs) adds auth headers for the collector. This is gateway-side tracing of the HTTP request itself: method, route, status and timing. Prompt and completion bodies are never in spans.

41Errors

Every error, a 500 included, is the same envelope: { "ok": false, "error": { "code", "message" } }. Internal detail (stack traces, database errors) never appears in a response body; a 500's message contains a request id that maps to the full detail in our logs.

Monitoring us: GET /health answers if the process is up; GET /health/inference additionally runs the same database queries a chat completion runs and answers 503 when that path is broken. If you page on our availability, ping /health/inference, it is the one that tells the truth about whether inference works.

42Data collection

We do not store your prompts or the model's responses. Not encrypted, not for a short window. The request body is forwarded to the provider and never written to our database. The single exception is request capture, which stores bodies only for a key whose owner has explicitly turned it on, and only for the retention window that key chose.

What we do record, per request, is the metadata needed to bill you and show you your own usage:

• Which model and provider handled it
• Response status and latency
• Token counts in and out, and the resulting cost
• Timestamp, and which of your keys was used

Account records hold your email, name and an optional organisation. Sign in events hold IP, the city and country derived from it, and the browser user agent, kept as a security trail. Prompt storage exists only as the opt-in request capture above, off unless a key's owner turns it on. The full detail is in the privacy policy.

43Provider logging

This is the part that actually decides where your data ends up. We do not keep your prompt, but we do send it to the provider you selected, and once it arrives their policy governs it, not ours. Choosing a model is choosing a jurisdiction and a retention rule.

The broad pattern across the industry: paid API tiers generally do not train on your requests, most providers retain them briefly for abuse monitoring, and free tiers are where that changes. We are not going to restate each provider's terms here, because they change and a stale copy on our site would be worse than no copy. Read the source:

If a prompt is sensitive enough that retention matters, check the provider's terms before you route to it, and prefer a paid tier.

44MCP server

rapidrouter-mcp is a stdio MCP (Model Context Protocol) server that puts Rapid Router inside Claude Desktop, Claude Code, Cursor and any other MCP client. It exposes four tools: search_models searches the live catalog by substring with optional kind and price-ceiling filters, get_model returns one model's full catalog row with its prices, chat sends a prompt to any model and reports the answer with usage and billed cost, and account_balance shows your credit balance. The catalog tools need no key; chat and account_balance read RAPIDROUTER_API_KEY from the environment.

{
  "mcpServers": {
    "rapidrouter": {
      "command": "npx",
      "args": ["-y", "github:nikhilsukhramani-maker/rapidrouter-mcp"],
      "env": { "RAPIDROUTER_API_KEY": "rr-..." }
    }
  }
}

The same snippet works in Claude Desktop's claude_desktop_config.json and Cursor's .cursor/mcp.json; Claude Code takes claude mcp add rapidrouter -e RAPIDROUTER_API_KEY=sk-rr-v1-... -- npx -y github:nikhilsukhramani-maker/rapidrouter-mcp. Source and full setup notes live in the repo at rapidrouter-mcp.