# Rapid Router docs

> The complete Rapid Router documentation in one Markdown file, generated from https://rapidrouter.ai/docs at build time. The index of pages is https://rapidrouter.ai/llms.txt

# Quickstart

> Your first request in two minutes: create a key, set the base URL, and call any model with curl, Python or Node.

Source: https://rapidrouter.ai/docs

## 1. Create a key

Sign in and create a key under [Dashboard, API keys](https://rapidrouter.ai/dashboard/keys). The secret is shown once and stored only as a hash, so copy it then. The [free models](https://rapidrouter.ai/docs/free-tier) work with a zero balance; every other model draws on prepaid credits.

## 2. Point your client at Rapid Router

Every endpoint lives under one base URL, `https://rapidrouter.ai/api/v1`. Keep the key in an environment variable rather than in code.

```bash
export RAPIDROUTER_API_KEY="sk-rr-v1-..."
```

## 3. Send your first request

Name any model in the catalog by its `vendor/model` id. Here, Claude Sonnet 5:

curl:

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [
      {
        "role": "user",
        "content": "Hi"
      }
    ]
  }'
```

python:

```python
# pip install openai   (the HTTP client package; every request goes to Rapid Router)
from openai import OpenAI

client = OpenAI(base_url="https://rapidrouter.ai/api/v1", api_key="$RAPIDROUTER_API_KEY")
r = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role":"user","content":"Hi"}],
)
print(r.choices[0].message.content)
```

node:

```javascript
// npm install openai   (the HTTP client package; every request goes to Rapid Router)
import OpenAI from "openai"

const client = new OpenAI({ baseURL: "https://rapidrouter.ai/api/v1", apiKey: "$RAPIDROUTER_API_KEY" })
const r = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-5",
  messages: [{"role":"user","content":"Hi"}],
})
console.log(r.choices[0].message.content)
```

## 4. Read the reply

The standard chat completion, with two additions: `provider` says which upstream served it, and `usage.cost` is what the request billed, in US dollars. Swap the model string and the same code calls GPT, Gemini or any of the others.

```json
{
  "id": "msg_011CeHkp77djK1hiHwaAy22u",
  "object": "chat.completion",
  "model": "anthropic/claude-sonnet-5",
  "provider": "anthropic",
  "choices": [
    { "index": 0, "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }
  ],
  "usage": { "prompt_tokens": 18, "completion_tokens": 96, "total_tokens": 114, "cost": 0.000996 }
}
```

## Already using an SDK?

Change two values and nothing else: the base URL, and the model string, which is namespaced (`anthropic/claude-haiku-4-5`, not `gpt-4`). That is all the OpenAI SDKs above needed, and the same holds for Anthropic's SDK, the Vercel AI SDK, LangChain and any tool that takes a custom base URL; see [frameworks and SDKs](https://rapidrouter.ai/docs/integrations).

anthropic:

```python
# pip install anthropic   (works because Rapid Router also speaks /v1/messages)
from anthropic import Anthropic

client = Anthropic(base_url="https://rapidrouter.ai/api/v1", api_key="$RAPIDROUTER_API_KEY")
m = client.messages.create(
    model="anthropic/claude-sonnet-5",
    max_tokens=1024,
    messages=[{"role":"user","content":"Hi"}],
)
print(m.content[0].text)
```

ai-sdk:

```javascript
// npm install ai @ai-sdk/openai-compatible
import { createOpenAICompatible } from "@ai-sdk/openai-compatible"
import { generateText } from "ai"

const rapidrouter = createOpenAICompatible({
  name: "rapid router",
  baseURL: "https://rapidrouter.ai/api/v1",
  apiKey: "$RAPIDROUTER_API_KEY",
})
const { text } = await generateText({ model: rapidrouter("anthropic/claude-sonnet-5"), prompt: "Hi" })
console.log(text)
```

langchain:

```python
# pip install langchain-openai
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="anthropic/claude-sonnet-5", base_url="https://rapidrouter.ai/api/v1", api_key="$RAPIDROUTER_API_KEY")
print(llm.invoke("Hi").content)
```

## Next

- [Streaming](https://rapidrouter.ai/docs/streaming): Server-sent events in the standard chunk shape, ending with a usage chunk that carries the billed cost.
- [Tool calling](https://rapidrouter.ai/docs/tools): Let a model call your functions using the OpenAI tools shape, translated for Anthropic models.
- [Structured output](https://rapidrouter.ai/docs/structured-output): JSON mode and JSON schema through response_format, emulated on Anthropic models.
- [Failover and routing](https://rapidrouter.ai/docs/routing): How a failing provider falls over to the next route, and the provider object that steers the choice.
- [Errors](https://rapidrouter.ai/docs/errors): Status codes, the one error envelope, and the health endpoint worth monitoring.
- [API reference](https://rapidrouter.ai/docs/reference): Base URL, authentication, the error envelope, and every endpoint in one table.

---

# Authentication

> Bearer API keys, per-key budget caps, and why a lost key is replaced rather than recovered.

Source: https://rapidrouter.ai/docs/auth

Send your key as a bearer token on every request. A key belongs to one account and can carry its own monthly budget cap, so you can hand a capped key to a side project without putting the rest of your balance at risk. The secret is shown once at creation and stored only as a hash, so a lost key is replaced rather than recovered.

```
Authorization: Bearer $RAPIDROUTER_API_KEY
Content-Type: application/json
```

---

# Frameworks and SDKs

> The OpenAI SDKs, LangChain, the Vercel AI SDK and any tool with a custom base URL work after two changes.

Source: https://rapidrouter.ai/docs/integrations

Rapid Router speaks the standard chat completions API, so anything built against it works by changing two values: the base URL and the model string. No adapter, no SDK of ours to install.

openai-python:

```python
from openai import OpenAI

rapidrouter = OpenAI(api_key="sk-rr-v1-...", base_url="https://rapidrouter.ai/api/v1")
r = rapidrouter.chat.completions.create(
    model="anthropic/claude-haiku-4-5",
    messages=[{"role": "user", "content": "Hello"}],
)
```

langchain:

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    api_key="sk-rr-v1-...",
    base_url="https://rapidrouter.ai/api/v1",
    model="anthropic/claude-haiku-4-5",
)
```

vercel-ai-sdk:

```javascript
import { createOpenAI } from '@ai-sdk/openai'

const rr = createOpenAI({
  apiKey: process.env.RR_KEY,
  baseURL: 'https://rapidrouter.ai/api/v1',
})
```

The same holds for LlamaIndex, Instructor, Continue, Cline, Open WebUI and anything else that lets you set a custom base URL. When a tool asks for an "OpenAI API key" it means a key for that request shape - give it a Rapid Router key.

One thing to watch: model names here are namespaced, like `anthropic/claude-haiku-4-5`, not `gpt-4`. A tool that hardcodes a bare model name needs that string changed too.

---

# MCP server

> Search the catalog and call any model from Claude Desktop, Claude Code, Cursor and other MCP clients.

Source: https://rapidrouter.ai/docs/mcp

`rapidrouter-mcp` is a stdio MCP (Model Context Protocol) server that puts Rapid Router inside Claude Desktop, Claude Code, Cursor and any other MCP client. It exposes four tools: `search_models` searches the live catalog by substring with optional kind and price-ceiling filters, `get_model` returns one model's full catalog row with its prices, `chat` sends a prompt to any model and reports the answer with usage and billed cost, and `account_balance` shows your credit balance. The catalog tools need no key; `chat` and `account_balance` read `RAPIDROUTER_API_KEY` from the environment.

```json
{
  "mcpServers": {
    "rapidrouter": {
      "command": "npx",
      "args": ["-y", "github:nikhilsukhramani-maker/rapidrouter-mcp"],
      "env": { "RAPIDROUTER_API_KEY": "rr-..." }
    }
  }
}
```

The same snippet works in Claude Desktop's `claude_desktop_config.json` and Cursor's `.cursor/mcp.json`; Claude Code takes `claude mcp add rapidrouter -e RAPIDROUTER_API_KEY=sk-rr-v1-... -- npx -y github:nikhilsukhramani-maker/rapidrouter-mcp`. Source and full setup notes live in the repo at [rapidrouter-mcp](https://github.com/nikhilsukhramani-maker/rapidrouter-mcp).

---

# Guides

> Task-by-task guides: streaming, tools, structured output, media, batches, routing, budgets, errors and limits.

Source: https://rapidrouter.ai/docs/guides

## Build

- [Streaming](https://rapidrouter.ai/docs/streaming): Server-sent events in the standard chunk shape, ending with a usage chunk that carries the billed cost.
- [Tool calling](https://rapidrouter.ai/docs/tools): Let a model call your functions using the OpenAI tools shape, translated for Anthropic models.
- [Structured output](https://rapidrouter.ai/docs/structured-output): JSON mode and JSON schema through response_format, emulated on Anthropic models.
- [Response healing](https://rapidrouter.ai/docs/healing): Repair almost-JSON answers (fences, trailing commas, truncation) without a second model call.
- [Reasoning models](https://rapidrouter.ai/docs/reasoning): How thinking tokens are billed and reported, and the max_tokens setting that leaves no room to answer.
- [PDF, audio and video input](https://rapidrouter.ai/docs/multimodal): Attach PDFs, audio and video to chat messages; the same request body works on every model.
- [Web search](https://rapidrouter.ai/docs/websearch): Ground any model in live web results with the web plugin or the :online suffix, with citations.

## Images, audio and batches

- [Images, speech and video](https://rapidrouter.ai/docs/media): Generate and edit images, text to speech, transcription, translation and video, billed per unit.
- [Voice pipeline](https://rapidrouter.ai/docs/voice): Audio in, spoken answer out: transcription, a chat model and speech in one call.
- [Batch API](https://rapidrouter.ai/docs/batch): Queue up to 500 chat completions in one call; they run within 24 hours and results are kept 30 days.

## Routing and fallbacks

- [Failover and routing](https://rapidrouter.ai/docs/routing): How a failing provider falls over to the next route, and the provider object that steers the choice.
- [Auto router](https://rapidrouter.ai/docs/auto-router): rapidrouter/auto picks the model per prompt: light, code, Indic or heavy.
- [Presets](https://rapidrouter.ai/docs/presets): Named, versioned request templates you change in the dashboard without a deploy.
- [Message transforms](https://rapidrouter.ai/docs/transforms): middle-out compression for conversations longer than the model can take.
- [Latency and performance](https://rapidrouter.ai/docs/performance): The four things that actually move response time, roughly in order of size.

## Cost and budgets

- [Budgets](https://rapidrouter.ai/docs/budgets): Cap spend per key, per account and per workspace, and the error a request gets once a cap is hit.
- [Spend alerts](https://rapidrouter.ai/docs/alerts): Email, WhatsApp or webhook alerts on daily and monthly thresholds, plus automatic anomaly alerts.
- [Bring your own keys](https://rapidrouter.ai/docs/byok): Route requests on your own provider keys and pay only the platform fee.
- [Prompt caching](https://rapidrouter.ai/docs/caching): Cached input billed at the provider cache rate, reported on every response.
- [Response caching](https://rapidrouter.ai/docs/response-cache): Opt-in replay of an identical request, billed at zero.
- [Free tier](https://rapidrouter.ai/docs/free-tier): Which models are free, the daily limits per account, and the 429 when they run out.

## Errors and limits

- [Errors](https://rapidrouter.ai/docs/errors): Status codes, the one error envelope, and the health endpoint worth monitoring.
- [Rate limits](https://rapidrouter.ai/docs/limits): Requests per minute per key and requests in flight per account, both growing with your balance.
- [Idempotency](https://rapidrouter.ai/docs/idempotency): Retry a request whose response was lost without paying twice, with the Idempotency-Key header.
- [Tracing](https://rapidrouter.ai/docs/tracing): x-request-id and W3C traceparent on every response, and OTLP export for self-hosted gateways.

## Apps and teams

- [Sign in with Rapid Router](https://rapidrouter.ai/docs/oauth): An OAuth PKCE flow so your users pay for your app with their own credits.
- [App attribution](https://rapidrouter.ai/docs/attribution): The HTTP-Referer and X-Title headers, and the public Apps ranking they feed.
- [Teams](https://rapidrouter.ai/docs/orgs): Organisations, roles, invite links, per-member or shared balances, and per-member spend.

## Data and privacy

- [Data collection](https://rapidrouter.ai/docs/data): What is recorded per request, and why prompts and responses are not stored unless you turn on capture.
- [Provider logging](https://rapidrouter.ai/docs/provider-data): Where a prompt goes after the gateway: links to each provider's own policy.
- [Request capture](https://rapidrouter.ai/docs/capture): Opt-in, per-key storage of request and response bodies for debugging, deleted on a schedule.

---

# Streaming

> Server-sent events in the standard chunk shape, ending with a usage chunk that carries the billed cost.

Source: https://rapidrouter.ai/docs/streaming

Set `"stream": true` for server-sent events. Deltas arrive in the standard chunk shape and the stream terminates with `data: [DONE]`. A final usage chunk carries the billed cost.

```
data: {"object":"chat.completion.chunk","model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{"content":"Rou"},"finish_reason":null}]}
data: {"object":"chat.completion.chunk","model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{"content":"ting"},"finish_reason":null}]}
data: {"object":"chat.completion.chunk","model":"anthropic/claude-sonnet-5","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":96,"total_tokens":114,"cost":0.00074}}
data: [DONE]
```

---

# Tool calling

> Let a model call your functions using the OpenAI tools shape, translated for Anthropic models.

Source: https://rapidrouter.ai/docs/tools

Send `tools` in the OpenAI function shape on `POST /chat/completions` and the model can answer with `tool_calls` instead of text. You run the function, append its result as a `role: "tool"` message carrying the `tool_call_id`, and call again. The shape is the same on every model: for Anthropic models the gateway turns the tools into Anthropic's format, and the `tool_use` blocks that come back into `tool_calls` with `finish_reason: "tool_calls"`.

request:

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [{ "role": "user", "content": "What is the weather in Indore?" }],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }]
  }'

# -> choices[0].finish_reason: "tool_calls"
#    choices[0].message.tool_calls: [{ "id": "toolu_...", "type": "function",
#      "function": { "name": "get_weather", "arguments": "{\"city\":\"Indore\"}" } }]
```

tool result:

```javascript
// Run get_weather yourself, then send the whole exchange back:
"messages": [
  { "role": "user", "content": "What is the weather in Indore?" },
  { "role": "assistant", "content": null, "tool_calls": [ /* exactly as returned */ ] },
  { "role": "tool", "tool_call_id": "toolu_...", "content": "{\"temp_c\": 31, \"sky\": \"clear\"}" }
]
// with the same tools array. The model now answers in text.
```

`tool_choice` takes `"auto"` (the default), `"none"`, `"required"`, or `{ "type": "function", "function": { "name": "get_weather" } }` to force one function. With `stream: true` the calls arrive as `tool_calls` deltas in the standard chunk shape, Anthropic models included.

**Which models.** `capabilities.tools` in [GET /models](https://rapidrouter.ai/docs/models) is true for models that take tools; filter on it before you send them. Where a model has several routes, `"provider": { "require_parameters": true }` keeps the request on routes known to honour `tools` ([routing](https://rapidrouter.ai/docs/routing)).

**Send tool calls back unchanged.** Gemini 3 attaches `extra_content.google.thought_signature` to a tool call and checks it on the next turn. If your client drops it, as some older SDKs do, the gateway fills in Google's documented placeholder so the call still succeeds, at the cost of that step's earlier reasoning.

The other request formats take tools in their own shape: [/v1/messages](https://rapidrouter.ai/docs/messages) with Anthropic's `input_schema`, and [/v1/responses](https://rapidrouter.ai/docs/responses) with flat function tools and `function_call` output items.

---

# Structured output

> JSON mode and JSON schema through response_format, emulated on Anthropic models.

Source: https://rapidrouter.ai/docs/structured-output

Ask for JSON with `response_format` on `POST /chat/completions`. `{ "type": "json_object" }` asks for any valid JSON object; `{ "type": "json_schema", "json_schema": { "name", "schema", "strict" } }` asks for JSON that matches your schema. The answer arrives as a JSON string in `choices[0].message.content`, ready to parse.

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.6-flash",
    "messages": [{ "role": "user", "content": "Extract: Asha, 31, Pune" }],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "age": { "type": "integer" },
            "city": { "type": "string" }
          },
          "required": ["name", "age", "city"],
          "additionalProperties": false
        }
      }
    }
  }'

# -> choices[0].message.content: "{\"name\":\"Asha\",\"age\":31,\"city\":\"Pune\"}"
```

**Anthropic models.** Anthropic's API has no `response_format`, so the gateway emulates it: the schema becomes a tool the model is made to call, and the arguments of that call come back as the message content with `finish_reason: "stop"`, the same as on any other model. One difference: with `stream: true` the JSON arrives in a single chunk rather than token by token.

**Which models.** `capabilities.json` in [GET /models](https://rapidrouter.ai/docs/models) marks the models that support JSON output. On [/v1/responses](https://rapidrouter.ai/docs/responses), `text.format` with `json_schema` or `json_object` does the same thing.

**Almost-JSON.** A model can still wrap the answer in a markdown fence, leave a trailing comma, or run out of `max_tokens` mid-object. Add the [response-healing](https://rapidrouter.ai/docs/healing) plugin and the gateway repairs those without a second model call.

---

# Response healing

> Repair almost-JSON answers (fences, trailing commas, truncation) without a second model call.

Source: https://rapidrouter.ai/docs/healing

Models asked for JSON sometimes return almost-JSON: wrapped in a markdown fence, a trailing comma, or truncated mid-object by `max_tokens`. Add `plugins: [{ "id": "response-healing" }]` to a request that uses `response_format` and the gateway repairs those deterministically: fences stripped, trailing commas removed, truncated brackets and quotes closed. No second model call, no extra cost, no change to latency worth measuring.

```json
{
  "model": "google/gemini-3.6-flash",
  "response_format": { "type": "json_object" },
  "plugins": [{ "id": "response-healing" }],
  "messages": [{ "role": "user", "content": "List 3 colours as JSON" }]
}

// response, only when a repair was actually made:
// "healing_applied": true
```

Valid JSON is never touched, requests without `response_format` are never touched, and a broken answer that cannot be repaired deterministically is returned as-is rather than guessed at. `healing_applied: true` appears only when a repair happened, so you can count how often your model actually needed it. Non-streaming requests only.

---

# Reasoning models

> How thinking tokens are billed and reported, and the max_tokens setting that leaves no room to answer.

Source: https://rapidrouter.ai/docs/reasoning

Some models think before they answer. Those thinking tokens are billed as output by every provider we route to, and they are usually not shown to you. We report them separately at `usage.completion_tokens_details.reasoning_tokens` so the bill is explicable. They are counted, never charged twice.

The failure worth knowing about: set `max_tokens` too low on a reasoning model and it can spend the entire budget thinking, then return an empty string with `finish_reason: "length"`. The request is billed and nothing in the standard response explains it. We add a warning that says so:

```json
"warning": {
  "code": "reasoning_consumed_budget",
  "message": "The model spent all 5 output tokens reasoning and had none left
              to answer. Raise max_tokens, or pick a non-reasoning model.
              This request was still billed."
}
```

Give reasoning models room. A budget that is generous for a normal model can be nothing at all once thinking is paid for first.

---

# PDF, audio and video input

> Attach PDFs, audio and video to chat messages; the same request body works on every model.

Source: https://rapidrouter.ai/docs/multimodal

Chat messages can carry more than text and images. Three additional content part shapes are accepted on `POST /chat/completions`, and every model can take them: where the provider reads the media natively it is passed through, everywhere else Rapid Router converts it server-side before routing.

pdf:

```json
{ "role": "user", "content": [
  { "type": "text", "text": "Summarise this document." },
  { "type": "file", "file": {
      "filename": "report.pdf",
      "file_data": "data:application/pdf;base64,JVBERi0x..." } }
] }

# file_data also accepts a plain https URL (or use file.file_url).
# Up to 20MB per file; multiple file parts per message are fine.
```

audio:

```json
{ "role": "user", "content": [
  { "type": "text", "text": "What is said in this recording?" },
  { "type": "input_audio", "input_audio": {
      "data": "<base64 audio bytes>",
      "format": "wav" } }
] }

# format: "wav" or "mp3"
```

video:

```json
{ "role": "user", "content": [
  { "type": "text", "text": "What happens in this video?" },
  { "type": "video_url", "video_url": {
      "url": "https://www.youtube.com/watch?v=..." } }
] }

# Google models only. YouTube links and direct video URLs.
```

**PDF.** Google and Anthropic models read the PDF natively, pages, layout, images and all, so scanned documents work there. On every other provider the text is extracted server-side and delivered to the model as `Contents of <filename>: ...`, capped at 200,000 characters with a truncation note. A corrupt or empty PDF answers 400 rather than a garbled prompt.

**One body, every model.** The vendor-specific shapes are normalised at the gateway, so the same request works whichever model serves it. Anthropic's `{ "type": "document" }` and `{ "type": "image" }` blocks, Google's PDF-as-`image_url` data URL, and the standard `image_url` image part are all accepted on every model; Rapid Router rewrites each into whatever the serving provider takes. Swapping `google/gemini-3.1-pro-preview` for `anthropic/claude-sonnet-5` needs no change to how the media is attached.

**Audio.** Google models hear the audio natively. For every other model the clip is transcribed by the cheapest connected speech-to-text model and the part becomes `[Audio transcript]: ...` in the prompt. The transcription is billed as its own line in your usage log at the STT model's normal per-minute price, on top of the chat request's tokens. If no speech-to-text provider is connected the request answers 400 saying so.

**Video.** Supported on Google models only, which watch the video from its URL. Any other model answers 400: `video input is only supported on Google models`. There is no server-side conversion for video.

---

# Web search

> Ground any model in live web results with the web plugin or the :online suffix, with citations.

Source: https://rapidrouter.ai/docs/websearch

Ground any model with live web results. Attach the `web` plugin, or append `:online` to the model name for the same thing with defaults. The last user message becomes the search query, the results are handed to the model as numbered sources it is asked to cite, and the response carries one `url_citation` annotation per source plus a top-level `web_search` field with the engine used and the result count.

plugin:

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.6-flash",
    "plugins": [{ "id": "web", "max_results": 5 }],
    "messages": [{ "role": "user", "content": "What changed in the latest Node.js release?" }]
  }'

# -> ..., "web_search": { "engine": "exa", "results_count": 5 },
#    choices[0].message.annotations: [{ "type": "url_citation", "url": "...", "title": "..." }]
```

:online:

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-3.6-flash:online",
    "messages": [{ "role": "user", "content": "What changed in the latest Node.js release?" }]
  }'
```

The plugin accepts `max_results` (default 5, capped at 10), `search_prompt` to replace the instruction that introduces the sources, and `engine` to pin one. Engines: `exa` and `serper` are the real search APIs and add a flat **$0.004 per searched request** on top of token cost, included in `usage.cost`; `ddg` is a free keyless fallback of noticeably lower quality, used automatically when no engine key is configured, and carries no fee.

A broken search never breaks the completion: if the engine fails or times out (8s), the request proceeds without results, nothing extra is billed, and the response says `"web_search": { "error": "search_failed" }`. Annotations are returned on non-streamed responses; a streamed request is still searched and billed the same way. Batch items are not searched.

---

# Images, speech and video

> Generate and edit images, text to speech, transcription, translation and video, billed per unit.

Source: https://rapidrouter.ai/docs/media

Five endpoints generate media rather than read it. They bill **per unit**, per image, per character, per minute of audio, per video, instead of per token, at the unit price shown in the catalog. As with tokens, nothing is added per request; our fee is charged when you buy credits. Media models never appear in `GET /models`; `GET /v1/media/models` lists them with their kind (`image`, `tts`, `stt`, `translate`, `video`). Responses use the `{ "ok": true, "data": ... }` envelope, and every call reports its billed `cost`. A failed generation bills nothing.

images:

```bash
curl https://rapidrouter.ai/api/v1/images/generations \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemini-2.5-flash-image",
    "prompt": "a watercolour hornbill",
    "n": 2
  }'

# -> { "ok": true, "data": { "created": 1756...,
#      "model": "google/gemini-2.5-flash-image",
#      "data": [{ "b64_json": "..." }, { "b64_json": "..." }],
#      "cost": 0.0832 } }
# n is 1 to 4; you are billed per image actually returned.
```

image edit:

```bash
curl https://rapidrouter.ai/api/v1/images/edits \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen-image-edit",
    "prompt": "make the sky a sunset",
    "image": "data:image/png;base64,iVBORw0..."
  }'

# image: a data: URL or an https URL (PNG, JPEG, WEBP, under 8 MB)
# -> { "ok": true, "data": { "model": "qwen/qwen-image-edit",
#      "data": [{ "b64_json": "..." }], "cost": 0.075 } }
# Edit models: qwen/qwen-image-edit, black-forest-labs/flux.1-kontext-dev,
# wan-ai/wan2.7-image-edit, bria/fibo_edit, prunaai/p-image-edit.
```

speech:

```bash
curl https://rapidrouter.ai/api/v1/audio/speech \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sarvam/bulbul-v3",
    "input": "Namaste, duniya",
    "voice": "ritu",
    "language": "hi-IN"
  }'

# -> { "ok": true, "data": { "model": "sarvam/bulbul-v3",
#      "audio": "<base64>", "format": "wav", "cost": 0.00003 } }
# Billed per character of input. OpenAI models return mp3, the rest wav.
```

transcribe:

```bash
curl https://rapidrouter.ai/api/v1/audio/transcriptions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/whisper-large-v3",
    "audio_b64": "<base64 wav bytes>",
    "language": "en"
  }'

# -> { "ok": true, "data": { "model": "openai/whisper-large-v3",
#      "text": "...", "language_code": "en", "cost": 0.0000234 } }
```

translate:

```bash
curl https://rapidrouter.ai/api/v1/translate \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "How are you?",
    "target_language_code": "hi-IN"
  }'

# Always served by sarvam/mayura, billed per character.
# source_language_code defaults to "auto".
# -> { "ok": true, "data": { "model": "sarvam/mayura",
#      "translated_text": "...", "source_language_code": "en-IN",
#      "cost": 0.0000003 } }
```

video:

```bash
# Video is async: create returns a job id, then poll.
curl https://rapidrouter.ai/api/v1/videos/generations \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "google/veo-3.1-fast", "prompt": "waves at dusk" }'

# -> { "ok": true, "data": { "model": "google/veo-3.1-fast",
#      "id": "models/veo-3.1-.../operations/abc123", "status": "processing",
#      "poll": "/v1/videos/generations/status", "cost": 1.56 } }

curl https://rapidrouter.ai/api/v1/videos/generations/status \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "id": "models/veo-3.1-.../operations/abc123" }'

# status: "processing" -> "done" (or "failed"). done carries "video"
# (a URL) - except openai/sora-2, whose bytes come back as
# "video_b64" with format "mp4". Small models that render in a few
# seconds answer "done" with the video on the create call itself;
# their id still polls, so the same client code works for both.
```

**Voices.** Each speech family has its own voice names; send `voice` to pick one, or send none and the model's default is used (`ritu` for Sarvam, `alloy` for `openai/tts-1`, `tara` for Orpheus, `af_bella` for Kokoro). Sarvam also takes `language`, defaulting to `hi-IN`.

**Transcription billing.** Duration-priced models bill the minutes the provider reports; when no duration comes back, the WAV header is read, and an unreadable header bills one full minute, never less. `sarvam/saarika-v2` bills a flat price per request instead.

**Video billing.** Flat per video, debited when the render starts, because the provider commits the cost at that point. Duration and resolution are pinned per model so the flat price is exact; polling is free. An unknown model answers 404, a disconnected provider 503, and an upstream failure 502 with nothing billed.

---

# Voice pipeline

> Audio in, spoken answer out: transcription, a chat model and speech in one call.

Source: https://rapidrouter.ai/docs/voice

One call, audio in, spoken answer out. `POST /v1/voice/chat` transcribes your audio, runs the transcript through the chat model of your choice, and speaks the reply back, returning the transcript, the reply text and the reply audio as base64 in a single response. Send the audio as a `multipart/form-data` file or as base64 JSON; audio up to 15MB, non-stream only.

multipart:

```bash
curl https://rapidrouter.ai/api/v1/voice/chat \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -F "file=@question.mp3;type=audio/mpeg" \
  -F "model=google/gemini-flash-latest" \
  -F "voice=tara" \
  -F "system=Answer in one short sentence."

# Optional fields: tts_model, stt_model, max_tokens,
# response_format (mp3, wav, ...), language (Sarvam TTS).
```

json:

```bash
curl https://rapidrouter.ai/api/v1/voice/chat \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "audio_b64": "<base64 audio bytes>",
    "mime_type": "audio/mpeg",
    "model": "google/gemini-flash-latest",
    "tts_model": "hexgrad/kokoro-82m",
    "system": "Answer in one short sentence."
  }'
```

response:

```json
{ "ok": true, "data": {
    "transcript": "what is the capital of france",
    "reply_text": "The capital of France is Paris.",
    "audio_b64": "<base64 reply audio>",
    "audio_mime": "audio/wav",
    "models": { "stt": "openai/whisper-large-v3-turbo",
                "chat": "google/gemini-flash-latest",
                "tts": "xiaomimimo/mimo-v2.5-tts" },
    "usage": { "stt_cost_usd": 0.0000002, "chat_cost_usd": 0.000031,
               "tts_cost_usd": 0, "cost_usd_total": 0.0000312 } } }
```

**Models.** `model` is the chat model (any chat model in the catalog). `stt_model` and `tts_model` pick specific media models from `GET /v1/media/models`; leave them out and the cheapest configured model of each kind is used. `voice` and `response_format` pass through to the TTS stage.

**Billing.** Each stage bills itself through its normal path, so your usage log shows three lines: the STT call per minute (or per request), the chat call per token, and the TTS call per character of the reply. The response's `usage` reports each stage's cost and the total. If a stage fails, the error code names it (`voice_stt_failed`, `voice_chat_failed`, `voice_tts_failed`) and `error.billed_stages` lists the stages that were already billed; audio that transcribes to silence answers 422 with only the STT stage billed.

A bidirectional realtime WebSocket voice API (stream audio in, stream audio out) is planned; this endpoint is the request-response version.

---

# Batch API

> Queue up to 500 chat completions in one call; they run within 24 hours and results are kept 30 days.

Source: https://rapidrouter.ai/docs/batch

Work that does not need an answer right now can be queued. Submit up to 500 chat completions in one call and they run within a 24 hour window at **standard per-token pricing**, through the same routing and fallbacks as live traffic. Requests are inline JSON, no file upload step, and results are retained for 30 days. Streaming is not supported inside batch items.

create:

```bash
curl https://rapidrouter.ai/api/v1/batches \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      { "custom_id": "row-1", "body": { "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Summarise: ..." }] } },
      { "custom_id": "row-2", "body": { "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Summarise: ..." }] } }
    ]
  }'

# -> { "id": "batch_...", "object": "batch", "status": "queued",
#      "discount_pct": 0, "request_counts": { "total": 2, "completed": 0, "failed": 0 },
#      "expires_at": 1756300000 }
```

poll:

```bash
curl https://rapidrouter.ai/api/v1/batches/batch_... \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY"

# status walks queued -> running -> completed. failed, expired and
# cancelled are the other terminal states.
```

results:

```bash
curl https://rapidrouter.ai/api/v1/batches/batch_.../results \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY"

# JSONL, one line per item in submission order:
{"custom_id":"row-1","response":{"object":"chat.completion","usage":{"cost":0.000021},"choices":[...]}}
{"custom_id":"row-2","error":{"code":"insufficient_credits","message":"..."}}
```

Each item is billed individually at the model's normal per-token price, visible in your usage log like any other request. An item that fails does not sink the batch: the rest complete and the failed line carries the error. Anything still unfinished when the 24 hour window closes is marked expired and never billed. `POST /batches/:id/cancel` stops a batch; items already served keep their results. Results are deleted 30 days after the batch was created, so collect them before then.

---

# Failover and routing

> How a failing provider falls over to the next route, and the provider object that steers the choice.

Source: https://rapidrouter.ai/docs/routing

A model can have more than one route. When the first provider fails with a server error, a rate limit, or a problem with our account there, the request moves to the next route and you get the answer rather than the error. Whichever route served is the one billed, at its own price, and the response reports `provider` and `fallbacks_tried`.

A genuine mistake in your request is not retried. A malformed body or a refusal is returned immediately, because sending it to three more providers only wastes your time.

You can steer this per request with a `provider` object carrying `only` or `ignore`, or set a default in your dashboard. If your preferences leave no usable route, you get `503` rather than a silent substitution.

| provider field | What it does |
| --- | --- |
| only | Provider slugs. Only these routes are used. |
| ignore | Provider slugs. These routes are skipped. |
| order | Provider slugs, tried first and in this order; the rest follow. |
| sort | "price" (the default, cheapest first), "throughput" (most output tokens a second) or "latency" (lowest median latency), from the last 24 hours of served traffic. The suffixes :floor and :nitro on a model id mean price and throughput. |
| max_price | { "prompt", "completion" }, US dollars per million tokens. Dearer routes are skipped; if none is left the answer is 404 no_endpoints_within_max_price. |
| require_parameters | true keeps only routes known to honour the request's tools and response_format; if none is left, 404 no_endpoints_support_parameters. |
| allow_fallbacks | false tries the first route only. |

```json
{
  "model": "meta-llama/llama-3.3-70b-instruct",
  "provider": { "sort": "throughput", "ignore": ["deepinfra"], "max_price": { "prompt": 1, "completion": 1 } },
  "messages": [{ "role": "user", "content": "Hi" }]
}
```

Defaults saved in the dashboard's Routing pane apply to every request, and a `provider` object in the request wins field by field. A key in a workspace with routing preferences of its own uses the workspace's instead of the account's.

---

# Auto router

> rapidrouter/auto picks the model per prompt: light, code, Indic or heavy.

Source: https://rapidrouter.ai/docs/auto-router

Send `"model": "rapidrouter/auto"` and the gateway picks the model for you. The prompt is classified locally, no extra model call and no added latency, into one of four buckets: `light` (short chat) routes to the cheapest live model, `code` to the strongest live coding model, `indic` (Indian languages and scripts) to Sarvam when live, and `heavy` (long context, tools, big output budgets) to a strong mid model. `openrouter/auto` is accepted as a drop-in alias.

curl:

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rapidrouter/auto",
    "auto": { "cost_tier": "low" },
    "messages": [{ "role": "user", "content": "Fix this function: ..." }]
  }'
```

response:

```json
{
  "object": "chat.completion",
  "model": "anthropic/claude-sonnet-4-5",
  "provider": "anthropic",
  "auto_router": { "bucket": "code", "chosen": "anthropic/claude-sonnet-4-5" },
  "choices": [ ... ],
  "usage": { ... }
}
```

The optional `auto` object biases the choice: `cost_tier` of `"low"`, `"medium"` (the default) or `"high"` shifts each bucket toward cheaper or stronger models. It is read by the router and stripped before forwarding. The response reports `auto_router.bucket` and `auto_router.chosen`, streaming included (as a leading chunk), and the request is billed at the chosen model's normal price.

---

# Presets

> Named, versioned request templates you change in the dashboard without a deploy.

Source: https://rapidrouter.ai/docs/presets

A preset is a named request template you manage in the dashboard: a model (with optional fallback models), a system prompt, sampling parameters (`temperature`, `top_p`, `max_tokens`), reasoning settings and provider routing preferences. Your code calls the name; the gateway resolves it server-side on every request, so changing the preset changes behaviour without a deploy. Three call forms:

preset field:

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "preset": "support-triage",
    "messages": [{ "role": "user", "content": "My invoice is wrong" }]
  }'

# -> { ..., "preset": { "slug": "support-triage", "version": 3 } }
```

@preset/ model:

```bash
# The model field itself can name the preset:
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "@preset/support-triage",
    "messages": [{ "role": "user", "content": "My invoice is wrong" }]
  }'
```

model@preset/:

```bash
# Pin the model, take everything else from the preset:
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o@preset/support-triage",
    "messages": [{ "role": "user", "content": "My invoice is wrong" }]
  }'
```

**Merging is field by field and the request always wins.** A model in the request beats the preset's; when the preset supplies it, its `model` then its `models` list are tried in order and the first with an available route serves (provider-level failover then applies to that model as usual). The system prompt is prepended only when the request carries no system message of its own. Sampling parameters fill gaps: a `temperature` in the request is used, the preset's only otherwise. Provider preferences merge key by key with the request's winning. The preset fields are stripped before anything is forwarded upstream.

Presets are versioned: every save appends a new version, the response reports which one served (`"preset": { "slug", "version" }`, on streams as a leading chunk), and any earlier version can be restored from the dashboard's history, itself recorded as a new version. Presets are per account; calling someone else's slug, or a deleted one, is a `400 preset_not_found`; `GET /v1/presets` lists the slugs your account has. Works on chat completions (streaming included), `/messages` and `/responses` via the model forms, in batches, and in the playground.

---

# Message transforms

> middle-out compression for conversations longer than the model can take.

Source: https://rapidrouter.ai/docs/transforms

A conversation that has outgrown the model's context normally just fails. Add `transforms: ["middle-out"]` to the request body and the gateway compresses it instead: messages are dropped from the **middle**, keeping the system message, the start and the end, which is where models actually look. It also applies whenever a conversation has more than 800 messages, since some providers cap message count outright.

```json
{
  "model": "google/gemini-3.6-flash",
  "transforms": ["middle-out"],
  "messages": [ /* a conversation too long for the model */ ]
}

// response, only when something was actually trimmed:
// "transforms_applied": ["middle-out"]
```

A request that already fits is forwarded untouched and the response carries no `transforms_applied` field, so its presence is meaningful. Token counts are estimated conservatively, and the field itself is stripped before forwarding, so no upstream ever sees it.

---

# Latency and performance

> The four things that actually move response time, roughly in order of size.

Source: https://rapidrouter.ai/docs/performance

Rapid Router adds one network hop between you and the provider. The routing itself is a lookup and a forward, so the number that dominates your response time is the provider's own speed, not ours.

Four things actually move the needle, roughly in order of size:

- **Stream.** Set `stream: true`. A 900 token answer takes seconds to finish but the first token lands in a fraction of that, and a user reading along does not feel the rest.
- **Pick for speed, not size.** A smaller model on fast silicon routinely beats a large model on ordinary hardware by several times, and for extraction, classification and routine drafting the answer is the same.
- **Keep the connection alive.** Reuse one client instead of constructing a new one per call. A fresh TLS handshake costs tens of milliseconds every time.
- **Cap your output.** `max_tokens` bounds the worst case. Without it a chatty model decides your latency for you.

Live latency per provider is on the [providers page](https://rapidrouter.ai/providers), measured from real traffic rather than a synthetic probe.

A provider that fails is routed around where the model has a second route. See [failover](https://rapidrouter.ai/docs/routing) for when that applies and when the error is returned to you instead.

---

# Budgets

> Cap spend per key, per account and per workspace, and the error a request gets once a cap is hit.

Source: https://rapidrouter.ai/docs/budgets

Four ceilings, each checked before every request, so a runaway loop or a leaked key stops at a number you chose. They count billed spend in US dollars. A request already running when a ceiling is crossed finishes; the next one is refused, and nothing is billed for the refusal.

| Ceiling | Covers | Resets | When hit |
| --- | --- | --- | --- |
| Key monthly cap | one key | on the 1st of the month (IST) | 402 key_cap_reached |
| Key limit | one key | daily, weekly or monthly (UTC), or never: a lifetime limit | 402 key_limit_reached |
| Account budget | every key on the account | by UTC day, ISO week and calendar month | 403 budget_exceeded |
| Workspace budget | every key in the workspace | daily, weekly, monthly, or never: lifetime | 403 workspace_budget_exceeded |

**Per key.** Set the cap and the limit in the dashboard's key editor, or in code through the [Provisioning API](https://rapidrouter.ai/docs/provisioning) with `monthly_cap_usd`, `limit_usd` and `limit_reset` (`daily`, `weekly`, `monthly`, or `null` for a lifetime limit). That is how you hand each customer or side project a key that cannot spend more than its share.

key limit:

```bash
curl https://rapidrouter.ai/api/v1/provisioning/keys \
  -H "Authorization: Bearer sk-rr-mgmt-..." \
  -H "Content-Type: application/json" \
  -d '{ "name": "customer-42", "limit_usd": 10, "limit_reset": "monthly" }'

# GET /provisioning/keys/:id reports limit_spent_usd,
# limit_remaining_usd and limit_resets_at.
```

account budget:

```bash
# Session-authenticated (dashboard login).
curl -X PUT https://rapidrouter.ai/api/v1/budget \
  -H "Authorization: Bearer <session JWT>" \
  -H "Content-Type: application/json" \
  -d '{ "daily_usd": 20, "weekly_usd": null, "monthly_usd": 300 }'

# Each window is a number of at least 0, or null for none.
# GET /budget reads them back.
```

refusal:

```json
{
  "ok": false,
  "error": {
    "code": "key_limit_reached",
    "message": "This key hit its monthly limit of $10. It resets automatically."
  }
}
```

**Per account and workspace.** The account budget covers spend from every key you own; a budget your organisation set for you can only be changed by its owner or an admin, and `PUT /budget` answers `403 managed`. A workspace's budgets are set under Workspaces in the dashboard and cover the keys in that workspace.

**Being told.** When a key limit or the account budget is hit, your webhook gets a `key.limit_reached` or `budget.exceeded` event. To hear before a ceiling rather than at it, set [spend alerts](https://rapidrouter.ai/docs/alerts).

---

# Spend alerts

> Email, WhatsApp or webhook alerts on daily and monthly thresholds, plus automatic anomaly alerts.

Source: https://rapidrouter.ai/docs/alerts

Set a daily or monthly dollar threshold and get told when your spend crosses it, by email, WhatsApp, or a webhook. Settings live at `GET /v1/alerts` and `PUT /v1/alerts`, session-authenticated.

configure:

```bash
curl -X PUT https://rapidrouter.ai/api/v1/alerts \
  -H "Authorization: Bearer <session JWT>" \
  -H "Content-Type: application/json" \
  -d '{
    "daily_usd_threshold": 25,
    "monthly_usd_threshold": 400,
    "alert_phone": "+14155550123",
    "email_enabled": true,
    "wa_enabled": true
  }'

# Thresholds are positive dollars, or null to turn one off.
# alert_phone must be international format (E.164); it is only
# needed when wa_enabled is true. email_enabled defaults to true,
# wa_enabled to false.
```

webhook payloads:

```javascript
// spend.threshold, at most once per period (UTC day / calendar month):
{ "event": "spend.threshold",
  "data": { "period": "daily", "spent_usd": 26.41, "threshold_usd": 25 },
  "created_at": "..." }

// spend.anomaly, at most once per day:
{ "event": "spend.anomaly",
  "data": { "last_hour_usd": 14.2, "baseline_hourly_usd": 0.8,
            "top_model": "openai/gpt-4o", "top_key_id": "key_..." },
  "created_at": "..." }
```

A sweep checks spend every 5 minutes and fires each alert **at most once per period**: once per UTC day for the daily threshold, once per calendar month for the monthly one. The dedupe is stored in the database, so a redeploy never re-sends the same breach.

**Anomaly detection** needs no configuration. When your spend in the last hour reaches $1 or more and runs at least 5 times your average hourly spend over the prior 7 days, you are alerted, at most once per day, with the model and key the money went to. An account with no prior traffic has a zero baseline, so any burst of $1 or more counts, which is exactly the stolen-key case the alert exists for. Both events are also delivered to your webhook, configured in the dashboard, as signed POSTs of `{ event, data, created_at }`.

---

# Bring your own keys

> Route requests on your own provider keys and pay only the platform fee.

Source: https://rapidrouter.ai/docs/byok

Save your own provider API key under **Dashboard -> Integrations** and every request routed to that provider runs on **your** key: the tokens bill your account with the provider directly, at whatever rate you have negotiated with them. Instead of the token price, we charge **the regular platform fee ([see pricing](https://rapidrouter.ai/pricing)) on what the usage would have cost** at the route's normal price, debited from your credits. A response served this way carries `"byok": true`, and `usage.cost` is that fee.

**Verification.** A key is checked against the provider with one real call before it is saved; a key the provider rejects is not stored. Saved keys are encrypted at rest and never returned by the API - the dashboard shows only the last four characters.

**Failure semantics.** While you have a key saved for a provider, requests to that provider are **never silently served on our key**. If the provider answers `401`/`403` (key revoked) or `402` (your provider account is out of balance), the request fails with `byok_key_rejected` or `byok_no_balance` naming your key, nothing is billed, and the key is marked in the dashboard. The error persists until you update or remove the key; after removal, requests route on our keys at normal prices again. Ordinary provider hiccups (429, 5xx) fail over between providers as usual.

```json
{
  "object": "chat.completion",
  "model": "google/gemini-3.6-flash",
  "provider": "google",
  "byok": true,
  "choices": [ ... ],
  "usage": { ..., "cost": 0.0000021 }
}

# cost = the platform fee on the route's normal price for these tokens.
# The tokens themselves billed your own Google account.
```

---

# Prompt caching

> Cached input billed at the provider cache rate, reported on every response.

Source: https://rapidrouter.ai/docs/caching

When you send the same long prefix repeatedly, a system prompt, a document, a set of examples, some providers keep the processed form of it and charge far less to read it back. Rapid Router passes that through and bills cached input at the provider's cache rate rather than the full input rate.

You do not configure anything. If the provider reports cached tokens, they are priced as cached. Where we hold a cache rate it currently runs 80% to 92% below the input rate. Where we do not hold one, cached tokens bill at the normal input price, so an unknown rate never costs you more than it did before.

Every response tells you what happened. `usage.prompt_tokens_details.cached_tokens` is how much of your prompt was served from cache, and `usage.cost_without_cache` appears only when a request actually saved money, so you can see the difference per call rather than guess at it monthly.

```json
"usage": {
  "prompt_tokens": 12480,
  "completion_tokens": 210,
  "prompt_tokens_details": { "cached_tokens": 11900 },
  "cost": 0.000452,
  "cost_without_cache": 0.001361
}
```

To get hits, keep the stable part of your prompt first and the variable part last. Caching keys on the prefix, so anything that changes early invalidates everything after it.

---

# Response caching

> Opt-in replay of an identical request, billed at zero.

Source: https://rapidrouter.ai/docs/response-cache

Different from prompt caching below: this replays the **whole response**, and a replay is completely free. Opt in per request with the `X-RR-Cache: true` header (`X-OpenRouter-Cache` is accepted for drop-in) and optionally set `X-RR-Cache-TTL` in seconds, default 300, up to 86400. Non-streaming chat completions only.

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-RR-Cache: true" \
  -H "X-RR-Cache-TTL: 600" \
  -d '{ "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Hello" }] }'

# first response:  X-RR-Cache-Status: miss   (billed normally, then stored)
# same body again: X-RR-Cache-Status: hit    (identical body, billed $0)
```

The cache key is the SHA-256 of your request body, minus gateway-only fields like `user` and `session_id`. JSON property order matters, like OpenRouter: the same fields serialised in a different order are a different entry, so keep your client's serialisation stable. Entries are scoped to your account and never shared, error responses are never stored, and a hit still appears in your usage log, at a cost of zero.

---

# Free tier

> Which models are free, the daily limits per account, and the 429 when they run out.

Source: https://rapidrouter.ai/docs/free-tier

A model is free on Rapid Router when its provider charges us nothing for it, and then it is free to you too. Nothing else is sponsored: every other model is billed at the provider's price, including older models whose id still ends in `:free` (the models page shows each one's price). A free request bills `$0`, works with a **zero balance**, and goes through the same routing, streaming and logging as everything else. Models that can be called free carry `"variants": ["free"]` in `GET /models`; using `:free` on any other model returns `400`.

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-31b-it:free",
    "messages": [{ "role": "user", "content": "Hi" }]
  }'

# usage.cost is 0. No credits are debited.
```

Limits are per account, per UTC day: **50 free requests** a day, raised to **1,000** once your lifetime paid usage reaches $10. Going over returns `429` with code `free_tier_exhausted`; the counter resets at midnight UTC. Rate limits and the concurrency ceiling still apply.

A free model whose catalog id itself ends in `:free` (tagged Free on the models page, priced $0) does not count against the per-account limit above. Should we ever price a model at $0 that its provider bills us for, that model shares a platform-wide daily limit, which resets daily at 18:30 UTC; when that is reached it returns `429` with code `free_pool_exhausted` until the reset.

---

# Errors

> Status codes, the one error envelope, and the health endpoint worth monitoring.

Source: https://rapidrouter.ai/docs/errors

| Code | What it means |
| --- | --- |
| 400 | Malformed request. model and messages are both required |
| 401 | Missing, revoked or malformed API key |
| 402 | Credit balance exhausted, or the key hit its monthly budget cap or its limit |
| 403 | The account or the key's workspace hit its budget, or your data policy rules out every route for the model |
| 404 | Unknown model. GET /models lists what is available |
| 429 | Over the per-minute limit, or too many requests already in flight |
| 500 | Something failed on our side. The message carries a request id to quote when reporting |
| 502 | The upstream provider failed or could not be reached |
| 503 | We list this model but its provider is not connected on this deployment yet |

Every error, a 500 included, is the same envelope: `{ "ok": false, "error": { "code", "message" } }`. Internal detail (stack traces, database errors) never appears in a response body; a 500's message contains a request id that maps to the full detail in our logs.

**Monitoring us:** `GET /health` answers if the process is up; `GET /health/inference` additionally runs the same database queries a chat completion runs and answers `503` when that path is broken. If you page on our availability, ping `/health/inference`, it is the one that tells the truth about whether inference works.

---

# Rate limits

> Requests per minute per key and requests in flight per account, both growing with your balance.

Source: https://rapidrouter.ai/docs/limits

Both limits grow with your credit balance. Requests per minute, counted per key, are one a second per dollar of balance: at least 60, at most 1,200. Requests in flight at once, counted per account, are one per $0.05 of balance: at least 8, at most 64. Free (`:free`) requests stay at 60 a minute and 8 in flight, since they bill nothing.

| Balance | Requests / minute, per key | In flight, per account |
| --- | --- | --- |
| $0.40 or less | 60 | 8 |
| $1 | 60 | 20 |
| $2 | 120 | 40 |
| $5 | 300 | 64 |
| $10 | 600 | 64 |
| $20 or more | 1,200 | 64 |

Each reply carries `X-RateLimit-Limit`, `X-RateLimit-Remaining` and `X-RateLimit-Reset`, and going over returns `429` with a `Retry-After` in seconds. A request beyond the in-flight ceiling gets `429` until one finishes. The per-minute limit counts per key, so one runaway job cannot starve your other keys, while the concurrency ceiling counts per account, because that is what protects the balance itself.

Running a batch workload (an OCR queue, a backfill) that needs more than your balance gives? Write to us and we raise the limit on that specific key; the new ceiling shows in `X-RateLimit-Limit` and on the key in the dashboard.

---

# Idempotency

> Retry a request whose response was lost without paying twice, with the Idempotency-Key header.

Source: https://rapidrouter.ai/docs/idempotency

The retry a caller cannot avoid: the request completed, the response was lost on the wire, and sending it again bills a second completion. Send an `Idempotency-Key` header on a non-streaming `POST /chat/completions` and the retry returns the stored first answer instead of running, and charging, again. The replay carries the response header `idempotency-replayed: true` and bills nothing.

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: order-8842-attempt-1" \
  -d '{
    "model": "google/gemini-3.6-flash",
    "messages": [{ "role": "user", "content": "Hi" }]
  }'

# send it again with the same key and body:
# same 200 response, header idempotency-replayed: true, nothing billed
```

The exact semantics:

- **Same key, same body:** the stored 200 replays, with `idempotency-replayed: true`.
- **Same key, different body:** `422` with code `idempotency_mismatch`. Silently answering with a response to a different question would be worse than failing, so use a fresh key per logical request.
- **Same key while the first send is still running:** `409` with code `idempotency_in_flight`. Retry after it finishes.
- **Only 200s are stored.** A failed call stays retryable with the same key.

Keys are scoped to your account, so they can never collide with another tenant's, and they live for **24 hours**. On a streaming request the header is ignored: replaying a stream as a stored blob would change its shape mid-integration.

---

# Tracing

> x-request-id and W3C traceparent on every response, and OTLP export for self-hosted gateways.

Source: https://rapidrouter.ai/docs/tracing

Every response carries two correlation headers. `x-request-id` is the gateway's own id for the request, the same one a 5xx error body references, so quoting it in a support report points us at the exact log line. `traceparent` is a W3C trace context header: if your request already carried a valid `traceparent`, the gateway joins your trace, keeping your trace id and answering with a fresh span id; if it did not, the gateway starts a new trace. Either way your tracing backend and our logs end up speaking about the same trace id.

```bash
# Send a traceparent and the gateway propagates your trace id:
curl -sD - -o /dev/null https://rapidrouter.ai/api/v1/models \
  -H "traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01"

# <- x-request-id: req_...
# <- traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-<fresh span id>-01
```

**Self-hosted collectors.** Two environment variables on the gateway turn on span export: `OTEL_EXPORTER_OTLP_ENDPOINT` makes the gateway POST one span per request, in OTLP/HTTP JSON, to `<endpoint>/v1/traces` in 5 second batches, and `OTEL_EXPORTER_OTLP_HEADERS` (comma-separated `key=value` pairs) adds auth headers for the collector. This is gateway-side tracing of the HTTP request itself: method, route, status and timing. Prompt and completion bodies are never in spans.

---

# Sign in with Rapid Router

> An OAuth PKCE flow so your users pay for your app with their own credits.

Source: https://rapidrouter.ai/docs/oauth

Let your users pay for your app's inference with **their own** Rapid Router credits, no key copy-pasting. An OAuth PKCE flow: you send the user to our consent page, they approve, and you exchange the returned code for an API key bound to **their** account. The key is a normal runtime key, visible in their dashboard and revocable by them at any time.

1. Generate a random `code_verifier`, hash it to a `code_challenge` (S256), and send the browser to `https://rapidrouter.ai/oauth/authorize`.
2. The user signs in if needed and approves. The browser lands back on your `callback_url` with `?code=...` (or `?error=access_denied` if they refuse).
3. POST the code and your verifier to `/v1/auth/keys`. No session or auth header needed: the code plus the matching verifier is the proof. The response carries the key secret, shown exactly once.

redirect:

```javascript
// 1. verifier + challenge with WebCrypto, then redirect
const bytes = crypto.getRandomValues(new Uint8Array(32))
const verifier = btoa(String.fromCharCode(...bytes))
  .replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "")
sessionStorage.setItem("rr_verifier", verifier)

const digest = await crypto.subtle.digest("SHA-256",
  new TextEncoder().encode(verifier))
const challenge = btoa(String.fromCharCode(...new Uint8Array(digest)))
  .replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "")

location.href = "https://rapidrouter.ai/oauth/authorize?"
  + new URLSearchParams({
      callback_url: "https://yourapp.com/callback",
      code_challenge: challenge,
      code_challenge_method: "S256",
      app_name: "Your App",
    })
```

exchange:

```javascript
// 2. back on your callback_url with ?code=...
const code = new URLSearchParams(location.search).get("code")

const res = await fetch("https://rapidrouter.ai/api/v1/auth/keys", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    code,
    code_verifier: sessionStorage.getItem("rr_verifier"),
  }),
})
const { data } = await res.json()
data.key // "sk-rr-v1-..." -- shown once, store it now
// (also at the top level as .key, as OpenRouter's flow returns it)
```

`callback_url` must be https, or `http://localhost` (any port) for local development; anything else is rejected before the consent screen renders. Codes expire after **10 minutes** and are **single-use**: a second exchange of the same code answers `400 code_consumed`, a stale one `400 code_expired`, and a wrong verifier `400 verifier_mismatch`. The exchange endpoint is limited to 10 attempts a minute per IP. Keys created this way are named `OAuth: <app name>` in the user's dashboard, and revoking one there cuts your app off immediately.

---

# App attribution

> The HTTP-Referer and X-Title headers, and the public Apps ranking they feed.

Source: https://rapidrouter.ai/docs/attribution

Two optional headers tell us which app a request came from: `HTTP-Referer`, your app's URL, and `X-Title`, its display name. Both are read by the gateway and stripped before forwarding. Apps that send them are listed publicly on the [Apps page](https://rapidrouter.ai/apps), ranked by 30-day token volume, once they pass a small traffic floor.

```bash
curl https://rapidrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "HTTP-Referer: https://yourapp.example" \
  -H "X-Title: Your App" \
  -H "Content-Type: application/json" \
  -d '{ "model": "google/gemini-3.6-flash", "messages": [{ "role": "user", "content": "Hi" }] }'
```

---

# Teams

> Organisations, roles, invite links, per-member or shared balances, and per-member spend.

Source: https://rapidrouter.ai/docs/orgs

An organisation groups accounts so a team can see who spent what. Create one from the dashboard's Team pane, invite people by link, and the usage view shows 30 days of per-member spend, requests and tokens, read from the same rows billing uses. One organisation per account. Credits are **per member** by default, each person topping up and spending their own balance; the owner can switch the organisation to a **shared balance** from the Team pane, after which every member's requests spend from the owner's credits.

- **Roles.** The creator is the owner; everyone else is `admin` or `member`. Owner and admins invite and remove people and see everyone's spend; a plain member sees the roster but only their own numbers. Only the owner changes roles, and the owner cannot leave or be removed.
- **Invites.** An invite is a link, not an email: the token is shown once at creation, only its hash is stored, and it expires after 7 days or on first use. Joining while already in an organisation answers `409`.

```bash
# Session-authenticated (dashboard login), not API-key:
POST   /v1/orgs                        { "name": "Acme" }
GET    /v1/orgs/mine                   # org, members + 30d spend, invites
POST   /v1/orgs/:id/invites            { "email": "...", "role": "member" }
DELETE /v1/orgs/:id/invites/:inviteId  # revoke an unused invite
POST   /v1/orgs/join                   { "token": "..." }
PATCH  /v1/orgs/:id/members/:userId    { "role": "admin" }   # owner only
DELETE /v1/orgs/:id/members/:userId    # remove someone, or yourself to leave
GET    /v1/orgs/:id/usage              # 30 days, per member
```

---

# Data collection

> What is recorded per request, and why prompts and responses are not stored unless you turn on capture.

Source: https://rapidrouter.ai/docs/data

We do not store your prompts or the model's responses. Not encrypted, not for a short window. The request body is forwarded to the provider and never written to our database. The single exception is [request capture](https://rapidrouter.ai/docs/capture), which stores bodies only for a key whose owner has explicitly turned it on, and only for the retention window that key chose.

What we do record, per request, is the metadata needed to bill you and show you your own usage:

- Which model and provider handled it
- Response status and latency
- Token counts in and out, and the resulting cost
- Timestamp, and which of your keys was used

Account records hold your email, name and an optional organisation. Sign in events hold IP, the city and country derived from it, and the browser user agent, kept as a security trail. Prompt storage exists only as the opt-in request capture above, off unless a key's owner turns it on. The full detail is in the [privacy policy](https://rapidrouter.ai/legal/privacy).

---

# Provider logging

> Where a prompt goes after the gateway: links to each provider's own policy.

Source: https://rapidrouter.ai/docs/provider-data

This is the part that actually decides where your data ends up. We do not keep your prompt, but we do send it to the provider you selected, and once it arrives their policy governs it, not ours. Choosing a model is choosing a jurisdiction and a retention rule.

The broad pattern across the industry: paid API tiers generally do not train on your requests, most providers retain them briefly for abuse monitoring, and free tiers are where that changes. We are not going to restate each provider's terms here, because they change and a stale copy on our site would be worse than no copy. Read the source:

| Provider | Policy |
| --- | --- |
| Anthropic | [www.anthropic.com/legal/privacy](https://www.anthropic.com/legal/privacy) |
| OpenAI | [openai.com/policies/privacy-policy](https://openai.com/policies/privacy-policy) |
| Google AI Studio | [ai.google.dev/gemini-api/terms](https://ai.google.dev/gemini-api/terms) |
| Together AI | [www.together.ai/privacy](https://www.together.ai/privacy) |
| Groq | [groq.com/privacy-policy](https://groq.com/privacy-policy/) |
| DeepInfra | [deepinfra.com/privacy](https://deepinfra.com/privacy) |
| Novita | [novita.ai/legal/privacy-policy](https://novita.ai/legal/privacy-policy) |
| Fireworks | [fireworks.ai/privacy-policy](https://fireworks.ai/privacy-policy) |
| Cerebras | [www.cerebras.ai/privacy-policy](https://www.cerebras.ai/privacy-policy) |
| SambaNova | [sambanova.ai/privacy-policy](https://sambanova.ai/privacy-policy) |
| Nebius | [nebius.com/privacy-policy](https://nebius.com/privacy-policy) |
| DeepSeek | [www.deepseek.com](https://www.deepseek.com/) |
| Mistral | [mistral.ai/terms](https://mistral.ai/terms) |
| xAI | [x.ai/legal/privacy-policy](https://x.ai/legal/privacy-policy) |
| Moonshot | [platform.moonshot.ai](https://platform.moonshot.ai/) |
| Z.ai | [z.ai](https://z.ai/) |
| Qwen (Alibaba) | [www.alibabacloud.com/help/en/model-studio](https://www.alibabacloud.com/help/en/model-studio/) |
| MiniMax | [www.minimax.io](https://www.minimax.io/) |
| Sarvam AI | [www.sarvam.ai](https://www.sarvam.ai/) |

If a prompt is sensitive enough that retention matters, check the provider's terms before you route to it, and prefer a paid tier.

---

# Request capture

> Opt-in, per-key storage of request and response bodies for debugging, deleted on a schedule.

Source: https://rapidrouter.ai/docs/capture

**Capture is off by default.** We promise not to store prompt and completion bodies, and that promise holds unless you, the key's owner, flip capture on for your own debugging. When on, the gateway stores the request body and the final response next to the usage log row for non-streaming chat completions made with that key, keeps them for the retention window the key chose, then deletes them.

enable:

```bash
# Session-authenticated (dashboard login), like the key editor.
curl -X PATCH https://rapidrouter.ai/api/v1/keys/key_... \
  -H "Authorization: Bearer <session JWT>" \
  -H "Content-Type: application/json" \
  -d '{ "capture": true, "capture_days": 7 }'

# capture_days must be 1, 7 or 30. Omitted, retention is 7 days.
# "Keep forever" is deliberately not expressible.
```

read:

```bash
# The id is the same id the usage log returns for the request.
curl https://rapidrouter.ai/api/v1/logs/<id>/body \
  -H "Authorization: Bearer <session JWT>"

# -> { "ok": true, "data": { "request": {...}, "response": {...},
#      "expires_at": "2026-09-06T..." } }
# No stored body (capture off, or expired) answers 404 "not_captured".
```

Retention is enforced by a sweep that deletes expired bodies, not by a policy document. Stored bodies are readable only by the account that owns the key, via `GET /v1/logs/:id/body`. Streaming requests are not captured.

---

# API reference

> Base URL, authentication, the error envelope, and every endpoint in one table.

Source: https://rapidrouter.ai/docs/reference

Everything here follows the standard chat-completions shape. If your client already talks to `/v1/chat/completions`, two changes are enough: the base URL, and the model string.

- **Base URL**: `https://rapidrouter.ai/api/v1`
- **Authentication**: `Authorization: Bearer $RAPIDROUTER_API_KEY` on every request ([keys](https://rapidrouter.ai/docs/auth))
- **Errors**: `{ "ok": false, "error": { "code", "message" } }` ([codes](https://rapidrouter.ai/docs/errors))
- **OpenAPI 3.1**: [rapidrouter.ai/api/openapi.json](https://rapidrouter.ai/api/openapi.json)
- **Docs as Markdown**: [llms.txt](https://rapidrouter.ai/llms.txt) (index) and [llms-full.txt](https://rapidrouter.ai/llms-full.txt) (every page)

## Endpoints

Paths are relative to the base URL. **key** is an API key as a bearer token, **session** the dashboard login, used by the account endpoints, and **management** a management key.

| Method | Path | Auth | Docs |
| --- | --- | --- | --- |
| POST | `/chat/completions` | key | [Chat completions](https://rapidrouter.ai/docs/chat) |
| POST | `/responses` | key | [Responses API](https://rapidrouter.ai/docs/responses) |
| POST | `/messages` | key | [Anthropic Messages](https://rapidrouter.ai/docs/messages) |
| POST | `/embeddings` | key | [Embeddings and rerank](https://rapidrouter.ai/docs/embeddings) |
| POST | `/rerank` | key | [Embeddings and rerank](https://rapidrouter.ai/docs/embeddings) |
| POST | `/chat/fanout` | key | [Fanout](https://rapidrouter.ai/docs/fanout) |
| POST | `/tokenize` | none | [Token counting](https://rapidrouter.ai/docs/tokenize) |
| POST | `/images/generations` | key | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/images/edits` | key | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/audio/speech` | key | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/audio/transcriptions` | key | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/translate` | key | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/videos/generations` | key | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/videos/generations/status` | key | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/voice/chat` | key | [Voice pipeline](https://rapidrouter.ai/docs/voice) |
| GET | `/media/models` | none | [Images, speech and video](https://rapidrouter.ai/docs/media) |
| POST | `/batches` | key | [Batch API](https://rapidrouter.ai/docs/batch) |
| GET | `/batches/:id` | key | [Batch API](https://rapidrouter.ai/docs/batch) |
| GET | `/batches/:id/results` | key | [Batch API](https://rapidrouter.ai/docs/batch) |
| POST | `/batches/:id/cancel` | key | [Batch API](https://rapidrouter.ai/docs/batch) |
| GET | `/models` | none | [Models](https://rapidrouter.ai/docs/models) |
| GET | `/models/:author/:slug/endpoints` | none | [Endpoints and uptime](https://rapidrouter.ai/docs/uptime) |
| GET | `/models/:author/:slug/uptime` | none | [Endpoints and uptime](https://rapidrouter.ai/docs/uptime) |
| GET | `/presets` | session | [Presets](https://rapidrouter.ai/docs/presets) |
| GET POST | `/provisioning/keys` | management | [Provisioning API](https://rapidrouter.ai/docs/provisioning) |
| GET PATCH DELETE | `/provisioning/keys/:id` | management | [Provisioning API](https://rapidrouter.ai/docs/provisioning) |
| POST | `/auth/keys` | none | [Sign in with Rapid Router](https://rapidrouter.ai/docs/oauth) |
| GET PUT | `/budget` | session | [Budgets](https://rapidrouter.ai/docs/budgets) |
| GET PUT | `/alerts` | session | [Spend alerts](https://rapidrouter.ai/docs/alerts) |
| GET PUT | `/radar` | session | [Release radar](https://rapidrouter.ai/docs/radar) |
| GET | `/usage/export` | session | [Usage export](https://rapidrouter.ai/docs/usage-export) |
| GET | `/audit` | session | [Audit log](https://rapidrouter.ai/docs/audit) |
| GET | `/logs/:id/body` | session | [Request capture](https://rapidrouter.ai/docs/capture) |
| POST | `/orgs` | session | [Teams](https://rapidrouter.ai/docs/orgs) |
| GET | `/orgs/mine` | session | [Teams](https://rapidrouter.ai/docs/orgs) |
| GET | `/orgs/:id/usage` | session | [Teams](https://rapidrouter.ai/docs/orgs) |

---

# Chat completions

> POST /chat/completions: the standard request and response, plus provider and usage.cost.

Source: https://rapidrouter.ai/docs/chat

`POST /chat/completions`. The response envelope is the standard one, with an extra `provider` field telling you which upstream actually served it.

```json
{
  "id": "msg_011CeHkp77djK1hiHwaAy22u",
  "object": "chat.completion",
  "model": "anthropic/claude-sonnet-5",
  "provider": "anthropic",
  "choices": [
    { "index": 0, "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }
  ],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 96,
    "total_tokens": 114,
    "cost": 0.000996
  }
}
```

---

# Responses API

> POST /responses in the OpenAI Responses shape: stateless, on every model, streaming included.

Source: https://rapidrouter.ai/docs/responses

`POST /responses` speaks the OpenAI Responses API shape: `input` as a string or an array of items, `instructions`, flat function tools, and typed output items (`message`, `function_call`, `reasoning`), with an `output_text` convenience field. It runs through the same routing, failover and billing as chat completions, on every model in the catalog.

It is **stateless**. Responses are never stored, so `store: true` and `previous_response_id` are rejected with `400`, and `GET /responses/:id` is always `404`. To continue a conversation, resend the previous output items in `input`, the way the multi-turn example below feeds a `function_call_output` back. Echo tool-call items exactly as they arrived: some models attach fields (Gemini 3's `extra_content.google.thought_signature`) that must survive the round trip.

curl:

```bash
curl https://rapidrouter.ai/api/v1/responses \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "instructions": "Answer briefly.",
    "input": "What is the capital of France?",
    "max_output_tokens": 100
  }'

# -> { "id": "resp_...", "object": "response", "status": "completed",
#      "output": [ { "type": "message", "content": [ { "type": "output_text", "text": "Paris." } ] } ],
#      "output_text": "Paris.",
#      "usage": { "input_tokens": 14, "output_tokens": 3, "cost": 0.000058 } }
```

javascript:

```javascript
import OpenAI from 'openai'

const rapidrouter = new OpenAI({ baseURL: 'https://rapidrouter.ai/api/v1', apiKey: process.env.RAPIDROUTER_API_KEY })

const r = await rapidrouter.responses.create({
  model: 'anthropic/claude-sonnet-5',
  input: 'What is the capital of France?',
})
console.log(r.output_text)

// Multi-turn is client-side: resend the prior output items in input.
const followUp = await rapidrouter.responses.create({
  model: 'anthropic/claude-sonnet-5',
  input: [
    { role: 'user', content: 'What is the capital of France?' },
    ...r.output,
    { role: 'user', content: 'And its population?' },
  ],
})
```

`stream: true` emits Responses-API events: `response.created`, `response.output_item.added`, an `response.output_text.delta` per text delta, then `response.output_text.done` and a final `response.completed` carrying the full response with usage and cost. Function calls arrive as complete items before `response.completed`.

---

# Anthropic Messages

> POST /messages in the native Anthropic format, so Anthropic SDKs work with a base URL change.

Source: https://rapidrouter.ai/docs/messages

`POST /v1/messages` speaks the native Anthropic Messages API: content blocks, a top-level `system`, tools with `input_schema`, `tool_choice`, `thinking` budgets, `stop_sequences` and a required `max_tokens`. Auth is your normal Rapid Router key, as a bearer token or in `x-api-key`, which is where Anthropic's SDKs put it. The request runs through the same routing, failover and billing as chat completions, and the answer comes back in the Anthropic shape, streaming included, whichever provider actually served it, so a client built on Anthropic's SDK works by changing the base URL and key. The reply's `usage.cost` and `provider` say what was billed and who served it; on a stream they ride the final `message_delta`.

```bash
curl https://rapidrouter.ai/api/v1/messages \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "max_tokens": 256,
    "messages": [{ "role": "user", "content": "Hi" }]
  }'
```

Provider routing preferences pass through the same `provider` object as chat completions, and `:free` works here too. Validation errors from this endpoint use Anthropic's error envelope; errors from deeper in the pipeline keep the gateway's usual shape.

---

# Embeddings and rerank

> POST /embeddings and POST /rerank, billed on input tokens.

Source: https://rapidrouter.ai/docs/embeddings

`POST /embeddings`, the standard shape. `input` takes a string or an array of strings, and embeddings bill input tokens only. Embedding models carry the same `vendor/model` ids as everything else and are listed under the Embeddings tab of the catalog; they never appear in `GET /models`, which is chat only. `encoding_format: "base64"` and `dimensions` pass through where the provider supports them.

curl:

```bash
curl https://rapidrouter.ai/api/v1/embeddings \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": ["first document", "second document"]
  }'
```

javascript:

```javascript
import OpenAI from 'openai'

const rapidrouter = new OpenAI({ baseURL: 'https://rapidrouter.ai/api/v1', apiKey: process.env.RAPIDROUTER_API_KEY })
const r = await rapidrouter.embeddings.create({
  model: 'openai/text-embedding-3-small',
  input: ['first document', 'second document'],
})
console.log(r.data[0].embedding.length) // 1536
```

response:

```json
{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.0138, -0.0219, ...] },
    { "object": "embedding", "index": 1, "embedding": [0.0042, 0.0177, ...] }
  ],
  "model": "openai/text-embedding-3-small",
  "provider": "openai",
  "usage": { "prompt_tokens": 4, "total_tokens": 4, "cost": 0.00000009 }
}
```

Rerankers ride the same catalog: `POST /rerank` with a `query` and `documents` returns them scored against the query, best first. Set `top_n` to cap how many come back and `return_documents: true` to echo the text alongside each score.

```bash
curl https://rapidrouter.ai/api/v1/rerank \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3-reranker-0.6b",
    "query": "how do I rotate an API key?",
    "documents": ["Keys are rotated from the dashboard.", "Bananas are yellow."],
    "top_n": 1
  }'
```

---

# Fanout

> One prompt to up to 4 models in parallel, each result billed on its own.

Source: https://rapidrouter.ai/docs/fanout

One prompt, up to **4 models**, one call. `POST /chat/fanout` takes a normal chat completion body plus a `models` array in place of `model`, runs the request against every model in parallel, and returns all the answers together.

```bash
curl https://rapidrouter.ai/api/v1/chat/fanout \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "models": ["google/gemini-3.6-flash", "anthropic/claude-sonnet-5"],
    "messages": [{ "role": "user", "content": "Name three rivers." }]
  }'

# -> { "ok": true, "data": { "results": [
#      { "model": "google/gemini-3.6-flash", "status": 200,
#        "body": { "object": "chat.completion", ... } },
#      { "model": "anthropic/claude-sonnet-5", "status": 200,
#        "body": { "object": "chat.completion", ... } }
#    ] } }
```

Each model runs as a real request through the full pipeline, key checks, allowlists, budgets, rate limits, routing, logging, and **each result is billed individually** at its model's normal price, exactly as if you had made the calls separately. One model failing does not sink the rest: its entry carries the error status and body while the others complete. Streaming is rejected with `400`; call `/chat/completions` per model when you want streams.

---

# Token counting

> POST /tokenize counts tokens without calling a model; exact for OpenAI models, estimated for the rest.

Source: https://rapidrouter.ai/docs/tokenize

`POST /v1/tokenize` counts the tokens in a string or a chat messages array without calling a model. It is unauthenticated on purpose, counting tokens reveals nothing about an account, and rate limited to **30 requests a minute per IP** with at most 1,000,000 characters of input per request.

text:

```bash
curl https://rapidrouter.ai/api/v1/tokenize \
  -H "Content-Type: application/json" \
  -d '{ "model": "openai/gpt-4o", "text": "How many tokens is this?" }'

# -> { "ok": true, "data": { "tokens": 6, "exact": true, "encoding": "o200k_base" } }
```

messages:

```bash
curl https://rapidrouter.ai/api/v1/tokenize \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [
      { "role": "system", "content": "Answer briefly." },
      { "role": "user", "content": "Hi" }
    ]
  }'

# message scaffolding is included: about 4 tokens per message plus 3
# priming the reply, per OpenAI's published chat recipe.
```

estimate:

```bash
curl https://rapidrouter.ai/api/v1/tokenize \
  -H "Content-Type: application/json" \
  -d '{ "model": "anthropic/claude-sonnet-5", "text": "How many tokens is this?" }'

# -> { "ok": true, "data": { "tokens": 7, "exact": false, "encoding": null } }
```

OpenAI-family models count exactly via tiktoken and report `exact: true` with the encoding used. Every other family gets a characters divided by 3.6 estimate, rounded up and clearly labelled `exact: false` with `encoding: null`: a made-up exact count for a tokenizer we do not have would be worse than an honest estimate.

---

# Models

> GET /models: ids, prices, context length, modalities and capabilities, live.

Source: https://rapidrouter.ai/docs/models

A model string is `vendor/model`, the same shape everywhere in the catalog. `GET /models` returns what we carry along with the per-million price we bill at. The `available` flag is the one to check before you call: it is true when the model's provider is connected on this deployment, and false when we list and price the model but cannot serve it yet.

```json
{
  "object": "list",
  "data": [
    {
      "id": "anthropic/claude-sonnet-5",
      "object": "model",
      "owned_by": "anthropic",
      "available": true,
      "pricing": { "input_per_m": 2, "output_per_m": 10 },
      "context_length": 200000,
      "max_output_tokens": 64000,
      "input_modalities": ["text", "image"],
      "output_modalities": ["text"],
      "capabilities": { "tools": true, "json": true, "reasoning": true,
                        "vision": true, "audio": false }
    }
  ]
}
```

The capability block is for choosing a model programmatically: filter on `capabilities.vision` to find models that take images, `input_modalities` for audio or PDF-capable ones, `context_length` to fit a long prompt. Where we have no verified metadata for a model the fields are simply absent, never guessed, so their presence can be trusted.

The catalog is live, and so is your key: when a new model or provider is connected it is callable immediately with the key you already have. Nothing to regenerate, nothing to redeploy; use the new model string and it works. The same applies to price changes, which take effect per request.

---

# Endpoints and uptime

> Per-provider price, latency, throughput and 72 hours of availability for one model.

Source: https://rapidrouter.ai/docs/uptime

Two public endpoints show a model's routes and how they have been behaving. `GET /v1/models/:author/:slug/endpoints` lists every provider serving the model with its price (cache read and write rates too, where the provider publishes them), whether it is connected right now, and its last 24 hours: request count, median latency, and throughput in output tokens per second. Those numbers come from served traffic. A route with fewer than 5 successful requests in the window shows the result of an hourly test request instead, a short prompt asking for about 20 tokens, and `stats.source` says which it is: `traffic` or `probe`. A route with neither reports zeros and nulls, not guesses.

endpoints:

```bash
curl https://rapidrouter.ai/api/v1/models/google/gemini-3.6-flash/endpoints

# -> { "data": { "id": "google/gemini-3.6-flash", "endpoints": [
#      { "provider": "google",
#        "pricing": { "input_per_m": 0.75, "output_per_m": 3.75,
#                     "cache_read_per_m": 0.075, "cache_write_per_m": null },
#        "healthy": true,
#        "stats": { "requests_24h": 1284, "p50_latency_ms": 912,
#                   "throughput_tps": 84.1, "source": "traffic" } },
#      ...
#    ] } }
```

uptime:

```bash
curl https://rapidrouter.ai/api/v1/models/google/gemini-3.6-flash/uptime

# 72 hourly buckets per provider, oldest first. availability is
# up / (up + down): up is a 200, down an upstream 5xx or a failed
# hourly test request. An hour with neither is null rather than
# dressed up as 100% or 0%.
# -> { "data": { "id": "google/gemini-3.6-flash", "providers": [
#      { "provider": "google", "buckets": [
#        { "hour": "2026-08-23T09:00:00.000Z", "availability": 1 },
#        { "hour": "2026-08-23T10:00:00.000Z", "availability": null },
#        ...
#      ] } ] } }
```

`GET /v1/models/:author/:slug/uptime` covers the last 72 hours. Client errors (4xx) count as neither up nor down, since a malformed request says nothing about the provider. The hourly test requests count too: one that times out, errors or is refused (401, 402, 403, 404 or a 5xx) is down. An unknown model answers `404` on both endpoints.

---

# Provisioning API

> Create, limit, rotate and disable runtime keys with a management key.

Source: https://rapidrouter.ai/docs/provisioning

Manage runtime API keys programmatically: issue a key per customer, rotate keys on a schedule, disable a key the moment a customer churns. It is driven by a **management key** (`sk-rr-mgmt-...`), created in the dashboard under Keys, which authenticates `/provisioning/*` and nothing else: a management key is rejected by every inference endpoint, and a runtime key is rejected by the provisioning endpoints. You can hold up to 5 active management keys.

create:

```bash
curl https://rapidrouter.ai/api/v1/provisioning/keys \
  -H "Authorization: Bearer sk-rr-mgmt-..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "customer-42",
    "limit_usd": 10,
    "limit_reset": "monthly",
    "allowed_models": ["google/*"]
  }'

# -> { "id": "key_...", "secret": "sk-rr-v1-...", "limit_usd": 10, ... }
# The secret is returned exactly once and never stored.
```

rotate:

```bash
# Rotation is create-then-delete: mint the replacement, hand it out,
# then revoke the old key.
curl https://rapidrouter.ai/api/v1/provisioning/keys \
  -H "Authorization: Bearer sk-rr-mgmt-..." \
  -H "Content-Type: application/json" \
  -d '{ "name": "customer-42 (rotated)" }'

curl -X DELETE https://rapidrouter.ai/api/v1/provisioning/keys/key_OLD \
  -H "Authorization: Bearer sk-rr-mgmt-..."
```

disable:

```bash
# Disabled is reversible, unlike a revoke. The key answers 401
# "key_disabled" on every inference call until re-enabled.
curl -X PATCH https://rapidrouter.ai/api/v1/provisioning/keys/key_... \
  -H "Authorization: Bearer sk-rr-mgmt-..." \
  -H "Content-Type: application/json" \
  -d '{ "disabled": true }'
```

`GET /provisioning/keys` lists the account's runtime keys (hashes are never returned), `GET /provisioning/keys/:id` returns one with its live limit state (`limit_spent_usd`, `limit_remaining_usd`, `limit_resets_at`), and `PATCH` accepts the same fields as the dashboard editor: `name`, `limit_usd`, `limit_reset`, `monthly_cap_usd`, `allowed_models`, plus `disabled`. Keys created here are ordinary runtime keys: they show up in the dashboard, spend the same credit balance and obey the same limits.

---

# Usage export

> Your usage as CSV, one row per day, key, model, provider and status.

Source: https://rapidrouter.ai/docs/usage-export

`GET /v1/usage/export?from=2026-08-01&to=2026-08-31` downloads your usage as CSV, session-authenticated, one row per day (days start at 18:30 UTC), key, model, provider and status. Columns: `date`, `key_id`, `model`, `provider`, `status`, `requests`, `tokens_in`, `tokens_out`, `tokens_cached`, `cost_usd`.

Dates are `YYYY-MM-DD`, `from` must be on or before `to`, and the range is capped at **92 days**, a quarter, so ask for shorter windows and stitch them if you need more.

---

# Release radar

> Get told by email, WhatsApp or webhook when a new model becomes callable.

Source: https://rapidrouter.ai/docs/radar

Subscribe and get told when a new model id becomes callable on the platform. `GET /v1/radar` reads your subscription, `PUT /v1/radar` sets it, session-authenticated.

```bash
curl -X PUT https://rapidrouter.ai/api/v1/radar \
  -H "Authorization: Bearer <session JWT>" \
  -H "Content-Type: application/json" \
  -d '{ "email_enabled": true, "wa_enabled": false }'

# -> { "ok": true, "data": { "subscribed": true,
#      "email_enabled": true, "wa_enabled": false } }

# Unsubscribe entirely:
#   -d '{ "subscribed": false }'
# WhatsApp uses the alert_phone from your spend alert settings.
```

A sweep every 6 hours compares the live catalog against what it has already seen. Email and WhatsApp are **batched**: one message per sweep listing the new models (the first 10 by name, then a count). The `model.added` webhook event fires once per new model with `{ "model_id": "..." }`, so automation gets one event per model rather than a digest to parse. The very first sweep seeds silently, so turning the feature on never announces the whole existing catalog.

---

# Audit log

> Who created, changed or removed which key, budget or team member, and when.

Source: https://rapidrouter.ai/docs/audit

Every account-shaping mutation writes an audit event: who did it, what it was, and to what. Recorded actions: `key.create`, `key.update`, `key.delete`, `budget.update`, `byok.add`, `byok.remove`, `org.invite.create`, `org.member.add` and `org.member.remove`. Inference requests are not audit events; they are the usage log.

```bash
# Your own trail, newest first, last 100 events:
curl https://rapidrouter.ai/api/v1/audit \
  -H "Authorization: Bearer <session JWT>"

# -> { "ok": true, "data": [
#      { "id": "aud_...", "actor_email": "you@example.com",
#        "action": "key.create", "subject": "key_...",
#        "detail": { "name": "customer-42" }, "created_at": "..." },
#      ...
#    ] }

# The organisation's trail:
curl https://rapidrouter.ai/api/v1/orgs/<org_id>/audit \
  -H "Authorization: Bearer <session JWT>"
```

`GET /v1/audit` returns your own events. `GET /v1/orgs/:id/audit` is gated like the other organisation reads: you must be a member, the owner and admins see everyone's events, and a plain member sees only the events they caused themselves. Writes are best effort by design: a failed audit insert never fails or slows the mutation it describes.