Batch API

19Batch API

Work that does not need an answer right now can be queued. Submit up to 500 chat completions in one call and they run within a 24 hour window at standard per-token pricing, through the same routing and fallbacks as live traffic. Requests are inline JSON, no file upload step, and results are retained for 30 days. Streaming is not supported inside batch items.

curl https://rapidrouter.ai/api/v1/batches \
  -H "Authorization: Bearer $RAPIDROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      { "custom_id": "row-1", "body": { "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Summarise: ..." }] } },
      { "custom_id": "row-2", "body": { "model": "google/gemini-3.6-flash",
        "messages": [{ "role": "user", "content": "Summarise: ..." }] } }
    ]
  }'

# -> { "id": "batch_...", "object": "batch", "status": "queued",
#      "discount_pct": 0, "request_counts": { "total": 2, "completed": 0, "failed": 0 },
#      "expires_at": 1756300000 }

Each item is billed individually at the model's normal per-token price, visible in your usage log like any other request. An item that fails does not sink the batch: the rest complete and the failed line carries the error. Anything still unfinished when the 24 hour window closes is marked expired and never billed. POST /batches/:id/cancel stops a batch; items already served keep their results. Results are deleted 30 days after the batch was created, so collect them before then.