How to cut LLM API costs
Eight changes that lower an AI API bill, biggest saving first.
1. Use the smallest model that does the job
Prices between a flagship and a small model differ by ten to fifty times. Extraction, classification, routing, summaries and short replies rarely need the flagship. Test a small model on a sample of real traffic, keep the big one for the requests that fail. The best cheap model ranking lists the cheapest models that still rate well on independent votes.
2. Let a router pick per prompt
Sending rapidrouter/auto as the model lets the gateway classify each prompt locally and route light chat to the cheapest live model, code to the strongest coding model and heavy work to a strong mid model. No extra model call, and you pay the chosen model's normal price. See the auto router docs.
3. Cache the part of the prompt that never changes
A long system prompt or reference document resent on every call is paid for every time. Providers that support prompt caching charge far less to read a prefix they have seen recently. Put the stable part first and the variable part last; Rapid Router bills cached tokens at the cache rate with no setup and reports the saving on each response (prompt caching).
4. Replay identical requests for free
If the same request repeats, for example a product description generated on every page view, opt in to the response cache with the X-RR-Cache: true header. A hit replays the stored answer at a cost of zero (response cache).
5. Cap the output
Output tokens cost several times more than input, and reasoning models spend output tokens thinking. Set max_tokens, ask for the format you need, and prefer a short JSON object to a paragraph you then parse.
6. Batch the work that can wait
The providers' own batch APIs advertise around half price for jobs that finish within a day, at the time of writing. Rapid Router's batch endpoint runs up to 500 requests in a 24 hour window at standard per-token prices: it saves you writing a queue, not money. For large offline jobs on one provider, compare both.
7. Mind the language
The same message in an Indian language can use several times the tokens of English, and the multiple differs by model. Our Indian-language token study measures it model by model; for Hindi-heavy traffic, that ratio can matter more than the price per token.
8. Put a ceiling on every key
Per-key monthly caps and an account budget are checked before each request, and spend alerts tell you before a ceiling rather than at it. That turns a leaked key or a retry loop into a refused request instead of a bill (budgets).
Common questions
What is the fastest way to cut an LLM bill?
Move the requests that do not need a flagship model to a smaller one. Model choice changes the price by ten times or more; every other lever is measured in percent.
Does prompt caching need code changes?
Usually not. Keep the fixed part of your prompt first. Where the provider caches it, Rapid Router bills the cached tokens at the provider's cache rate, currently 80% to 92% below the input rate where we hold one.
Is a batch API always cheaper?
On the providers' own batch APIs, usually yes: OpenAI, Anthropic and Google advertise around half price for work that can wait a day, at the time of writing. Rapid Router's batch endpoint runs at standard per-token prices; it saves engineering work, not money.
How do I stop a runaway bill?
Set a monthly cap on each key and an account budget. Every request is checked against them first, so a loop or a leaked key stops at the number you chose.