The three levers, and why they stack
Most "cut your LLM costs 90%" advice is real — it just only applies to part of your traffic, and the headline numbers assume the best case. This calculator makes the share explicit. Prompt caching discounts the repeated prefix of your input (system prompt, RAG context, a long document) to 10–50% of the input rate; it does nothing for output tokens or for input that changes every call. Batch processing gives a flat 50% on any request that can run asynchronously — great for classification, summarization and report generation, useless for a live chat. Model routing sends the easy requests to a smaller, far cheaper model and keeps the frontier model for the hard ones. Applied to the slices of traffic each one can actually reach, they compound into a genuinely large reduction.
Why the effective discount is lower than the headlines
If 90% caching, 50% batch and 85% routing all hit your whole bill you'd pay almost nothing — but they don't. Caching only touches cacheable input; batch only touches async traffic; routing only touches requests a cheaper model can handle. Set those shares honestly and the calculator returns the blended discount across your real workload. That blended number — often 40–70% rather than 90% — is the one to put in a budget. Validate the individual levers with the prompt caching calculator, the batch savings calculator and the model routing calculator.
Order of operations matters
The levers interact: caching reduces input cost first, then batch and routing apply to what's left, so stacking them isn't simple addition. This tool applies caching to the cacheable input slice, then applies batch and routing discounts to their shares of the remaining spend, which is why the combined discount is less than the sum of the parts. Before optimizing, get your baseline right with the LLM token cost calculator and find the cheapest base model with cheapest LLM API.
How to use it
1. Enter your current monthly LLM spend and roughly what share is input vs output tokens.
2. Set how much of your input is repeated/cacheable and your provider's cache-hit price (10% Anthropic, 25–50% OpenAI).
3. Set the share of traffic you can batch and the share you can route to a cheaper model.
4. Read the optimized bill and effective discount — then implement the levers with the biggest "Saves" rows first.
FAQ
Do caching and batching stack?
Yes — a batched request can also use a cached prefix. This tool applies caching to input first, then batch/routing discounts to the remaining spend, so the combined effect is multiplicative, not additive.
What cache-hit price should I use?
Anthropic bills cache reads at 10% of the input rate; OpenAI at 25–50% depending on model; Google context caching at ~10%. Use your provider's figure for the cacheable-input slice.
Is routing risky for quality?
Only if you route hard requests to a weak model. The safe pattern is a classifier or rules layer that sends simple/short requests to the cheap model and escalates the rest. Model the spend split with the routing calculator.
Estimate only. Discounts depend on your exact traffic mix and provider pricing — verify current rates before budgeting.