How prompt caching cuts your LLM bill by up to 90%

Published 2026-06-21 · reference numbers, verify before budgeting

If your app sends the same long system prompt, the same retrieved documents, or the same growing chat history on every call, you're paying full price to re-read text the model has already seen seconds ago. Prompt caching is the fix, and it's the single biggest line-item discount most teams never turn on. Done right it cuts input cost by 50% to 90%. Here's how it works, the 2026 numbers, and where it quietly doesn't help.

What prompt caching actually does

An LLM bill is mostly input tokens × price + output tokens × price. Caching attacks the input half. When you mark a chunk of the prompt as cacheable, the provider stores its processed form for a short window; on the next call that reuses the exact same prefix, you're billed a fraction of the normal input rate for those tokens instead of the full amount. The output price is untouched — caching only discounts the part you keep resending.

The 2026 discounts (reference)

Provider	Cached input price	Notes
OpenAI	~50% of input	Automatic on long prompts, no code change
Anthropic (Claude)	~10% of input on reads	Up to 90% off — but ~25% write premium first time
Google Gemini	~25% of input	Context caching; may add a small storage fee

⚠️ Reference figures, June 2026 — exact discounts, cache lifetimes and any storage fees change. Confirm on each provider's pricing page. Anthropic and Gemini use explicit cache markers; OpenAI applies it automatically to qualifying prefixes. · Report outdated price →

A worked example

Say a support assistant sends a 2,000-token system prompt + knowledge base on every message, plus ~200 tokens of the user's actual question, and returns ~300 output tokens. At 10,000 conversations a month, the fixed 2,000-token block is re-sent 10,000 times. Using a model at $2.50 / 1M input:

Without caching: 2,000 × 10,000 = 20M input tokens on the static block = $50/mo just for the repeated prefix
With caching at ~50% off (OpenAI-style): ~$25/mo — half, for flipping one setting
With caching at ~90% off (Claude reads): ~$5/mo on that block

Scale that to 100k or 1M conversations and the static prefix is the difference between a comfortable bill and an alarming one. The bigger and more repeated your fixed context, the more caching is worth — RAG apps and long-system-prompt agents benefit most.

When caching does NOT help (the gotchas)

The prefix must be identical. Cache matches on the exact repeated prefix. Put anything variable (a timestamp, the user's name, a random ID) before the static block and you break the match — every call misses. Keep variable content at the end.
The write premium. Anthropic charges ~25% extra the first time it caches a block. If a prefix is used only once or twice before the cache expires, you can pay more, not less. Caching wins on reuse, not novelty.
Short cache lifetime. Caches live minutes, not hours. Low-traffic apps with long gaps between calls may never get a cache hit.
Output isn't cached. If your cost is dominated by long generated answers, caching barely moves the bill — trim output instead.

The rule of thumb

Cache when you re-send a large, identical block of context frequently and quickly: system prompts, tool definitions, RAG documents, few-shot examples, long chat history. Don't bother for one-off calls, highly variable prompts, or output-heavy workloads. Structure the prompt static-first, variable-last, and the discount is close to free money. Put a number on your own case with the prompt caching savings calculator, and compare models on the AI API cost calculator.

FAQ

How much can prompt caching save?

Typically 50–90% off the input price for the cached portion, depending on provider. It only discounts repeated input tokens, not output.

Does prompt caching change the model's answers?

No. It's a billing/latency optimization — the model sees the same tokens, you just pay less to re-send the cached prefix. Responses are unchanged.

Why did caching make my bill go up?

Almost always the write premium (e.g. Anthropic's ~25% surcharge to create the cache) on prefixes that aren't reused enough before expiry, or a prefix that changes every call so it never hits. Caching pays off on frequent, identical reuse.

Reference estimates — always verify discounts and fees on the provider's official pricing page.