A cache only helps if the next request beats the clock. If your average idle gap between requests is g minutes and the cache TTL is T minutes, the steady-state hit rate is 1 − e−T/g. A 5-minute cache and a 1-hour cache on the exact same code can have wildly different hit rates purely because of timing — and a miss pays the write premium instead of the cheap read.
No cache vs 5-minute vs 1-hour TTL
Same traffic, three strategies. Hit rate is computed from your idle gap; a miss pays the write premium (1.25× for 5-min, 2.0× for 1-hour), a hit pays the read rate.
| Strategy | Cache hit rate | Cost / request | Cost / month | vs no cache |
|---|
Why the hit rate is a timing problem, not a guess
Prompt caching bills in three states: a cache write (first time a prefix is seen, or after it expired) at a premium over the base token price, a cache read (the prefix is still alive) at a deep discount, and plain uncached tokens at full price. Which state you land in on any given request is decided by one thing: did the previous request happen recently enough that the cached prefix is still alive? Model the idle gap between requests as memoryless — the standard assumption for independent arrivals — and the probability the gap is shorter than the TTL T is exactly 1 − e−T/g, where g is your average gap. That's your hit rate. It's why a support bot handling one message every few minutes gets a great hit rate on a 1-hour cache but a poor one on a 5-minute cache, running the identical code.
The 5-minute vs 1-hour trade-off
The 1-hour cache isn't strictly better — it costs more to write (around 2.0× base versus 1.25× for the 5-minute tier). The question is whether the longer lifetime saves enough expensive misses to pay for the pricier writes. When your idle gap is small relative to five minutes (tight bursts), the cheap 5-minute write already gives you a near-100% hit rate and the 1-hour premium is pure waste. When your gap is minutes-long, the 5-minute cache expires constantly and re-pays the write on almost every request, while the 1-hour cache stays warm — so despite the higher write multiplier the 1-hour tier ends up cheaper. The crossover depends entirely on your g, which is why guessing a flat hit rate misleads you. This tool finds the crossover for your numbers.
What changed in 2026 and how to respond
When a provider shortens the default cache lifetime — the widely-felt move from roughly an hour down to five minutes — any workload with spaced-out requests silently slips from mostly-reads to mostly-writes, and bills climb 30–60% with no code change. Two responses work: keep the cache warm with a lightweight heartbeat request inside the shorter window, or explicitly opt into a longer-TTL tier when the write premium pays for itself. Decide which by running your real idle gap and request volume above, then compare with the prompt caching savings calculator for the flat-hit-rate view and the cache write break-even calculator for the one-shot reuse case.
How this calculator works
It computes your steady-state cache hit rate as 1 − e^(−T/g) from your average idle gap g and each tier's TTL T (5 minutes or 60 minutes), then prices every request as a blend of cache reads (hits) and cache writes (misses) on the cached prefix, plus full-price uncached tokens. It compares no-cache, the 5-minute tier and the 1-hour tier, and reports the cheapest for your traffic and volume.
Frequently asked questions
How do I know my real prompt-cache hit rate?
You derive it from how often requests arrive versus how long the cache lives. With an average idle gap of g minutes and a cache TTL of T minutes, the steady-state hit rate is 1 − e^(−T/g). The same code can have a 10% hit rate on a 5-minute cache and a 95% hit rate on a 1-hour cache purely because of timing.
Is the 1-hour prompt cache worth the higher write cost?
It depends on your idle gap. The 1-hour cache writes at ~2× base versus ~1.25× for 5-minute, but a longer TTL avoids expensive misses. Minutes-apart requests usually favour the 1-hour tier; tight bursts a few seconds apart are fine on the cheaper 5-minute write.
Why did my prompt-caching bill go up in 2026?
Because the default cache lifetime got shorter. Moving the standard TTL from ~1 hour to ~5 minutes makes any traffic with idle gaps longer than a few minutes miss the cache and re-pay the write premium, quietly raising bills 30–60% for spaced-out workloads.