—total bill / mo
—is the preamble
—cache saves / mo
Where your input tokens go
The monthly bill split into the fixed preamble you re-send every time, the dynamic user and context tokens, and the output. The first row is the tax — it does not shrink when users say less.
| Part | Tokens / request | Cost / mo | Share |
|---|
How prompt size scales the tax
Each row is a different system-prompt length at your current volume. The uncached column is the naive cost; the cached column applies your hit rate and discount. This is why a prompt that grew from 500 to 4,000 tokens quietly multiplied your bill.
| System prompt | Uncached / mo | Cached / mo | vs yours |
|---|
The prompt you wrote once, you pay for a million times
A system prompt feels free because you write it once and forget it, but the API is stateless, so that preamble rides along on every single call and is billed as input tokens each time. The trap is short, high-volume requests: a classifier or a router that receives twenty tokens of user text and a two-thousand-token instruction block is spending 99% of its input budget on words the user never sent. The fix is boring and effective — read your system prompt as if you were paying per word, because you are, and cut the polite filler, the duplicated rules and the tool schemas you almost never invoke. Then cache what survives, since a stable preamble is exactly what prompt caching was built for and a cached read costs roughly a tenth of a fresh one. The reason trimming comes first is that it helps unconditionally, on cache misses and hits alike, while caching only pays off on the requests that arrive while the cache is warm and only if the prefix stays byte-for-byte identical — one timestamp near the top and the discount evaporates. Size the caching lever precisely on the prompt caching savings calculator, check the write-versus-read break-even on the cache write break-even calculator, and count the tokens in your current prompt with the token counter.
How this calculator works
It counts your monthly requests as requests per day times 30.4. Every request pays for the system prompt plus the dynamic input at the input rate, and the output at the output rate. The system-prompt cost is its token count times monthly requests at the input rate; caching reduces the hit-rate share of that to the discounted rate while the miss share stays full price. Total bill sums the fixed preamble, the dynamic input and the output. The share is the system-prompt cost over that total, and the cache saving is the uncached preamble cost minus the cached one. The scaling table re-runs the preamble cost at several prompt lengths so you can see how it grows, and the trim figure applies your percentage cut to the current prompt.
Frequently asked questions
Why does the system prompt cost money on every request?
Because the API is stateless — it does not remember your previous call — so anything you want the model to know each time has to be sent each time. The system prompt, persona, few-shot examples and tool schemas all live in that fixed preamble, and every one is counted as input tokens on every request. A user who types a single word still pays for the entire preamble that rode along with it, which is why a bloated system prompt behaves like a fixed tax charged in full on the smallest and largest requests alike.
How much of my bill is the system prompt?
It depends on the ratio of your fixed preamble to the variable content of each request. A two-thousand-token prompt against a five-hundred-token user turn is most of your input tokens and can dominate the bill — especially on short, high-volume calls like classification where there is barely any user text to dilute it. If your requests carry long documents or chat histories, the preamble is a smaller slice. The dangerous case is high volume plus short requests plus a long prompt.
Does prompt caching remove the system prompt cost?
It removes most of it, on the calls that hit the cache. Caching stores a stable prefix — your system prompt is the perfect candidate — and bills reads against it at a steep discount, commonly around 90% off, so a cached system prompt costs roughly a tenth of an uncached one. The catch is that only requests arriving while the cache is warm get the discount, there is sometimes a small write premium, and the prefix has to be byte-for-byte identical, so a timestamp near the top breaks it.
Is it better to trim the system prompt or cache it?
Do both, but they solve different problems. Trimming cuts the tokens in the preamble, lowering cost on every request including cache misses, and frees context window. Caching leaves the prompt long but makes repeated reads cheap on warm calls. Trimming is the more robust win because it helps unconditionally, while caching depends on your traffic keeping the cache warm. Strip anything not earning its tokens first, then cache what remains.