What it costs to fill a large context window every call.
Long context windows are marketed as a feature; each one is input tokens you pay for repeatedly. Trim what you resend and cache the rest β see the prompt caching calculator and RAG chatbot cost.
The Context Window Cost Calculator estimates what it costs to fill a large context window on each model call β the scenario behind RAG pipelines and long-document prompts. It multiplies your context tokens per call by the input price per million tokens, adds your output tokens per call times the output price per million, and scales the total by your calls per month to show both per-call and monthly figures. The dominant driver is almost always the context size: input tokens repeat on every call, so a heavy retrieval payload or a long system prompt sets the baseline cost far more than the answer you get back.
The key trade-off is context depth versus spend. Doubling retrieved passages or padding the prompt with extra documents can multiply your bill without improving answers. Use the calculator to test whether trimming context, tightening retrieval, or switching to a cheaper input tier lowers the monthly total, and watch the per-call number when volume is high β small per-call amounts compound quickly across thousands of requests.
You pay input price on every token in the context, every single call. A 32k-token prompt at $2.50/1M is $0.08 of input before the model writes anything β and that repeats on each request.
Retrieve fewer and smaller chunks, summarize history, and cache the static portion of the prompt. Context size is usually the single biggest lever on a RAG or long-doc bill.