What it costs to fill a large context window every call.
Long context windows are marketed as a feature; each one is input tokens you pay for repeatedly. Trim what you resend and cache the rest — see the prompt caching calculator and RAG chatbot cost.
Free context-window cost calculator — how much filling a big context (RAG, long docs) costs per call and per month.
You pay input price on every token in the context, every single call. A 32k-token prompt at $2.50/1M is $0.08 of input before the model writes anything — and that repeats on each request.
Retrieve fewer and smaller chunks, summarize history, and cache the static portion of the prompt. Context size is usually the single biggest lever on a RAG or long-doc bill.