What a context window is

The context window is the model's working memory for a single request, measured in tokens. It has to hold everything the model considers at once: your system prompt, any documents or examples you include, the conversation history, the current question, and the space reserved for the model's answer. Together those must fit within the window's token limit.

Models advertise windows of varying sizes, from a few thousand tokens to hundreds of thousands or more. A bigger window lets you feed in more at once, an entire book, a large codebase, but it does not change the fundamental billing rule: you pay for what you put in.

You pay for the whole window you use, every call

Here is the part that catches people out. Having a 200,000-token window does not mean you pay for 200,000 tokens. It means you can use up to that many, and you are billed for however many you actually include, on every single call.

So if you stuff 150,000 tokens of documents into the window and ask 10 questions, you pay for roughly 150,000 input tokens ten times, 1.5 million tokens, not once. The window is a ceiling, not a subscription. Long context is powerful, but it is billed relentlessly per call.

Why long prompts get expensive fast

Because input tokens are billed per call, prompt length multiplies with call volume. A prompt that is 10x longer costs roughly 10x more in input tokens for the same number of calls. Two habits push prompts long without people noticing:

• Growing chat history, resent in full on every turn, so late messages in a long conversation carry the entire transcript.
• Retrieval (RAG), where you paste large retrieved documents into the prompt to give the model context. Retrieve too much and every query pays for all of it.

Neither is wrong, but both need discipline, because the cost is silent until the invoice arrives.

A worked illustration

Illustrative example (invented rate of $3 per million input tokens). Suppose each query includes a 50,000-token retrieved context. One query costs 50,000 / 1,000,000 x $3 = $0.15 in input alone. Run 100,000 such queries a month and that is $15,000 just for the context you are re-sending, before output costs.

Now trim the retrieved context to 10,000 tokens by retrieving more selectively. The same 100,000 queries cost $3,000, a fifth as much, for context that is often just as useful because the model was drowning in the extra 40,000 tokens anyway. This is why context discipline is one of the highest-leverage cost controls you have.

Long context can also hurt quality

More context is not always better answers. Models can lose track of details buried in the middle of a very long prompt, a pattern where information at the start and end is used well but the middle is neglected. So a bloated context window can cost more and produce worse results.

Feeding the model only the most relevant material often improves both the bill and the answer. Precision beats volume: a tight, well-chosen 4,000-token context frequently outperforms a sprawling 100,000-token dump.

How to keep context lean

Practical tactics to control context cost:

• Retrieve selectively. In RAG, return only the top few most relevant chunks, not everything that vaguely matches.
• Summarize old turns. Replace long past conversation with a short running summary instead of resending the full transcript.
• Cache the fixed part. If a large chunk of context repeats across calls, prompt caching (see that guide) can slash the cost of re-sending it.
• Right-size the window. Choose a model whose window matches your real needs rather than defaulting to the largest available.

To see how prompt length maps to real money for each model, put your token counts into the LLM cost calculator and check per-model window pricing on the model comparison and /models pages.

Continue learning

Related tools

Frequently asked questions

Do I pay for the model's full context window?

No. You pay only for the tokens you actually include in each request, up to the window's limit. The window is a maximum capacity, and every token you use inside it is billed on every call.

Does a bigger context window cost more per token?

Not necessarily per token, but a bigger window tempts you to include far more text, and since every token is billed per call, larger prompts drive up the total. Some providers also charge higher rates above certain context lengths.

Is more context always better for quality?

No. Models can overlook details buried in very long prompts, so excess context can both raise cost and lower answer quality. A smaller, well-chosen context often performs better.