How LLM API Pricing Works

Everything is priced per token

LLM providers don't charge per request or per word — they charge per token. A token is a chunk of text, roughly ¾ of a word in English (about 4 characters). "APICostCalc is useful" is ~5 tokens. Both the text you send (input) and the text the model generates (output) are counted, and they're usually priced differently.

Input vs output — output is the expensive one

On almost every model, output tokens cost more than input tokens — often 3–5× more — because generating text is more compute-intensive than reading it. This is why a chatbot that writes long answers costs far more than one that classifies text into a single word, even at the same request volume.

The context window is a cost ceiling, not a free space

A model's context window (e.g. 128K or 1M tokens) is the maximum it can read at once — but every token you put in it is billed on every call. Stuffing a whole document into context each request is convenient and expensive; that's the problem RAG and caching solve.

Why the same task varies 50× between models

Frontier models can cost tens of dollars per million tokens; small or open models a few cents. If a cheap model handles the task at acceptable quality, you can cut cost by an order of magnitude with one line of config. The skill is matching model strength to task difficulty — see the cost-cutting guide.

Worked example: nano vs flagship on the same task

Say you're classifying 100,000 support tickets a month. Each call sends ~2,000 input tokens (ticket text + a few-shot prompt) and returns ~500 output tokens (label + short explanation) — 200M input tokens and 50M output tokens total.

Same task, same volume — the flagship model costs 62× more than the nano model, purely from per-token pricing. For a bounded task like classification, testing whether a cheap model's accuracy clears your bar is usually worth more than any other cost optimization.

Two ways to cut the bill without switching models

Not every saving comes from picking a cheaper model. Two provider-side mechanisms discount the same model on the same task, just by changing how you call it:

They stack with model choice, not against it: a support-ticket classifier that already runs on a cheap model can often add batch processing on top, since overnight labeling has no latency requirement — see the prompt caching savings calculator and batch API guide for the exact math on your workload.

Common mistakes when estimating LLM costs

• Averaging input and output into one token count. Since output is priced 3–5× higher, blending them hides how much a verbose response style actually costs.
• Forgetting the system prompt is billed every call. A long system prompt or conversation history counts as input tokens on each request, not once per session.
• Benchmarking against the wrong point release. "GPT-5" and "GPT-5.5" can price several times apart — always check the exact model ID you'll actually call, not the family name.
• Ignoring hidden reasoning tokens. "Thinking" models bill internal reasoning as output tokens even though it never appears in the visible response — see the reasoning token calculator.

Continue learning

Frequently asked questions

What is a token in an LLM API?

A token is a small chunk of text the model processes — in English, roughly ¾ of a word or about 4 characters. Providers bill per token for both the text you send (input) and the text the model generates (output).

Why do output tokens cost more than input tokens?

Generating text requires more computation than reading it, so providers price output tokens higher — commonly 3–5× the input rate. This makes verbose model responses a major cost driver.

Does a bigger context window cost more?

Only for the tokens you actually use. The window is a maximum capacity, but every token you place in context is billed on each call, so sending large prompts repeatedly gets expensive fast.

How can the same task cost 50× more on one model?

Model prices per million tokens vary enormously between frontier and small/open models. If a cheaper model meets your quality bar for a given task, switching to it can cut costs by an order of magnitude.

Can I cut costs without switching models?

Yes — prompt caching discounts the repeated prefix of a request by around 90%, and the batch API discounts an entire non-urgent job by around 50%. Both apply to the exact same model, so they stack with any model-choice savings.