Why your LLM app throws 429 errors — the TPM math nobody explains

Published 2026-06-28 · reference limits, verify in your provider dashboard

You read the docs. Your tier allows 500 requests per minute. You're sending maybe eight. And yet the logs are full of 429 Too Many Requests. Nothing is wrong with your code — you're hitting the other limit, the one most tutorials skip: tokens per minute (TPM). Every major LLM provider meters you on RPM and TPM at the same time, and for real prompts the token limit almost always bites first.

Two limits, and the one that actually binds

Take OpenAI's entry tier for GPT-4o: roughly 500 RPM and 30,000 TPM (reference, June 2026). The RPM number looks generous. But a retrieval-augmented prompt with a few documents stuffed in can easily be 4,000 tokens in and 500 out — call it 4,500 tokens per request. Now do the division the docs don't:

Prices are reference estimates, July 2026. Report outdated price →

So your real ceiling is under seven requests a minute, not 500. The token limit caps you at 1.3% of what the request limit implied. That's why the 429s start at single-digit traffic — and why "send slower" or "add a retry" treats the symptom, not the cause.

Turn the limit into something you can plan: concurrent users

Requests-per-minute is abstract. What you actually want to know is how many people can use the app at once. If each active user fires roughly two requests a minute, that 6.7 RPM ceiling is about three concurrent users before requests start queueing. Fine for a demo; a wall for a launch. The fix is rarely a faster loop — it's one of four levers:

• Shrink the prompt. The biggest lever by far. Cutting that 4,000-token context to 1,500 nearly triples your throughput on the same tier — TPM is linear in tokens.
• Cap output. Output tokens count toward TPM too. A chatty model with no max_tokens burns the budget on rambling answers.
• Move non-urgent work to batch. Async batch endpoints have their own, far higher limits and a discount — at the cost of latency.
• Climb a tier. Limits rise automatically as you spend. OpenAI's Tier 2 jumps GPT-4o to ~450,000 TPM — a 15× lift that quietly dissolves the wall.

It's not just OpenAI

The same trap exists everywhere, sometimes tighter. Anthropic's entry Claude tier is around 50 RPM / 40,000 TPM; Google's free Gemini 2.5 Flash tier is generous on tokens (~250,000 TPM) but stingy on requests (~10 RPM). The binding limit flips depending on whether your prompts are big or your traffic is spiky — which is exactly why a single rule of thumb fails. You have to plug in your token size.

That's what our new rate limit calculator does: enter a tier (or your real TPM/RPM), your input and output tokens, and how often each user calls, and it tells you the effective requests per minute, the concurrent users you can serve, and which limit is the bottleneck. It's the fastest way to find out whether a launch will hit a wall before your users do.

The takeaway

429 errors below your RPM limit are not a bug — they're the token-per-minute limit doing its job. Size capacity on tokens, not request counts; trim context before you reach for retries; and remember the limit rises with spend. Once you know your real ceiling, price the traffic with the LLM token cost calculator and model the whole app on the AI app cost estimator.

Rate limits are reference figures (June 2026) and vary by provider, model and account tier — always confirm in your dashboard. Related: Rate Limit Calculator · Batch API Savings · Cheapest LLM API.