| Metric | Per day | Per month |
|---|
What actually costs money with retries?
429 errors themselves are free — no tokens are billed when a request is rate-limited before processing starts. The real costs are:
- Retry requests — each retry sends the full input prompt again. If the retry succeeds, you pay for the tokens. If your input is 2,000 tokens, 3 retries on 3% of requests add 0.09 extra inputs per successful request.
- Timeout duplicates — when a request takes too long and your code retries, but the original was actually processed. You pay for two completions. This is especially costly with large-output requests.
- Failed requests — if all retries are exhausted and the request fails, you wasted latency and any partial input token cost if the provider charges on receipt.
Fix it: Use a local token-per-minute rate limiter (e.g. LangChain's ContextWindowExceededHandler or a simple token bucket) to smooth bursts before they hit the API. This eliminates most 429s without increasing cost.
Related: API budget planner · Cost forecast · Free tier runway
Frequently asked questions
Do I get charged for API calls that return a 429 rate limit error?
No — 429 errors are rejected before processing and you are not charged for them. The real cost is retry logic, delayed responses, and sometimes duplicate completions if retry logic has bugs.
What is exponential backoff and why does it increase latency cost?
Exponential backoff means waiting 1s, 2s, 4s, 8s... between retry attempts. A single 429 with 2 retries adds 3–7 seconds of latency. If downstream systems have SLAs, this has real operational cost even if the token cost is zero.
How do retries cause duplicate charges?
If a request succeeds but times out before the response arrives, a naive retry resends — and if the first succeeded, you pay for two completions. Always use idempotency keys or check for existing results before retrying on timeout.
What is the average 429 rate for LLM APIs?
Bursty traffic can hit 10–30% error rates. Steady traffic well below limits typically sees less than 0.1%. Most providers now use token-per-minute limits, which punish large-context requests more than request-count limits.
How do I reduce API retry costs?
Four approaches: (1) Rate limit locally with a token bucket. (2) Use a queue with controlled concurrency. (3) Set up async processing so retries happen in the background. (4) Cache responses for identical prompts.