Published 2026-08-07 · reference numbers, verify before budgeting
OpenAI, Anthropic and Gemini all run the same trade: submit a job instead of a request, wait up to 24 hours, pay roughly half. Same model, same weights, same output quality — the only thing that changes is when the answer shows up. Teams treat this as a volume decision, something you turn on once a workload gets big enough to bother with. It isn't. It's an interaction decision, and it has nothing to do with size.
All three major providers batch the same way: upload a file of prompts, the job runs asynchronously within a 24-hour window, and the per-token price comes in at roughly 50% of the standard synchronous rate. It isn't a smaller or dumber model doing the work — it's the identical model, generating the identical output it would have generated in real time. The saving comes from scheduling, not from cutting a corner: providers hold real-time capacity ready for traffic spikes, and that idle headroom is expensive. A batch job can run whenever spare capacity exists instead, so the provider passes part of that efficiency back.
The math doesn't need a calculator to feel the shape of it: a job that costs $200 at the standard rate costs $100 through the batch endpoint, for the same output. On a workload you run once, that's a nice line item. On a workload you run every night for a year, it's the difference between a real budget item and a rounding error.
Classifying 1 million short documents — 500 input tokens and 20 output tokens each — against a reference rate of $2.50 / $10.00 per 1M tokens (input/output; illustrative reference pricing, confirm your model's actual rate before budgeting):
| Path | Input cost | Output cost | Total |
|---|---|---|---|
| Synchronous (standard rate) | $1,250.00 | $200.00 | $1,450.00 |
| Batch (~50% off) | $625.00 | $100.00 | $725.00 |
| Saved | $725.00 (50%) | ||
| 1M docs × 500 in / 20 out tokens. Reference pricing — run your own rate on the batch API savings calculator. | |||
$725 saved on one classification pass. Run that same job nightly for a re-indexing pipeline and the batch endpoint has paid for itself in the first week — for zero change in output quality, because nothing about what the model produces is different.
Every provider's batch documentation uses the same careful word: target. The 24-hour window is not an SLA. Jobs frequently finish faster — sometimes in minutes — but nothing obligates that, and a pipeline built assuming a fast turnaround will eventually sit waiting for hours with no recourse. Very large batches can also hit size or rate limits, which means a genuinely huge job may need to be split into several submissions rather than one, adding orchestration a synchronous call never needed in the first place.
None of that matters for a job nothing is waiting on. It matters completely for a job a person is staring at a spinner for. That's the actual dividing line — not "is this workload big enough," but "is anything blocked on this response right now."
Skip the volume threshold. Ask one question: is a human or a downstream system waiting synchronously for this specific response? If no — a nightly embeddings backfill, a bulk reclassification, a backlog of documents nobody's watching in real time — batch is close to a free 50% cut, full stop, take it. If yes — a chatbot reply, a live search result, anything rendering behind a loading spinner — batch isn't a discount you can access at all, regardless of how much volume would justify it on paper. The size of the job never enters the decision. Only whether something is waiting on it does.
That also means the same application can use both paths at once: synchronous for the live chat turn a user is watching, batch for the overnight job that re-embeds everything that chat touched that day. Price the synchronous side with the prompt caching savings calculator if it's a repeat-context workload, and the batch side with the batch API savings calculator — they're solving different halves of the same bill. For the mechanics of what counts as a batch job and which providers support what, see batch API explained; for whether your pipeline can actually tolerate the wait, the batch processing time calculator prices the turnaround side directly.
Methodology: the ~50% batch discount and 24-hour target window reflect OpenAI, Anthropic and Gemini's published batch API terms as of August 2026. The worked example uses a round, clearly-labeled reference rate ($2.50/$10.00 per 1M tokens) rather than any single provider's exact current price — confirm your model's actual synchronous and batch rates before budgeting. The example is a constructed scenario meant to show how the discount scales, not telemetry from a production system.