What a batch API is

A batch API lets you submit many requests at once as a single job, rather than one at a time in real time. You upload a file containing all your prompts, the provider processes them asynchronously over a window of time, and you download the results when they are ready.

The trade you are making is simple: you give up speed and get a lower price. Instead of a response in seconds, you might wait minutes or hours. In return, the per-token cost is typically cut by around half compared to the standard synchronous rate.

Why providers can offer the discount

Real-time API traffic is spiky and unpredictable. Providers must keep spare capacity ready for demand peaks, and that idle headroom is expensive. Batch jobs are flexible: they can be run whenever spare capacity exists, smoothing out the load.

Because you let the provider schedule your work during quieter periods, they can use hardware more efficiently and pass part of that saving back to you. The discount is not a lower-quality model, it is the exact same model run on a relaxed schedule.

How the workflow looks

The typical batch flow has four steps:

• Prepare a file where each line is one request, usually in a structured format with a custom ID so you can match results back to inputs.
• Submit the batch and receive a job ID.
• Poll for status while the provider works through the queue, often with a stated completion target such as within 24 hours.
• Retrieve the output file once complete, then map each result back to its request using the IDs.

Your application logic barely changes, the same prompts and the same models are used. What changes is that you are not blocking on each response.

Which jobs fit the batch model

Batch is ideal whenever a human is not waiting for the answer:

• Bulk classification or tagging of large datasets.
• Content generation for catalogs, product descriptions, or summaries produced ahead of time.
• Data extraction from large document collections.
• Evaluations and backtesting where you run a prompt across thousands of test cases.
• Embeddings generation for a whole corpus at once.

It is a poor fit for anything interactive: chatbots, live search, or any feature where a user is staring at a loading spinner.

The economics of waiting

The saving is straightforward to reason about. If a job would cost $200 at the standard rate and the batch discount is 50%, it costs $100 instead, for identical output. On large recurring workloads that difference compounds month after month.

Here is an illustrative example (invented rates). Classifying 1 million short documents at, say, 500 input and 20 output tokens each might cost roughly $X at synchronous pricing; the same job through the batch endpoint at half price costs roughly $X/2. The LLM cost calculator lets you enter your token counts and volume, then halve the total to preview the batch scenario against the real-time one.

Things to watch for

A few practical caveats. Completion times are targets, not guarantees, so build your pipeline to tolerate variable delays. Very large batches may be subject to size or rate limits, so you may need to split enormous jobs into chunks. And you still pay for both input and output tokens, the discount applies to the rate, not to the token count, so prompt efficiency still matters.

Finally, check each provider's specifics on the /models pages, since the exact discount, turnaround target, and supported features (like whether caching or tools work inside batch) differ between providers.

Continue learning

Related tools

Frequently asked questions

Is the batch API a weaker model?

No. You get the same model and the same quality of output as the real-time API. The only difference is that results arrive later, which is why the price is lower.

How long does a batch job take?

Providers usually give a completion target such as within 24 hours, and many jobs finish much faster when capacity is available. It is not instant, so only use it when nobody is waiting on the answer.

How much do batch APIs typically save?

Around 50% off the standard per-token rate is common across major providers, applied to both input and output tokens. Always confirm the current discount for your specific provider and model.