Training vs running, month by month
Cumulative cost of the fine-tuned path (training + higher inference) against staying on the base model with the longer prompt.
| Month | Fine-tuned (cumulative) | Base model (cumulative) | Ahead? |
|---|
Fine-tuning's bill arrives twice
The number everyone quotes is the training cost: dataset tokens × epochs × the provider's per-token training rate. It's real but it's the smaller half. The half that compounds is inference — a fine-tuned model is billed at a higher per-token rate than the base model, often roughly double, on every single request for as long as you run it. At low volume the training fee dominates; at high volume the inference premium quietly becomes the whole story. A model serving millions of calls a month can pay more in its premium than the training ever cost.
The thing that pays it back: a shorter prompt
Fine-tuning earns its keep when it lets you delete a long few-shot prompt. If you were stuffing 600 tokens of examples into every call to get the right format, and the fine-tuned model produces that format from a one-line instruction, you save those input tokens forever. Set "prompt tokens saved" above and the calculator credits that saving against the higher rate — that's the break-even it solves for. If the saving outweighs the premium, fine-tuning gets cheaper the more you use it; if not, you're paying more per call for the privilege.
Before you fine-tune, try the cheaper paths
Fine-tuning is maintenance: re-training when data drifts, versioning, evals. Often a bigger base model with a good prompt or retrieval (RAG) already hits the quality bar with zero training cost and nothing to maintain. Price that path on the RAG cost calculator, see how a reusable system prompt cuts input cost on the prompt caching savings calculator, and compare base model rates on the LLM token cost calculator.
Related tools & guides
LLM token cost calculator · Embeddings cost calculator · RAG cost calculator · Prompt caching savings · All AI APIs