What each one actually does
RAG (Retrieval-Augmented Generation) leaves the model unchanged. At query time you search your own data — usually a vector database of embedded document chunks — pull back the few passages most relevant to the user's question, and paste them into the prompt as context. The model then answers using that injected text. Your knowledge lives outside the model and is looked up fresh on every call.
Fine-tuning changes the model itself. You take a base model and continue training it on your own examples so that the knowledge, tone, or behaviour becomes baked in to the weights. After that, the model produces your style or answers your domain questions without you having to supply the reference material each time — but it only "knows" what it was trained on up to that point.
A short way to remember it: RAG gives the model an open book to read from at answer time; fine-tuning makes the model study until the material is memorised.
The cost structure is completely different
This is where the two approaches diverge most, and why a naïve "which is cheaper" question has no single answer.
RAG — mostly ongoing, per-query cost
- Embedding cost: every document is embedded once (cheap), but many setups also embed each incoming query — a small recurring per-request charge.
- Larger input prompts: you send the retrieved chunks on every call, so your input-token count is inflated on each request. Since you pay per token, this is the biggest ongoing driver.
- Vector database: a hosted vector store or the infrastructure to self-host it — a monthly fixed cost that grows with corpus size.
- Near-zero upfront: no training run, so you can ship in days.
Fine-tuning — upfront cost, leaner per call
- Training cost: a one-time (or periodic) charge to run the fine-tune, priced by tokens in your training set and the number of epochs.
- Hosting / inference: a fine-tuned model often costs more per token to serve than the shared base model, and some providers add a hosting fee to keep your custom model available.
- Smaller prompts: because behaviour and knowledge are baked in, you send far fewer instruction and context tokens per call — the recurring saving that pays back the training cost.
- Re-training on change: when your data shifts, you pay to fine-tune again.
When RAG wins vs when fine-tuning wins
| Choose RAG when… | Choose fine-tuning when… |
|---|---|
| Knowledge changes often (docs, prices, policies) | Style, format, or behaviour is fixed and repeatable |
| Corpus is large and still growing | The task is narrow and stable |
| You need citations / "where did this come from" | You want short prompts and lower latency per call |
| Volume is low to medium | Volume is very high, so smaller prompts amortise training |
| You need to add or remove facts instantly | You need consistent tone the model can't be prompted into |
The two are not mutually exclusive. A common mature setup fine-tunes for behaviour and format while using RAG for current facts — the model reliably answers in your voice and cites live data.
Worked comparison: low volume vs high volume
Illustrative round numbers to show the shape of the trade-off — always confirm with current provider rates. Assume a mid-tier model, RAG adding ~1,500 extra input tokens of retrieved context per call, and a fine-tune that removes ~1,200 tokens of instructions/examples per call for a one-time training cost of about $300 plus a small monthly hosting fee.
| Scenario | RAG | Fine-tuning | Winner |
|---|---|---|---|
| Upfront cost | ~$0 (build vector DB) | ~$300 training | RAG |
| 10,000 calls / month | lower total — extra tokens are cheap at this scale, no training to recoup | higher — $300 spread over few calls dominates | RAG |
| 2,000,000 calls / month | higher — 1,500 extra tokens × 2M adds up every month, forever | lower — training cost is trivial per call, prompts are lean | Fine-tuning |
| Facts change weekly | update the index, no re-train | re-train repeatedly — cost and lag pile up | RAG |
The pattern: at low volume RAG almost always wins because there's no training cost to recover. At very high volume, the per-call token saving from fine-tuning eventually outweighs the fixed training cost — the "break-even" is a volume threshold, not a fixed rule. Model the two side by side before committing.
Fine-tune vs prompting calculator →The honest answer: try the cheap thing first
Fine-tuning feels like the "serious" option, but for most teams it's the wrong first move. Better prompting and RAG solve the majority of "the model doesn't know our stuff" problems faster, cheaper, and with far less to maintain. Fine-tuning adds a training pipeline, versioning, evaluation, and re-training whenever your data drifts — real operational weight.
A sensible order: (1) improve the prompt and add examples; (2) if it needs your data, add RAG; (3) fine-tune only when the numbers clearly favour it — very high volume where lean prompts pay back training — or when you need behaviour that no prompt can reliably produce. Reach for fine-tuning when the evidence says so, not by default. For broader ways to trim spend, see the cost-cutting guide.
Cut your LLM bill →More Learn guides →Frequently asked questions
Is RAG cheaper than fine-tuning?
It depends on volume. RAG has near-zero upfront cost but adds ongoing per-query cost from embeddings, a vector database, and larger input prompts. Fine-tuning has a fixed upfront training cost and usually smaller prompts, so it only becomes cheaper per request at very high volume where the training cost is spread across many calls.
When should I fine-tune instead of using RAG?
Fine-tune when you need a fixed style, tone, format, or behaviour baked in, when your task is stable rather than changing daily, when latency matters and you want short prompts, or when your request volume is high enough that smaller prompts save more than the training cost. For knowledge that changes often, RAG is usually the better fit.
Can I use RAG and fine-tuning together?
Yes, and it is common. Fine-tuning teaches the model your format and behaviour while RAG supplies current facts at query time. Most teams should still start with prompting or RAG first and only add fine-tuning once the numbers or behaviour clearly justify the extra cost and complexity.
Educational reference only — prices are estimates; confirm current rates on each provider's pricing page.