—
savings using GPU rental vs managed API—managed provider cost
—GPU rental cost
—GPU hours needed
Managed provider rates vs GPU rental, at your dataset size
Same training-token count as above (dataset × avg tokens × epochs), priced against each managed provider's published LoRA rate, next to your GPU rental settings.
Cost by dataset size
Everything else held at the values above, swept across dataset size (training examples), comparing the selected managed rate against your GPU rental settings. The highlighted row is closest to your current dataset size.
| Examples | Training tokens | Managed cost | GPU cost | Cheaper option |
|---|
How this connects to other tools
This calculator prices one specific decision: whether to pay a managed provider's per-token rate for LoRA/QLoRA training, or rent a GPU yourself and run the job directly. If your job is a full fine-tune — updating every parameter, not a small set of low-rank adapters — that's a different cost shape entirely, priced against OpenAI, Google and Mistral's own training-token rates on the fine-tuning cost calculator; for full fine-tuning cost across those providers, see that tool instead of this one. If you're weighing fine-tuning against just prompting a base model well, the prompt vs fine-tune calculator and RAG vs fine-tuning pages cover that earlier decision. And once a LoRA-tuned model is trained, serving it long-term is its own build-vs-buy question — the self-hosted LLM vs API calculator prices that ongoing inference decision the same way this page prices the one-time training decision.
How this calculator works
The LoRA / QLoRA Fine-Tuning: Managed API vs GPU Rental Calculator starts from the total training workload: dataset size (training examples) × avg tokens per example × epochs gives the total training tokens the job has to process — at the defaults, 50,000 × 512 × 2 is 51,200,000 training tokens. On the managed side, that token count divided by 1,000,000 and multiplied by the selected provider's rate per 1M training tokens gives the managed provider cost — $24.58 at the Together AI default. On the GPU rental side, the same training-token count divided by (training throughput in tokens/sec × 3,600) gives the GPU hours needed, which multiplied by the GPU rate per hour gives the raw compute cost; adding setup hours × your hourly rate (optional, $0 by default) gives the full GPU rental cost — about $1.79 at the A100 default with $0 setup cost.
Subtracting GPU cost from managed cost gives the savings from renting a GPU yourself, and dividing that by the managed cost gives the cost gap percentage — roughly 92.7% cheaper via GPU rental at the defaults. This can go negative: if your throughput estimate is too low, your GPU rate too high, or you set a nonzero hourly rate for enough setup hours, the managed API can come out cheaper, and this calculator shows that honestly rather than assuming DIY always wins. The training throughput field is the single most sensitive input — it varies enormously by model parameter count, so the 15,000 tokens/sec default is only a starting estimate; replace it with a number benchmarked on your actual model size and GPU before trusting the dollar output for a real budget decision.
Frequently asked questions
Is LoRA fine-tuning cheaper via API or renting a GPU?
At typical dataset sizes, renting a GPU yourself and running the LoRA/QLoRA training job directly is usually far cheaper in raw dollar terms than paying a managed provider's per-token training rate. At this calculator's defaults (50,000 examples, 512 tokens/example, 2 epochs), a managed LoRA job on Together AI ($0.48 per 1M training tokens) costs about $24.58, while the same 51.2 million training tokens on a rented A100 80GB at $1.89/hr and 15,000 tokens/sec throughput costs about $1.79 — over 90% less. But that gap is a raw compute-cost comparison, not a full total-cost-of-ownership one: it doesn't include your time setting up the training pipeline, handling checkpointing, or debugging failures, none of which a managed API makes you do. Whether GPU rental is actually cheaper for you depends on how much you value that setup time, which is why this calculator has an optional setup-hours field you can raise above zero to see the picture change.
What GPU do I need for LoRA fine-tuning?
For most LoRA/QLoRA jobs on models up to roughly 13B-34B parameters, a single 80GB GPU — an A100 80GB or H100 80GB — is enough, because LoRA only trains a small set of added low-rank adapter weights rather than the full model, and QLoRA quantizes the frozen base weights to 4-bit to shrink memory further, so the whole job fits on one card instead of the multi-GPU cluster a full fine-tune would need. Larger base models or longer context windows push memory needs up and may require a smaller batch size, more aggressive quantization, or a bigger GPU, and actual training throughput (tokens/second) varies a lot by parameter count. This calculator defaults to 15,000 tokens/sec as an illustrative estimate, but you should adjust it to a number benchmarked on your own model size and GPU before trusting the dollar output.
Why is there such a big cost gap between managed and DIY?
Managed LoRA training APIs like Together AI and Fireworks aren't just billing for raw GPU-seconds — the per-token rate bundles infrastructure reliability, job orchestration, automatic checkpointing, queueing and retries, and a support team you can escalate to if a training run fails partway through, none of which you get renting a bare GPU instance yourself. That bundle carries a real markup over the underlying compute cost, and because a rented A100 or H100 billed by the hour has no such markup baked in beyond the cloud provider's own margin, the gap between the two can look enormous when you only compare the compute-cost line. The honest caveat is that DIY isn't free in the way that comparison makes it look — someone has to write and debug the training script, manage checkpointing, and handle failures with no support line to call, which is real time even if it never shows up as a per-token charge.
What's the difference between LoRA and full fine-tuning cost?
Full fine-tuning updates every parameter in the model, which is why our full fine-tuning calculator models training cost as dataset tokens times epochs times a provider training rate on top of a base model you're modifying entirely — and it typically needs enough GPU memory (or managed-training budget) to hold gradients and optimizer state for the whole parameter count, which is expensive at any real model size. LoRA and QLoRA instead freeze the base model and train a small number of added low-rank adapter weights, so the compute and memory footprint — and therefore the cost, on both the managed-API and GPU-rental side — is a small fraction of full fine-tuning for a comparable dataset. That's why this calculator and the full fine-tuning calculator use different cost shapes: full fine-tuning is priced primarily against provider training rates because DIY full fine-tuning is rarely practical on a single rented GPU, while LoRA/QLoRA is cheap and small enough that renting a single GPU yourself becomes a realistic, often far cheaper, alternative to a managed API.