Question 1

Is LoRA fine-tuning cheaper via API or renting a GPU?

Accepted Answer

At typical dataset sizes, renting a GPU yourself and running the LoRA/QLoRA training job directly is usually far cheaper in raw dollar terms than paying a managed provider's per-token training rate. At this calculator's defaults (50,000 examples, 512 tokens/example, 2 epochs), a managed LoRA job on Together AI ($0.48 per 1M training tokens) costs about $24.58, while the same 51.2 million training tokens on a rented A100 80GB at $1.89/hr and 15,000 tokens/sec throughput costs about $1.79 — over 90% less. But that gap is a raw compute-cost comparison, not a full total-cost-of-ownership one: it doesn't include your time setting up the training pipeline, handling checkpointing, or debugging failures, none of which a managed API makes you do. Whether GPU rental is actually cheaper for you depends on how much you value that setup time, which is why this calculator has an optional setup-hours field you can raise above zero to see the picture change.

Question 2

What GPU do I need for LoRA fine-tuning?

Accepted Answer

For most LoRA/QLoRA jobs on models up to roughly 13B-34B parameters, a single 80GB GPU — an A100 80GB or H100 80GB — is enough, because LoRA only trains a small set of added low-rank adapter weights rather than the full model, and QLoRA quantizes the frozen base weights to 4-bit to shrink memory further, so the whole job fits on one card instead of the multi-GPU cluster a full fine-tune would need. Larger base models or longer context windows push memory needs up and may require a smaller batch size, more aggressive quantization, or a bigger GPU, and actual training throughput (tokens/second) varies a lot by parameter count. This calculator defaults to 15,000 tokens/sec as an illustrative estimate, but you should adjust it to a number benchmarked on your own model size and GPU before trusting the dollar output.

Question 3

Why is there such a big cost gap between managed and DIY?

Accepted Answer

Managed LoRA training APIs like Together AI and Fireworks aren't just billing for raw GPU-seconds — the per-token rate bundles infrastructure reliability, job orchestration, automatic checkpointing, queueing and retries, and a support team you can escalate to if a training run fails partway through, none of which you get renting a bare GPU instance yourself. That bundle carries a real markup over the underlying compute cost, and because a rented A100 or H100 billed by the hour has no such markup baked in beyond the cloud provider's own margin, the gap between the two can look enormous when you only compare the compute-cost line. The honest caveat is that DIY isn't free in the way that comparison makes it look — someone has to write and debug the training script, manage checkpointing, and handle failures with no support line to call, which is real time even if it never shows up as a per-token charge.

Question 4

What's the difference between LoRA and full fine-tuning cost?

Accepted Answer

Full fine-tuning updates every parameter in the model, which is why our full fine-tuning calculator models training cost as dataset tokens times epochs times a provider training rate on top of a base model you're modifying entirely — and it typically needs enough GPU memory (or managed-training budget) to hold gradients and optimizer state for the whole parameter count, which is expensive at any real model size. LoRA and QLoRA instead freeze the base model and train a small number of added low-rank adapter weights, so the compute and memory footprint — and therefore the cost, on both the managed-API and GPU-rental side — is a small fraction of full fine-tuning for a comparable dataset. That's why this calculator and the full fine-tuning calculator use different cost shapes: full fine-tuning is priced primarily against provider training rates because DIY full fine-tuning is rarely practical on a single rented GPU, while LoRA/QLoRA is cheap and small enough that renting a single GPU yourself becomes a realistic, often far cheaper, alternative to a managed API.

Ottimizzazione di LoRA/QLoRA: calcolatore di noleggio API gestite e GPU

Tariffe dei provider gestiti rispetto al noleggio della GPU, in base alle dimensioni del tuo set di dati

Costo in base alle dimensioni del set di dati

Come si collega ad altri strumenti

Come funziona questa calcolatrice

Domande frequenti

La messa a punto di LoRA è più economica tramite API o noleggiando una GPU?

Di quale GPU ho bisogno per la messa a punto di LoRA?

Perché esiste un divario di costo così grande tra il gestito e il fai-da-te?

Qual è la differenza tra LoRA e il costo completo di ottimizzazione?