Two very different cost shapes

The core difference is the shape of the cost. A hosted API is pure variable cost: you pay per token, nothing when idle, and it scales smoothly from one call to millions. Self-hosting is mostly fixed cost: you rent or buy GPUs that cost the same whether they are busy or idle, plus the engineering time to run them.

That single difference drives the whole decision. APIs win when usage is low, spiky, or unpredictable. Self-hosting can win only when usage is high, steady, and predictable enough to keep expensive hardware busy.

What self-hosting actually costs

The GPU rental or purchase is just the headline number. The full bill includes:

• Compute, GPU instances priced by the hour, which you pay for 24/7 if you want the model always available.
• Engineering time, to deploy, optimize, monitor, patch, and keep the service up. This is a recurring salary cost, not a one-off.
• Idle waste, every hour your GPU sits below full utilization is money spent for nothing.
• Reliability overhead, redundancy, failover, and scaling for traffic peaks, which a hosted API handles for you invisibly.

Teams routinely underestimate the last three. The GPU bill is visible; the humans and the idle time are not.

What the API price includes

When you pay per token to a hosted provider, you are not just buying compute. You are buying the hardware, the uptime, the scaling, the security, the ops team, and the ability to go from zero to huge volume instantly. The per-token price bundles all of that.

You are also free of capacity planning. No decision about how many GPUs to keep warm, no 3 a.m. page when a node dies. For many teams, that operational relief is worth more than the raw compute margin the provider charges.

The break-even logic

Break-even is about utilization. A self-hosted GPU has a roughly fixed monthly cost and a maximum throughput of tokens it can produce. Divide the monthly cost by the tokens you actually push through it to get your effective per-token cost. If your traffic keeps the GPU near full load, that effective cost can undercut the API. If the GPU is half idle, your effective per-token cost doubles, and the API likely wins.

Illustrative example (invented numbers): if a GPU instance costs $1,500/month and can realistically serve, say, 500 million tokens/month at full tilt, your floor cost is $3 per million tokens, but only if you actually use all 500 million. Run just 100 million and your real cost is $15 per million, five times worse. Compare that effective figure against the API rate on the model comparison pages.

Quality and capability differences

Cost is not the only axis. The strongest frontier models are generally available only through hosted APIs, while self-hosting means running open-weight models, which are excellent but may trail the very best closed models on the hardest tasks. If your use case needs top-tier capability, self-hosting may not even be an option.

On the other hand, self-hosting gives you data control (nothing leaves your infrastructure), no rate limits beyond your own hardware, and freedom from provider pricing changes. For regulated industries or privacy-sensitive data, those can outweigh a higher effective cost.

A practical decision path

A reasonable rule of thumb:

• Start on a hosted API. It is the cheapest way to reach real volume and validate your product without capital outlay.
• Track your monthly spend. When it grows large and steady, model the self-host alternative honestly, including engineering time and realistic (not theoretical) utilization.
• Consider a hybrid. Serve high-volume simple tasks on a small self-hosted open model, and route hard queries to a hosted frontier API.

Use the LLM cost calculator to price your current API usage, then compare it against a fully loaded self-host estimate. Do not forget to include the salary cost of the people who will run it, that line is what most break-even calculations quietly omit.

Continue learning

Related tools

Frequently asked questions

At what point does self-hosting get cheaper?

Only when your usage is high enough and steady enough to keep expensive GPUs near full utilization, and after accounting for engineering time. At low or spiky volume, a hosted API is almost always cheaper.

Can I run the same quality model myself?

You can run strong open-weight models, but the most capable frontier models are typically API-only. If your task needs top-tier performance, self-hosting may not match it.

What cost do people forget when self-hosting?

Engineering and operations time, plus idle GPU hours. The hardware bill is visible, but the salaries to run it and the capacity you pay for but do not use are the costs that sink most estimates.