Turn GPU rent and throughput into a real cost per million self-hosted tokens.
Cost per token is GPU rent divided by tokens actually served — so utilisation, not raw speed, sets your bill. Decide if self-hosting beats a provider on the self-host vs API calculator, and price the underlying hardware on the GPU cloud cost calculator.
Free GPU inference cost calculator — convert GPU hourly price, throughput and utilisation into the true cost per million tokens for a self-hosted model.
Take the GPU hourly price and divide by how many tokens it actually serves per hour. That is throughput (tokens/sec) times 3600, scaled by your real utilisation. A GPU billed continuously but half-idle serves half the tokens, so its cost per token doubles versus the datasheet throughput.
Almost always utilisation and batching. Benchmark throughput assumes full, batched load; real traffic is bursty, so the GPU sits idle between requests while the meter runs. Continuous batching, right-sizing the GPU, and consolidating traffic onto fewer, busier GPUs are the main ways to bring the per-token cost down.