Question 1

Why does Baseten cost more than RunPod for the same GPU?

Accepted Answer

Not because the underlying GPU rate is different — in this comparison both are billed at the same $/hour — but because of how each platform rounds the clock. RunPod's serverless billing is per-second: you pay for exactly the seconds the container is active, cold-start load time included, and nothing more. Baseten bills per-minute and rounds every invocation UP to the next full minute. A warm 3-second inference call still gets billed as a full 60-second minute, and an 11-second cold-start call gets billed as that same full minute too, because both round up to 1. For a bursty, low-QPS workload where most individual calls are only a few seconds long, that rounding is brutal: you're effectively paying for 15-20x more compute time than you actually used on every single call. The gap isn't a pricing gimmick, it's pure minute-granularity rounding applied to very short jobs, and it compounds every time the workload is this short and this frequent.

Question 2

What is a 'cold start' and why does it cost money?

Accepted Answer

A serverless GPU platform doesn't keep a container running (and billing you) when nothing is happening — it scales the container down to zero after a period of inactivity to save you money on idle time. The tradeoff is that the next request after a scale-to-zero has to wait for a brand-new container to spin up: pull the container image, load the model weights onto the GPU, initialize the runtime, and only then start actual inference. That load process is the 'cold start,' and on GPU-hosted LLM or inference workloads it commonly takes several seconds to tens of seconds, because model weights can be gigabytes and moving them onto GPU memory isn't instant. Crucially, that loading time isn't free compute time being donated to you — the GPU is fully allocated and billing during it on every platform modeled here, exactly like it would be during actual inference. So every cold-start request effectively costs you the cold-start seconds PLUS the actual inference seconds, and the more spread out (bursty) your traffic is, the more of your requests hit a cold container instead of a still-warm one, and the more of your total bill is pure loading overhead rather than useful compute.

Question 3

Should I use a keep-warm buffer for my inference workload?

Accepted Answer

It depends on whether you're optimizing for latency or for cost, and those two goals pull in opposite directions for bursty traffic. A keep-warm buffer (modeled here as Modal's approach) holds the container alive and loaded for a window after each request finishes, so if the next request arrives inside that window it skips the cold start entirely and responds fast. That's a real user-experience win for latency-sensitive traffic. But you are paying full GPU-active pricing for every idle second the container sits there waiting, whether or not another request ever arrives in time — the calculator's Modal model adds the idle buffer as billed seconds on every request that benefits from a warm container. For very sporadic traffic — long, unpredictable gaps between calls — that idle time is mostly wasted money, because the buffer expires before the next request shows up anyway and you paid for it for nothing. Keep-warm buffers make the most economic sense when request gaps cluster just inside the buffer window; for truly random or rare traffic, eating the occasional cold start is usually cheaper than paying to keep a GPU warm on the off chance the next call arrives soon.

Question 4

Is per-second or per-minute GPU billing better for bursty traffic?

Accepted Answer

For bursty, short-duration traffic, per-second billing is almost always better, and the shorter your typical inference call is relative to a minute, the bigger the advantage. Per-minute billing that rounds up (like Baseten's model here) charges a full minute no matter whether the actual compute took 3 seconds or 55 seconds — the billing granularity is coarser than the workload, so short calls get taxed the hardest in percentage terms. Per-second billing (like RunPod's and Modal's active-compute portion here) charges close to exactly what was used, so a 3-second call costs roughly a twentieth of a 60-second call instead of the same amount. The one place per-minute billing stops being a clear disadvantage is when your typical call duration is already close to or above a minute — the rounding waste shrinks toward zero as call length approaches the billing granularity. For an internal tool or sporadic API called a few hundred times a day with multi-second responses, minute-rounded billing can easily turn out to be an order of magnitude more expensive than per-second billing at the identical nominal hourly GPU rate.

Calculateur de facturation de plate-forme GPU sans serveur : RunPod vs Baseten vs Modal

Répartition de la facturation, par plateforme

Coût mensuel par taux de démarrage à froid (rapidité du trafic)

Comment cela se connecte à d'autres outils

Comment fonctionne cette calculatrice

Questions fréquemment posées

Pourquoi Baseten coûte-t-il plus cher que RunPod pour le même GPU ?

Qu'est-ce qu'un « démarrage à froid » et pourquoi cela coûte-t-il de l'argent ?

Dois-je utiliser un tampon de maintien au chaud pour ma charge de travail d'inférence ?

La facturation GPU à la seconde ou à la minute est-elle meilleure pour le trafic intense ?