Question 1

What is an LLM cascade and how does it save money?

Accepted Answer

An LLM cascade answers each request with the cheapest model that can handle it, and only reaches for an expensive model when it has to. The request goes to a cheap model first; a verifier — a scorer, a self-consistency check, a small judge model, or a confidence threshold — decides whether that answer is good enough; if it is, you are done at a fraction of the cost, and if it is not, the request escalates to a premium model. This is the pattern the FrugalGPT paper made famous, and in the right conditions it cuts cost by half or more while matching premium-model quality, because most requests in a typical workload are easy and the cheap model gets them right. The saving comes from not paying premium prices for easy work. But it is not a free lunch: you pay the cheap model on every single request, you pay for the verifier on every request, and you still pay the premium model on the share you escalate. This calculator adds all three up so the saving you see is the real one, net of the overhead that makes a cascade more than just a cheap model.

Question 2

How is a cascade different from routing traffic between two models?

Accepted Answer

A router decides up front, before any model runs, which single model each request goes to — a classifier looks at the request and sends the easy ones to the cheap model and the hard ones to the premium model, so each request pays for exactly one model. A cascade decides after the fact: it runs the cheap model on everything, then escalates the failures, so an escalated request pays for the cheap model and the premium model, plus the verifier in between. That difference matters for the cost. A router never double-pays, so at the same escalation rate it is cheaper per request than a cascade; a cascade double-pays on every escalation but needs no accurate up-front classifier, only a good verifier of finished answers, which is often far easier to build well. The quality profile differs too: a router can misroute a hard request to the cheap model and ship a wrong answer, whereas a cascade always gives the hard request a second, better attempt. This calculator shows both — the cascade cost and, for comparison, the plain traffic-split cost at the same escalation rate — so you can see exactly what the cascade's second attempt and verifier are costing you versus a router that commits up front.

Question 3

At what escalation rate does a cascade stop saving money?

Accepted Answer

There is a break-even escalation rate, and above it a cascade costs more than simply calling the premium model on everything. The logic is straightforward: a cascade always pays the cheap model and the verifier on the full volume, and then adds the premium model on the escalated share. If almost everything escalates, you have paid the cheap model and verifier for nothing and still paid full premium price on top — worse than going straight to premium. The break-even is the escalation rate where the premium cost you avoid on the requests that stop at the cheap model exactly equals the cheap-plus-verifier cost you spend on the whole volume. In formula terms it is one minus the ratio of the cheap-plus-verifier per-request cost to the premium per-request cost, so the cheaper your small model and verifier are relative to the premium model, the higher your escalation rate can climb before the cascade turns into a loss. This calculator computes that break-even for your numbers and steps the escalation rate around it in a table, so you can see how much headroom you have before the cascade stops being worth it.

Question 4

What counts as the verifier cost and why does it matter?

Accepted Answer

The verifier is whatever decides that a cheap answer is good enough to ship or bad enough to escalate, and its cost is easy to forget because it feels like plumbing rather than a model call. In practice it is often another model call: a small judge model scoring the answer, a self-consistency pass that samples the cheap model several times and checks agreement, or an embedding-and-threshold check against a reference. Each of those has a token cost, and because the verifier runs on every single request — not just the escalated ones — a verifier that looks trivial per call can quietly become a meaningful slice of the bill at volume. It also sets a floor on how cheap a cascade can be: even if the cheap model handled everything and nothing escalated, you would still pay the cheap model plus the verifier on the whole volume. That is why this calculator treats the verifier as a first-class input rather than rounding it to zero — a cascade whose verifier costs as much as the cheap model is a very different economic proposition from one whose verifier is a cheap heuristic, and the break-even escalation rate moves accordingly.

LLM カスケード節約計算ツール

カスケードコストはどこから来るのか

エスカレーション率全体にわたる節約

まず安いモデルはデフォルトでは安い請求額ではありません

この計算機の仕組み

よくある質問

LLM カスケードとは何ですか?また、どのようにコストを節約しますか?

カスケードは 2 つのモデル間のルーティングとどう違うのですか?

カスケードはどのようなエスカレーションレートで保存を停止しますか?

検証者のコストとして何が考慮されますか? それがなぜ重要ですか?