Question 1

What is cost per correct answer and why does it matter more than token price?

Accepted Answer

Cost per correct answer is what you actually pay to get one usable result out of a model, rather than what a single call costs. Token price tells you the cost of one attempt; it says nothing about whether that attempt was right. If a model gets the task correct only 70% of the time and you retry or discard the failures, you are paying for roughly one and a half attempts per good answer, so its effective cost is its sticker price divided by its accuracy. That single division changes the ranking: a model that is 40% cheaper per call but noticeably less accurate can end up more expensive per correct answer than a pricier model that gets it right almost every time. Optimising on token price alone quietly optimises for the wrong number — this calculator optimises for the one that hits your bill, the cost of a result you can ship.

Question 2

How do I calculate the accuracy-adjusted cost of a model?

Accepted Answer

Take the cost of one call and divide it by the model's success rate as a fraction. A call that costs two cents at 95% accuracy has an effective cost of about 2.1 cents per correct answer; the same two-cent call at 65% accuracy costs about 3.1 cents per correct answer, because roughly one and a half calls are needed on average to land a good one. The multiplier is one over the accuracy — 95% accuracy means a 1.05x tax, 65% means a 1.54x tax, 50% means you pay double. This model assumes you can tell a correct answer from a wrong one and simply retry the failures, which is the case when you have a verifier, a test suite, human review, or a downstream check. If bad answers slip through undetected the real cost is higher still, because you also pay the downstream cost of being wrong.

Question 3

How much accuracy does a cheaper model need to actually be cheaper?

Accepted Answer

There is a clean break-even: the cheaper model wins on cost per correct answer only if its accuracy is at least the pricier model's accuracy scaled by the ratio of their prices. If the premium model is 95% accurate and the cheaper one costs 70% as much per call, the cheaper model needs at least 95% times 0.70, about 66.5% accuracy, to break even. Below that its retries eat up the price saving and the premium model is genuinely cheaper per usable result; above it the cheaper model wins. This is why a headline discount is meaningless without an accuracy number attached — a 30% cheaper model that is 20 points less accurate is usually a false economy. This calculator solves for that break-even accuracy directly so you know exactly what the cheaper model has to hit.

Question 4

Does this apply to classification, extraction and agents too?

Accepted Answer

Yes, anywhere you can score whether an output is right. For classification and extraction, accuracy is your pass rate against a labelled set. For agentic tasks it is the fraction of runs that reach a correct final result, and there the effect is amplified because a failed agent run can burn far more tokens than a single call — so a low success rate on an expensive agent is punishing per correct completion. For open-ended generation where there is no clean right-or-wrong, the framing is fuzzier, but you can still use an acceptance rate — the share of outputs a human keeps without a rewrite — as the accuracy input. The rule holds in every case: divide the cost of an attempt by the probability the attempt is good, and compare models on that, not on their per-token sticker.

正解あたりのコスト計算ツール

ステッカーの価格と結果の価格

安価なモデルにはどの程度の精度が必要か

お金がかかる割引

この計算機の仕組み

よくある質問

正解あたりのコストとは何ですか? なぜトークンの価格よりも重要なのでしょうか?

モデルの精度調整後のコストを計算するにはどうすればよいですか?

安価なモデルを実際に安価にするためには、どの程度の精度が必要ですか?

これは分類、抽出、エージェントにも適用されますか?