Rerank the shortlist, not the haystack
A reranker improves RAG quality cheaply if you feed it a small candidate set. Reranking hundreds per query is the cost trap. Size the retrieval side on the RAG chatbot cost calculator.
Reranking: small model, real bill
Rerankers price per 1,000 searches ($1–$2) or per document scored. The multiplication that surprises: reranking 100 candidates per query at $1/1k searches with 50k monthly queries is $50 — fine. But naive pipelines rerank on every keystroke of search-as-you-type, multiplying query volume 5–10×. Debouncing and reranking only the final query restores sanity. The cost-effectiveness case is strong though: a reranker in front of a RAG pipeline typically lets you retrieve fewer, better chunks — cutting LLM input tokens enough that the reranker pays for itself. Test top-20 rerank versus top-5 raw retrieval; the token savings usually win.
How this calculator works
The Reranking API Cost Calculator estimates your monthly bill for using a reranking API, which reorders retrieved documents by relevance to a query. It multiplies your queries per month by the documents reranked per query to find your total volume, then converts that into billable search units and applies your price per 1,000 search units. The main cost drivers are simple: how often you rerank (query volume) and how many documents you pass through the model each time. Because both inputs multiply together, raising either one increases cost proportionally, and raising both compounds quickly.
The key trade-off to watch is candidate set size. Sending more documents per query can improve result quality, but each additional document adds to your search-unit count and your bill. A practical tip is to use a cheap first-pass retriever to narrow candidates, then rerank only the top handful rather than everything. Test where quality plateaus — often the top 20 to 50 documents capture nearly all the benefit, so reranking beyond that mostly adds cost without improving answers.
Frequently asked questions
How is reranking priced?
Usually per search unit — roughly one unit per document scored against a query. Cost = queries × docs reranked × per-unit price. Reranking a huge candidate set per query multiplies fast.
Is reranking worth the cost?
Often yes — it noticeably improves RAG answer quality by reordering retrieved chunks. The trick is reranking only the top candidates (say 20–50), not hundreds, to keep cost sane.