A RAG chatbot's cost is dominated by the retrieved context you stuff into the prompt, not the question. Each query = embed + vector lookup + an LLM call carrying your chunks. Enter your usage to see the monthly bill.
—
per month—per query
—per year
—queries/mo
Cost breakdown per query
| Component | $ / query | % of total |
|---|
⚠️ Reference estimates (July 2026). Embedding ~$0.02/1M tokens and a vector query are tiny next to the LLM call carrying retrieved context. Prices vary by model and provider — type your real rates. Vector DB storage is billed separately (see the vector database cost calculator). · Report outdated price →
Why retrieved context is the cost driver
People assume the question drives RAG cost. It doesn't — the retrieved chunks you paste into the prompt do. Pull 8 chunks of 500 tokens and every query carries 4,000 input tokens before the model writes a word. That's why retrieving fewer, smaller, better chunks beats any other optimization, followed by caching the static system prompt. Size the vector store with the vector database cost calculator, and compare models on the AI API cost calculator. For a non-RAG bot, see the chatbot cost calculator.