Budgeting 100,000 Retrieval Answers Before Rollout

A retrieval-augmented answer is not priced by the length of its final paragraph. Its cost starts with everything the model receives: instructions, conversation history, the question, retrieved passages, and formatting overhead. For a team planning 100,000 answers, small mistakes in that accounting become recurring budget errors. This walkthrough builds a token estimate you can validate before rollout.

01 · Step 1: Define what counts as one completed answer

Start with the full request path, not just the final generation call. Does your application rewrite the question, retrieve documents, rerank passages, and then generate an answer? Does a failed validation trigger another attempt?

List every model call required to deliver one completed answer. Keep embedding, vector database, and other infrastructure charges separate from generation-token charges. They belong in the total service budget, but mixing them together makes token optimization harder to evaluate.

02 · Step 2: Count the complete generation input

Build a representative request using the exact message structure your application sends. Count tokens with the tokenizer or counting facility appropriate to the selected model, then compare that estimate with reported usage from a test call. Character-based shortcuts are useful for rough planning, not reliable billing forecasts.

For an illustrative request, assume these token counts:

• › Shared instructions and answer format: 1,200 tokens.
• › User question and conversation history: 800 tokens.
• › Six retrieved passages at 500 tokens each: 3,000 tokens.
• › Document labels, separators, and message overhead: 200 tokens.

The estimated input is 5,200 tokens. These are planning assumptions, not measured performance claims. Count passage metadata too: repeated titles, document identifiers, and timestamps can materially expand a retrieval payload.

03 · Step 3: Estimate generated tokens separately

Assume the answer produces 700 output tokens. Your baseline request now has 5,200 input tokens and 700 output tokens, but those quantities should remain separate because their rates can differ.

An output limit is a ceiling, not a prediction. Measure actual completions across short questions, multi-document comparisons, and requests that require detailed explanations. If the selected model reports additional billable reasoning or other token categories, include them according to its billing rules rather than assuming visible answer text captures everything.

04 · Step 4: Apply rates without inventing a price

Let I be the uncached input rate per million tokens and O the output rate per million tokens. With no cache activity or additional billable categories, the illustrative answer costs (5,200 × I + 700 × O) / 1,000,000.

Use the current rates for your chosen model. If evaluating Claude Sonnet 5.5 and Gemini 2.5 Flash, calculate each separately and measure answer quality on the same retrieval tasks. Token counts and completion lengths can differ, so changing only the rate is not a complete comparison.

05 · Step 5: Split cache writes from cache reads

Caching changes the accounting, not the need to count tokens. For this estimate, divide input into mutually exclusive buckets: uncached input U, cache-write tokens W, and cache-read tokens R. Let their respective per-million-token rates be I, Cw, and Cr.

The request formula becomes (U × I + W × Cw + R × Cr + output tokens × O) / 1,000,000. Confirm the provider's usage-field definitions before applying it, so cached tokens are not counted twice.

Do not assume all 5,200 input tokens qualify for caching. A stable instruction prefix may be reusable while retrieved passages change on every question. Estimate cold requests, warm requests, expiration, and cache eligibility separately.

06 · Step 6: Scale measured averages to 100,000 answers

For the illustrative uncached workload, 100,000 single-call answers would consume 520 million input tokens and 70 million output tokens. That forecast excludes retries, auxiliary calls, and changes in request size.

Run a representative pilot and divide its total metered cost by successfully completed answers. Multiply that observed cost by planned volume. Also inspect high-token requests and burst periods: an acceptable average can conceal expensive conversation histories or oversized retrieval bundles.

07 · Step 7: Meter the rollout and tighten retrieval

Quanta fits teams that want pay-as-you-go access across models through one balance, with calls metered on real tokens and cache writes and reads priced separately. Open a Quanta account to validate your estimate against metered pilot usage before expanding traffic.

Track cost per completed answer alongside correctness and retrieval quality. Reduce redundant passages, cap conversation history deliberately, and test smaller retrieval bundles. The target is not the fewest tokens; it is the lowest measured cost that still meets your answer-quality requirements.