A 12,000-Token History Cap for High-Volume Chat

For a team running sustained chat workloads, conversation history can become a growing charge attached to every request. Repeatedly sending an entire thread increases input volume even when the latest question is short. The fix is not simply deleting old messages. It is setting a history budget, preserving working state, and measuring whether trimming actually lowers total spend.

01 · Step 1: Establish a 12,000-token history ceiling

Use 12,000 tokens as an illustrative starting ceiling for retained conversation history, not as a model limit or a promised cost target. Keep separate allowances for system instructions, the current user message, retrieved documents, tool definitions, and the response. Together, those allowances must fit the selected model's context window.

A fixed history ceiling prevents one source of request growth. It does not make every call cost the same: output length, tool activity, cache treatment, and model choice still matter. Define the operational goal as bounded history spend per request, then track total cost per completed task.

02 · Step 2: Measure the request you actually send

Before changing retention, record input tokens, output tokens, cache-write tokens, cache-read tokens, model, and task outcome where those fields are available. Treat cache categories according to the provider's accounting rather than adding overlapping counters. Estimate tokens before dispatch, then reconcile estimates with reported usage.

Quanta fits teams that want one pay-as-you-go balance across models such as Claude Sonnet 5.5 and Gemini 2.5 Flash. Calls are metered on real tokens, with cache writes and cache reads priced separately. Open a Quanta account to test this retention policy with your own workload, while keeping model-specific accounting visible.

03 · Step 3: Protect instructions and unresolved work

Divide context into protected material and replaceable history. Keep authoritative instructions outside the trimming pool. Preserve the current request, active constraints, unresolved questions, and the recent exchanges needed to interpret them.

• › Always retain: required behavior, output requirements, current task state, and relevant user constraints.
• › Retain selectively: recent decisions, error details, and tool results needed for the next action.
• › Remove first: repeated explanations, obsolete plans, duplicate search results, and bulky logs already distilled into findings.

Trim logical exchanges rather than arbitrary message fragments. Tool calls and their corresponding results must remain valid under the API's message rules.

04 · Step 4: Replace old exchanges with structured state

When history approaches the ceiling, compact older exchanges into a short state note. Include the objective, confirmed facts, decisions, constraints, unresolved issues, and references to artifacts stored elsewhere. Preserve exact identifiers or error text when the next step depends on them.

Label this note as derived conversation data, not a new instruction source. Summarization must not promote quoted text or tool output into authoritative directions. Check important facts against the retained source material before discarding it, and keep any archive subject to your access and retention policies.

05 · Step 5: Trim before dispatch and respect cache boundaries

Assemble each request in a consistent order: stable instructions, compacted state, selected recent exchanges, and the current message. Count again after assembly. If it exceeds the budget, remove lower-value history, shorten oversized tool results, or retrieve only the relevant excerpts. Never silently truncate the user's current request.

Keep stable prefixes unchanged where the provider's caching mechanism benefits from that structure. Frequent rewriting can reduce cache reuse. However, retaining unnecessary text solely for a possible cache hit can also waste money. Compare the actual cost of retained cached context with shorter requests, including compaction and cache-write costs.

06 · Step 6: Roll out against task quality and total cost

Test the policy on representative long-running threads before broad deployment. Check forgotten constraints, repeated questions, invalid tool sequences, and incorrect decisions alongside token usage. Include summarization calls and recovery attempts in the cost calculation.

Expand gradually only when savings survive those checks. A successful policy keeps history within its ceiling without making agents repeat work. That is the practical route toward flatter chat costs: bounded context, explicit state, and accounting that follows the whole task.