A 12,000-Token Prompt Across 10,000 Team Calls

A team running sustained API workloads can spend heavily before a model produces its first useful word. Instructions, conversation history, tool definitions and retrieved documents all contribute to input. The leak often starts with material that was useful once and then became permanent.

Consider an illustrative workflow with a 12,000-token system prompt sent across 10,000 calls. That represents 120 million input tokens from the system prompt alone, before user messages or tool results. Caching may change the cost of eligible tokens, but the starting question remains: what deserves to travel with every request?

01 · Step 1: Trace one complete request through the workflow

Choose a common task and inspect the complete assembled request. Include instructions inserted by the application, conversation messages, tool schemas, retrieved passages and tool outputs. Reading only the visible user prompt misses much of the payload.

Count tokens using the tokenizer or counting method appropriate to the selected model where available. Then compare those estimates with returned usage data. Character counts can help locate large blocks, but they are an unreliable substitute for token accounting.

• › Persistent instructions: rules required across tasks.
• › Task context: information needed for the current operation.
• › Accumulated history: earlier messages and results carried forward.

This breakdown shows whether the largest opportunity sits in the system prompt, the conversation loop or document retrieval.

02 · Step 2: Cut the 12,000-token prompt by responsibility

Read each instruction and ask which behavior it controls. Remove duplicated rules, obsolete exceptions and examples that repeat an already clear requirement. Resolve conflicting directions instead of keeping both and hoping the model chooses correctly.

Move specialist guidance into task-specific instructions when the application can reliably identify the task. A repository investigation may need code navigation guidance; a routine classification request probably does not need the same material.

Preserve rules that protect correctness and authorization boundaries. Shortening a prompt is useful only if the resulting workflow still behaves as intended. Keep an original version so changes can be compared and reversed.

03 · Step 3: Stop sending finished work back into every turn

Agent loops often accumulate entire tool responses: file contents, search results, logs and intermediate explanations. Each subsequent request can carry those tokens again when the application resends the history.

Replace completed portions with compact working state: the objective, decisions made, relevant evidence, unresolved questions and references the agent can use to retrieve details. Preserve exact text when the task depends on exact wording, such as a configuration value or an error message.

Apply the same discipline to retrieval. Send passages that support the current decision, with enough surrounding context to interpret them. Repeatedly attaching a whole document because one paragraph matters makes every later turn heavier.

04 · Step 4: Separate cache savings from payload reductions

A stable prompt prefix can be a useful caching candidate where the selected model and API support it. Keep reusable instructions consistent and place changing task material after them, following the applicable caching requirements. Confirm eligibility and usage rather than assuming repeated text received a cache hit.

Quanta fits this audit because every call is metered on real tokens, with cache writes and cache reads priced separately. Review those categories alongside ordinary input and output usage. A shorter request and a cached request produce different accounting outcomes; both deserve measurement.

Open a Quanta account to evaluate your workload through one pay-as-you-go balance, using Claude Sonnet 5.5 or GPT-6 Astra as examples where they fit your tasks.

05 · Step 5: Roll out prompt changes against task outcomes

Replay representative tasks with the original and revised context. Compare input tokens, cache activity, output tokens, tool calls, retries and task completion. Include long sessions, where repeated context has more opportunities to accumulate.

Start with a limited rollout and expand when results hold. A smaller prompt that causes extra retries can erase its savings. Track total cost per completed task, then assign ownership of shared instructions so new requirements do not quietly rebuild the oversized prompt.