Quanta

← Back to Briefings

Claude Haiku 4.5 Answers a 300-Ticket Week

Team playbooksEN3 reads

01 · What a Knowledge-Base Assistant Owns on a 300-Ticket Week

Before you write a single prompt, decide which slice of the support queue the assistant is allowed to touch. A team handling roughly 300 tickets a week usually has three or four recurring intents: account access, billing changes, configuration errors, and outages. That clustering is your first design input. The assistant should own the high-volume, well-documented intents and hand everything else to a human, rather than trying to be a general oracle on day one.

Write the ownership boundary as a short operating document that the whole team can point at. It should state which intents are automated, which are answered with a draft that an agent approves, and which are escalated immediately. It should also name the sources the assistant is allowed to cite internally: the help center, the runbook library, the changelog, and the incident notes. Anything outside that set is out of scope, and the assistant should say so instead of improvising.

  • › Answered automatically: questions whose answer exists verbatim in a current article.
  • › Answered as a draft: questions that need an account-specific detail or a judgment call.
  • › Escalated: security reports, billing disputes, legal requests, and anything a customer has already complained about twice.

That boundary also gives you a rollout metric that matters more than model benchmarks: the share of tickets that clear without an agent rewrite. Track it weekly and move intents into the automated bucket only after two consecutive weeks of clean drafts.

02 · Retrieval Beats Training for a Document Set That Changes Weekly

Support content rots. Articles get edited after incidents, product names change, and pricing pages move. Fine-tuning a model on your corpus locks in a snapshot and creates a second artifact you have to retrain every time the docs change. Retrieval keeps the knowledge in a store you can update in minutes: index the articles, keep the article identifier with each chunk, and pass only the relevant chunks into the request.

Chunking is where most teams lose accuracy. Split on section headings rather than fixed character counts, keep a heading breadcrumb at the top of every chunk, and never let a chunk cross a code sample or a table. Store the article version or last-edited date with each chunk so the assistant can refuse to answer from a document that predates a major release.

For the model itself, start with a fast, inexpensive tier for retrieval queries and draft answers, and escalate the small share of long or ambiguous threads to a stronger tier. In practice that means Claude Haiku 4.5 for the bulk of short ticket replies and Claude Sonnet 5.5 for threads that require reasoning across several articles. Keep the router simple at first; you can make it smarter once you have real failure examples.

03 · Context Budgets, Cache Writes and Cache Reads Decide Your Bill

Every call is metered on real tokens, and the assistant's cost profile is dominated by what you re-send on every turn. A support thread grows: the customer's history, the retrieved articles, the system instructions, and the running summary all sit in context by the third or fourth reply. If you resend the full document bundle on every turn, you pay for the same tokens over and over.

Split the request into a stable prefix and a changing tail. Put the system instructions, the escalation policy, and the most-frequently-used reference articles at the front so they cache well, and keep the live conversation and the retrieved chunks at the end. Cache writes and cache reads are priced separately, so the discipline is simple: write once for the stable material, read it cheaply on every subsequent turn, and avoid invalidating the prefix with things that change often, such as a timestamp in the system prompt.

  • › Summarize the thread after each turn instead of resending every prior message.
  • › Cap retrieved chunks at a fixed count and drop the least relevant ones.
  • › Cache the system prompt and core policy articles; do not cache the customer's message.
  • › Log tokens per ticket category so you can see which intents are expensive.

Watch the ratio between cached and uncached tokens week over week. If the uncached share climbs while ticket volume stays flat, something in your prompt is moving and breaking the cache, usually a rotating timestamp or an injected queue status.

04 · Guardrails That Keep a Confident Wrong Answer Out of the Queue

The main risk with a support assistant is not silence; it is fluency. A plausible-sounding answer that contradicts the current documentation is worse than a ticket that waits for an agent. Build the guardrails around that failure mode rather than around model personality.

Require the assistant to quote or reference the article it relied on, and reject any draft that cannot name a source from your index. Ask it to state the answer, the conditions under which the answer changes, and the one thing the customer should verify before acting. That three-part shape surfaces contradictions quickly during review and gives agents something concrete to correct.

  • › Grounding check: a draft with no matching article is routed to an agent.
  • › Recency check: chunks older than the current release are flagged before the answer is sent.
  • › Scope check: requests outside the ownership boundary return a short handoff message.
  • › Tone check: no refunds, no policy exceptions, and no commitments on dates.

Sample review matters more than volume review. Pull ten drafts a week per intent, have two agents score them for grounding, completeness, and tone, and feed the failures back as retrieval queries to test. Within a few weeks you will have a failure library that is worth more than any prompt tweak.

05 · A Six-Week Rollout With a Metered Balance and One Dashboard

Run the rollout in three two-week phases. Weeks one and two: index the corpus, build retrieval, and generate drafts only, with every draft reviewed by an agent. Weeks three and four: automate the top two intents and keep the rest on drafts. Weeks five and six: expand to the next intents and start measuring deflection and rewrite rate together.

Budget the workload on a pay-as-you-go relay so spend tracks real usage instead of a fixed seat count. Quanta meters every call on real tokens, prices cache writes and cache reads separately, and gives you a single balance across Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and other models, so you can move a tier up or down without renegotiating a contract. It works with Claude Code, Codex, Gemini CLI and any OpenAI SDK, which means the assistant prototype can stay in the same toolchain your engineers already use.

Put four numbers on one dashboard and review them every Monday: tickets touched, tickets cleared without a rewrite, uncached tokens per ticket, and escalations that should have been automated. Those four tell you whether the assistant is saving the queue money or quietly creating rework. If you have not opened an account yet, open one and start with a single intent so your first cost readings are clean.