Pick the Right Model for Customer Support

Choosing a model for customer-support automation starts with a workflow, not a leaderboard. Your goal is to resolve the right requests accurately, within an acceptable response time and operating budget. This walkthrough turns that goal into a repeatable selection process for engineering and operations teams.

01 · Step 1: Separate answers from account actions

List the work your automation will handle: explaining product features, checking order status, troubleshooting failures, or preparing refund requests. Separate informational replies from actions that change customer records or move money. Start with the simplest workflow that can complete each task; add autonomous decision-making only when a fixed sequence proves insufficient.

For every category, specify the information required, permitted tools, completion conditions, and handoff destination. Make those requirements the contract every candidate must satisfy.

02 · Step 2: Build a test set from actual tickets

Collect representative historical conversations under your organization’s data-handling rules. Include straightforward requests, incomplete descriptions, policy exceptions, frustrated customers, and unsuccessful interactions. Write an expected outcome for each case, including situations where asking a clarifying question or escalating is the correct result. Evaluate the full interaction and resulting state, not merely whether the final reply sounds helpful.

• › Accuracy: Does the answer match the approved policy and available evidence?
• › Completion: Does the workflow actually achieve the permitted outcome?
• › Restraint: Does it avoid unsupported promises and unauthorized actions?
• › Handoff: Does an escalation preserve the context a human needs?

03 · Step 3: Compare a small shortlist fairly

Choose two candidates for an initial comparison, such as Claude Haiku 4.5 and Gemini 2.5 Flash. Treat these as candidates, not assumed winners. Run both against the same tickets, policy documents, tool definitions, and output requirements. Begin with a shared baseline, then allow documented adjustments for each candidate. Use the evaluation results to decide whether additional workflow complexity is justified.

Quanta fits naturally here: its pay-as-you-go API relay lets teams compare models using one balance. For a sustained workload, that makes it a practical place to organize trials without treating the first model choice as permanent. Keep application logic separate from model configuration, and verify the required behavior for every candidate.

04 · Step 4: Calculate cost per acceptable resolution

Define an acceptable resolution before comparing costs. Then divide the measured operating cost of the trial by the number of cases meeting that definition. Keep human-review costs visible rather than counting every generated answer as a successful resolution.

Quanta meters calls on real tokens and prices cache writes and cache reads separately. Record those categories alongside input and output usage. Test both first-time requests and repeated-policy conversations, then include retries, escalations, and tool activity in your own accounting. Compare candidates using the workload you expect to operate, not an isolated short prompt.

05 · Step 5: Put permissions outside the conversation

Require server-side authorization for account access and state-changing tools. Treat customer messages and retrieved text as untrusted input, not instructions that can redefine permissions. Validate tool arguments, restrict available actions, and require approval for high-impact operations. Test attempts to redirect the assistant, expose private information, or bypass policy. Model selection should sit inside these controls, not replace them.

06 · Step 6: Add routing only when tests justify it

Try a routing rule for a clearly defined request category only after your baseline works. For example, keep routine informational requests on the candidate that meets your quality and cost requirements, while sending unresolved cases to human support. Compare this design with a single-model workflow, including routing mistakes, extra calls, and handoff delays. Routing is a workflow pattern to evaluate, not an automatic improvement.

07 · Step 7: Roll out narrowly and retest changes

Start with a limited, reversible deployment. Review failures and turn them into regression cases. Repeat the evaluation whenever you change the model, prompt, retrieval source, or tools; preserve a rollback configuration and a human-support route. An expanding evaluation suite gives your team a concrete way to judge whether changes improve the system.

Open a Quanta account to run your shortlist against your own support workload. Choose the configuration that demonstrates acceptable resolutions, controlled permissions, and sustainable operating costs—not the one that produces the most impressive demonstration.