The Prompt Log That Caught Claude Sonnet 5.5 Drift

01 · A Silent Regression After the Claude Sonnet 5.5 Rollout

Every team running heavy, sustained workloads eventually meets the same quiet problem. Output quality slips, the code that produced the output looks identical to last week's, and nobody can say which variable moved. On a busy week, three things change at once: someone edits the system prompt in a shared document, a model version is pinned or unpinned in a config file, and a client tool updates its own default behaviour. A month later the incident channel fills with arguments about what caused a spike in rejected drafts or a drop in extraction accuracy.

Most of those arguments end without evidence. A prompt change log exists to make them end with evidence. It is not a design document and not a retrospective. It is a dated, append-only record of exactly what was sent to the model, which model received it, and what happened next. A team lead who keeps one can answer a question like what did we change the week Claude Haiku 4.5 started summarising tables differently in minutes instead of through archaeology across four repositories and two chat threads.

The habit pays for itself the first time a rollout is questioned by finance, by a customer, or by the engineer who has to roll it back at two in the morning.

02 · What a Prompt and Model Change Log Records on Every Entry

Keep the format boring and uniform. A spreadsheet row, a markdown file in the repository, or a dedicated table works, as long as the fields are consistent and the record is append-only. One entry per change is enough; do not batch a week of edits into one summary, because the whole value lies in the granularity.

• › Timestamp and author. When the change went live and who made it. Not who proposed it, but who shipped it.
• › Change identifier. A monotonic number or short slug so incidents can reference an exact revision rather than a description of one.
• › Prompt revision. The full text as it is now sent, stored alongside the entry, with a short diff note such as shorter system preamble or new chain-of-thought instruction.
• › Model string as requested and as returned. Record both. A pinned version and an alias that resolves to a newer build are not the same thing, and the difference is exactly what you will be asked about.
• › Parameters and tool wiring. Temperature, output budget, tool definitions, retrieval settings, and the client in use, whether that is a CLI assistant or a raw SDK call.
• › Scope and rollout state. Which workflow the prompt belongs to, what share of traffic it currently serves, and whether it is in canary, general availability, or retired.
• › Observed result. The evaluation slice it was checked against, a one-line verdict, and a link or reference to the comparison run.

Two rules keep the log honest. Never edit an old entry; add a corrective entry instead. And never record a change before it ships, because the log must describe production, not intent.

03 · Pinning Model Versions When One Balance Serves Many Families

Teams with sustained volume rarely run on a single model. They route batch summarisation to one family, latency-sensitive classification to another, and long reasoning work to a third. With a relay such as Quanta, one balance covers Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen and others, and the same log covers all of them. That is convenient for billing and for tooling, but it multiplies the number of silent variables: a switch from Claude Sonnet 5.5 to Gemini 3.1 Pro is a production change and deserves an entry of its own, even when nothing in the codebase changed except one string.

Write down the exact model identifier in every request record. Aliases are convenient during development and treacherous during an audit, because an alias can start pointing at a newer build without anyone touching your repository. If a provider deprecates a version and your request silently falls back, the response metadata is the only place that truth survives, so capture it.

Treat model swaps as higher risk than prompt edits. A prompt change usually degrades one behaviour. A model change can alter formatting, refusal patterns, tool-call syntax and reasoning depth simultaneously. Log the swap, run the same evaluation slice that the incumbent passed, and hold the old version in the entry so rollback is a copy-paste rather than a reconstruction.

04 · Reading the Log During an Incident, a Rollback and a Budget Review

The log earns its place when something goes wrong. In an incident, the questions are narrow and the clock is loud.

• › What changed in the last 24 hours? Sort by timestamp, find the most recent entry touching the affected workflow, and test the hypothesis that it caused the regression before touching anything else.
• › What exactly was the model asked to do? Open the stored prompt revision instead of the current one in the repository. The two frequently differ.
• › How do we undo it? Restore the previous prompt revision and the previous model string, then confirm with the same evaluation slice.

Budget reviews ask different questions of the same rows. When token spend climbs, the log shows whether the cause is a prompt that grew three hundred tokens of preamble, a move to a larger model for a workflow that never needed one, or simply more traffic. Traffic growth is a success story; an accidental prompt inflation is a fix. Without the log, both look identical on the invoice.

Rollout reviews work the same way. Entries tagged as canary tell you how long a revision has been in partial production and which workflow it has not yet reached.

05 · Cache Writes, Cache Reads and the Cost Behind Each Prompt Revision

Cost data belongs in the log because prompt edits have a cost dimension that is easy to miss. In a metered environment where calls are billed on real tokens and cache writes and cache reads are priced separately, a small edit to the top of a system prompt can be expensive. Move one sentence near the front of a long preamble and you may invalidate the cached prefix for every subsequent call, turning cheap cache reads into full-price input and fresh cache writes, all while the output looks unchanged.

So record, per revision, the input token volume of the stable prefix you intend to cache, and watch the split between cache reads and cache writes after each deploy. A healthy steady workload reads from cache most of the time; a prompt that is being churned will show write-heavy behaviour. If you route across several models on one balance, a relay that meters real tokens and prices cache activity separately gives you the clean per-call data this comparison needs, and the log gives it a place to land.

Two practical habits help. Keep the shared instructions and retrieved context in a stable, ordered prefix and put volatile material at the end. And when a revision is expected to be cost-neutral, write that expectation in the entry, so the next reviewer knows whether an increase was planned.

06 · Rolling Prompt Versions Out Through Codex, Gemini CLI and SDK Clients

Prompts live wherever clients run, and at team scale that means a CLI assistant here, an SDK integration there, and a scheduled job somewhere else. If each client keeps its own copy of the instructions, the change log describes a fiction. Centralise the prompt text in one versioned store, have clients request a revision by identifier, and let the log record which revision each workload is actually serving.

Roll out the way you would roll out code. Ship a new revision to a small share of traffic, compare it against the incumbent on a fixed evaluation slice, promote it, and retire the old one by marking its entry rather than deleting it. Archive retired revisions; they explain historical behaviour and they make rollbacks possible months later.

The same structure scales to model migrations. When a team decides that a workflow should move from Claude Opus 5.5 to a faster variant for cost reasons, the log carries the rationale, the evaluation evidence and the rollback target in one place, and the next lead inherits the context instead of guessing at it.

Start the log this week, even if it is one file with five fields. Add the model string to it immediately, then the prompt text, then the cost split. Within a quarter it becomes the document you open first during any incident, any budget conversation and any rollout review. If your team is building that discipline on top of a single metered balance across many models, open an account with Quanta and let the per-call token and cache data feed the log you keep.