429 Backoff for 50-Worker Claude Sonnet 5.5 Teams

01 · 1. Treat a 429 as scheduling feedback

A 429 response means your worker has sent more traffic than the service can accept at that moment. It is usually temporary, so failing the job immediately wastes work and can create unnecessary alerts. It is also not permission to retry in a tight loop. Each immediate retry adds pressure to the same limit and can delay recovery for every worker sharing the queue.

Start by separating rate-limit failures from permanent failures. Record the HTTP status, response body, request identifier, model, queue name, and attempt number. For a workload using Claude Sonnet 5.5, for example, keep its rate-limit behavior visible separately from other model traffic. Different models, credentials, or upstream routes may have different limits.

02 · 2. Read Retry-After before calculating a delay

When the response includes a Retry-After header, use it as the server's preferred delay. The value may be a number of seconds or a date, so parse both forms and clamp the result to a reasonable maximum for your application. Add a small random spread so thousands of workers do not wake at the same instant.

If Retry-After is missing or malformed, use exponential backoff with jitter. A practical schedule might start at two seconds, double after each retry, and stop increasing at several minutes. Full jitter selects a random delay between zero and the calculated maximum. This prevents synchronized retry waves while keeping the average recovery time reasonable.

03 · 3. Keep retry state with the job

Do not store retry counters only in process memory. A worker can restart, a deployment can move the job, or multiple processes can handle the same queue item. Persist the attempt number, next eligible time, last status, and the original request metadata with the job.

When a 429 occurs, acknowledge the current delivery only after writing the delayed retry state. Otherwise, a crash between those operations can lose the job or execute it twice. Use a queue feature such as delayed visibility, scheduled messages, or a dedicated retry queue. Set a dead-letter path for jobs that exceed the retry limit.

04 · 4. Control concurrency before the limit is reached

Retries alone do not solve excessive concurrency. Add a shared limiter that controls requests per second, in-flight requests, or both. The limiter must be shared across worker instances if the upstream limit is shared. A local limit on each of 50 workers can still produce 50 times the intended traffic.

Use separate concurrency pools when traffic has different priorities. Interactive tasks may need a short queue, while batch summarization can wait longer. You can also reserve capacity by model. A deployment that uses both Claude Haiku 4.5 and Gemini 3.1 Pro should avoid allowing a surge in one workload to consume every available worker slot.

05 · 5. Make retries safe for non-idempotent work

A rate-limited request may have reached the provider even when your worker received a 429 or a network error. Before retrying, decide whether repeating the operation can create a duplicate side effect. For generation jobs, store a stable job identifier and request fingerprint. For actions that update external systems, use idempotency keys where supported and make the result write conditional on that identifier.

Keep the original prompt, model selection, token settings, and cache options unchanged during a retry unless your policy explicitly allows a fallback. Changing these values can make accounting, output comparison, and debugging difficult.

06 · 6. Measure recovery and budget impact

Track 429 counts by model, route, worker pool, and time window. Also measure retry delay, attempts per successful job, queue age, dead-letter count, and tokens spent on completed requests. Alert on sustained queue growth rather than on a single 429. A brief burst may be normal; a growing backlog indicates that demand exceeds configured capacity.

For teams running heavy workloads through Quanta, centralizing traffic through one pay-as-you-go relay can simplify usage accounting across supported models. Quanta meters real tokens, with cache writes and cache reads priced separately, so retry policy should account for the cost of repeated requests and cache behavior. Open a Quanta account when you are ready to consolidate model traffic and apply these worker controls in one operating workflow.

07 · 7. Test the policy under an artificial limit

Before production, make a test endpoint or middleware return 429 responses with and without Retry-After. Verify that jobs wait, attempt counts persist, workers spread their retries, and permanent failures reach the dead-letter queue. Confirm that shutdowns do not lose scheduled work and that dashboards distinguish successful first attempts from successful retries.

A graceful design turns 429 responses into controlled queue timing. The result is steadier throughput, fewer synchronized retry storms, clearer budget decisions, and a worker fleet that can absorb sustained demand without treating temporary pressure as a system failure.