180-Second Idle Limits for a Busy Streaming Queue

01 · Step 1: Separate silence from total runtime

A busy engineering queue needs more than a single request timeout. Long streaming responses can be healthy for minutes, while a stalled connection can occupy a worker without producing useful output. Treat those as different conditions before changing production settings.

Map four limits: connection establishment, time to first useful event, inactivity between incoming chunks, and total request duration. Also inspect any deadline imposed by your job runner. A generous client setting cannot override an earlier cutoff in a proxy or worker supervisor.

For workloads using Claude Sonnet 5.5 or GPT-6.1 Sol, measure these stages separately rather than assuming every request has the same timing profile. Prompt size, requested output length, tool execution, and network conditions can all change the observed behavior.

02 · Step 2: Start with an explicit 180-second idle budget

Use a documented starting configuration, not an unexplained large number. The following values are illustrative rollout settings, not Quanta defaults or guarantees:

• › Connection establishment: allow 15 seconds to establish the connection.
• › First useful event: allow 120 seconds before the first application-relevant stream event.
• › Read inactivity: allow 180 seconds between incoming chunks.
• › Total runtime: cap the complete operation at 20 minutes.

Confirm what your client actually measures. A read timeout may apply to each socket read rather than the complete response. A total deadline should continue advancing even when chunks arrive regularly. Define whether reasoning events, tool-call deltas, or other structured events count as useful progress; visible answer text is not always the first meaningful event.

03 · Step 3: Align proxies without hiding stalled generation

Trace the full path from the worker through gateways, load balancers, and outbound proxies. For each component you control, check its documented streaming behavior and idle limits. A proxy with a shorter inactivity threshold can terminate a response before your client's 180-second budget expires.

Where appropriate, give that proxy a slightly larger idle allowance, such as 240 seconds. Ensure response buffering is disabled where necessary so chunks reach the client promptly. Validate the result through the production route, not only from a developer laptop.

Keep connection activity separate from generation progress. Heartbeats may preserve a transport connection without indicating useful model output. Track both the last received bytes and the last meaningful event, while retaining the absolute operation deadline.

04 · Step 4: Preserve partial output before considering retries

Persist stream events incrementally with an internal job identifier, attempt number, timestamps, and completion state. Mark an interrupted response as incomplete even if its text looks finished. Use the protocol's completion signal, not punctuation, to decide whether generation ended successfully.

Do not automatically replay a request after partial output. A replacement attempt can repeat generated material, consume additional tokens, or duplicate downstream actions. For tool-enabled jobs, make side effects idempotent and require explicit handling of uncertain execution outcomes.

When cancellation is necessary, close the stream and propagate cancellation wherever supported. Do not assume a local timeout proves that upstream generation stopped immediately or that no additional usage occurred.

05 · Step 5: Budget long streams as occupied capacity

Longer deadlines increase the time each request can hold a worker, connection, and concurrency slot. Set per-queue concurrency limits and bound pending work so a few slow generations cannot crowd out short operational requests.

Quanta fits teams that want one pay-as-you-go balance across supported models while managing sustained workloads. Calls are metered on real tokens, with cache writes and cache reads priced separately. Track those categories alongside retries and incomplete attempts when evaluating cost. Open a Quanta account to test your streaming policy against a controlled workload before expanding traffic.

06 · Step 6: Roll out with failure-path tests

Test delayed first events, long pauses, abrupt disconnects, and continuously active streams that exceed the total deadline. Confirm that interrupted jobs release capacity and remain distinguishable from completed jobs.

Roll out to a small queue first. Compare completion rates, inactivity gaps, runtime, occupied concurrency, and token usage before increasing exposure. Adjust each timeout from observed behavior; do not solve every failure by extending every limit.