Streaming failures are rarely the model — they're the plumbing between you and it. Idle-timeout proxies kill quiet connections, nginx buffers eat events, and half-consumed streams look like "the API stopped responding". Work through the plumbing first.
The error
- Stream stops mid-sentence, connection closed (no error event)
- Client hangs after the last token, never sees [DONE]
- usage is null on streamed responses
- Works in curl, dies behind nginx / a corporate proxyCauses and fixes at a glance
| Cause | Fix |
|---|---|
| Proxy/load-balancer idle timeout (60s default in many setups) | Raise read timeouts for the API path; long thinking pauses look idle to a proxy. |
| Buffering in front of SSE (nginx proxy_buffering, some CDNs) | Disable buffering for the streaming route (X-Accel-Buffering: no / proxy_buffering off). |
| Client stops consuming (await missing, iterator dropped) | Consume to the end or explicitly close — a GC'd iterator mid-stream is indistinguishable from a cutoff. |
| Expecting usage without asking for it | OpenAI-wire: pass stream_options: {"include_usage": true} — usage arrives in the final chunk. |
Reproduce with curl -N straight to the API
Bypass every proxy. If raw SSE flows fine for the full generation, your app's path is the problem — reintroduce hops one at a time:
curl -N https://api.kunavo.com/v1/chat/completions \
-H "Authorization: Bearer $KUNAVO_API_KEY" \
-H "content-type: application/json" \
-d '{"model":"claude-sonnet-4-6","stream":true,
"stream_options":{"include_usage":true},
"max_tokens":300,
"messages":[{"role":"user","content":"Count slowly to 20 in words."}]}'Fix the hop that breaks it
nginx: proxy_buffering off + proxy_read_timeout 300s for the route. Serverless: check the platform's response-streaming limits. Corporate proxies: some can't do SSE at all — fall back to non-streaming there.
Handle the tail correctly
Streamed billing data arrives last: the final chunk carries usage (when requested) before [DONE]. Aggregate deltas, read usage from the last chunk, and treat an early close (no finish_reason) as retryable.
If you're calling through Kunavo
Kunavo streams standard OpenAI-wire SSE (with stream_options.include_usage supported) and Anthropic-wire events on /v1/messages, so the curl reproduction above is also the compatibility test. Streamed responses that fail before completion are not billed.
FAQ
Why is usage null on my streamed responses?
On OpenAI-wire APIs usage isn't included in streams unless you pass stream_options: {"include_usage": true}; it then arrives in the final chunk. Anthropic's native stream reports usage in message_start/message_delta events.
How do I detect a stream that was cut off versus one that finished?
A finished stream ends with a finish_reason (or Anthropic's message_stop) and then [DONE]. A connection that closes without those markers was cut — treat it as a retryable failure, not a short answer.
More error semantics live in the error reference; getting a key takes a minute via sign up and the authentication docs.