LLM streaming errors — SSE cutoffs, hanging streams and missing usage

Reproduce with curl -N straight to the API

Bypass every proxy. If raw SSE flows fine for the full generation, your app's path is the problem — reintroduce hops one at a time:

curl -N https://api.kunavo.com/v1/chat/completions \
  -H "Authorization: Bearer $KUNAVO_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"claude-sonnet-5","stream":true,
       "stream_options":{"include_usage":true},
       "max_tokens":300,
       "messages":[{"role":"user","content":"Count slowly to 20 in words."}]}'

Fix the hop that breaks it

nginx: proxy_buffering off + proxy_read_timeout 300s for the route. Serverless: check the platform's response-streaming limits. Corporate proxies: some can't do SSE at all — fall back to non-streaming there.

Handle the tail correctly

Streamed billing data arrives last: the final chunk carries usage (when requested) before [DONE]. Aggregate deltas, read usage from the last chunk, and treat an early close (no finish_reason) as retryable.

“SSE stream ended without [DONE]” — was the answer whole?

That message is your client's own check, not an error the API sent: the connection closed before the data: [DONE] line an OpenAI-compatible stream ends with. Three things cause it: a hop in between closed the connection (the proxy timeout and buffering causes above); the server failed mid-answer and closed without a terminal frame; or the answer was complete and only the sentinel was lost. The last chunk you received decides which: a finish_reason there means the text is whole, and no finish_reason means it was cut and should be retried. Do not leave that check to the SDK — the OpenAI Python SDK ends its loop quietly when the connection closes without [DONE], and raises only when a chunk carries an error object. Put the check in your own code:

from openai import OpenAI

client = OpenAI(base_url="https://api.kunavo.com/v1", api_key="sk-kn-...")

finish_reason, parts = None, []
stream = client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": "Count slowly to 20 in words."}],
    stream=True,
)
for chunk in stream:  # raises openai.APIError on a chunk that carries "error"
    for choice in chunk.choices:
        parts.append(choice.delta.content or "")
        finish_reason = choice.finish_reason or finish_reason

if finish_reason is None:
    raise RuntimeError("stream closed without a finish_reason: cut off, retry it")
print("".join(parts))

An empty stream: 200, then nothing

Sometimes the stream opens with HTTP 200 and ends without a single content chunk — no text, no tool call, sometimes not even the role chunk. That is almost always an upstream that failed after the headers went out: an overloaded provider, a gateway whose own upstream refused the request, or a proxy that dropped the body. Treat it like a cut-off stream and retry with backoff, and log the raw body of one failing response, because the cause is often an error event inside the stream that your SDK skipped. Claude Code reacts to the same condition by retrying the request without streaming.

Why is usage null on my streamed responses?

On OpenAI-wire APIs usage isn't included in streams unless you pass stream_options: {"include_usage": true}; it then arrives in the final chunk. Anthropic's native stream reports usage in message_start/message_delta events.

How do I detect a stream that was cut off versus one that finished?

A finished stream ends with a finish_reason (or Anthropic's message_stop) and then [DONE]. A connection that closes without those markers was cut — treat it as a retryable failure, not a short answer.

What does “SSE stream ended without [DONE]” mean?

Your client read the stream to the end of the connection and never saw the data: [DONE] line that closes an OpenAI-compatible stream. The API did not send that message; your client or agent wrote it. If the last chunk carried a finish_reason, the answer is complete and only the close was lost. If it did not, the answer was cut off by a proxy, a timeout or a server-side failure, and the request should be retried.

What does “stream ended without finish_reason” mean?

The client read the stream to its end and no chunk carried a finish_reason — the field an OpenAI-compatible stream uses to say the answer is complete (stop, length, tool_calls). Without it the text you have is a fragment, however natural its last sentence looks. Retry the request; if it repeats, look for a proxy timeout or buffering between you and the API.

Why would an LLM API return an empty stream?

Because the failure happened after the HTTP headers were sent: the provider was overloaded, a gateway's upstream refused the request, or a proxy dropped the body. The status line still says 200, so status checks miss it. Treat a stream with no content chunk as a failed request, retry it with backoff, and log one raw response to find the error event inside it.

Is it safe to ignore a missing [DONE] if I already have text?

Only when the last chunk carried a finish_reason. Without one, the text you have is a fragment that can end mid-sentence or mid tool-call, with its JSON arguments incomplete. Retry it rather than saving it as the answer.