For AI agents: markdown of this page — /docs-content-en/ai/chat/streaming.md documentation index — /llms.txt
Streaming (Server-Sent Events)
With stream: true the model response arrives as a stream of events while tokens are generated, rather than as a single chunk at the end. The first byte reaches the client within seconds regardless of the full response length.
Response headers: Content-Type: text/event-stream, Cache-Control: no-cache, Connection: keep-alive. Each event is a data: {JSON}\n\n line, and the terminating event is data: [DONE]\n\n.
Event format
data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"delta":{"role":"assistant","content":"CRM"}}]}
data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"delta":{"content":" is"}}]}
data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"finish_reason":"stop","delta":{}}],"usage":{"prompt_tokens":10,"completion_tokens":15,"total_tokens":25}}
data: [DONE]usage arrives in the last event before [DONE]. Accumulate delta.content from all events to assemble the full response text.
Stream handling example
const response = await fetch('https://vibecode.bitrix24.com/v1/chat/completions', {
method: 'POST',
headers: {
'X-Api-Key': 'YOUR_API_KEY',
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'bitrix/bitrixgpt-5.5',
messages: [{ role: 'user', content: 'Tell me about CRM' }],
stream: true,
}),
})
const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ''
while (true) {
const { done, value } = await reader.read()
if (done) break
buffer += decoder.decode(value, { stream: true })
const lines = buffer.split('\n')
buffer = lines.pop() ?? ''
for (const line of lines) {
if (!line.startsWith('data: ')) continue
const payload = line.slice(6)
if (payload === '[DONE]') return
const chunk = JSON.parse(payload)
const delta = chunk.choices?.[0]?.delta?.content
if (delta) process.stdout.write(delta)
}
}Choosing a mode under load
In synchronous mode (stream: false) the full model response is assembled on the server and sent to the client as a single response — the first byte arrives together with the last. For a model with a long reasoning phase — bitrix/bitrixgpt-5.5-thinking — generation on a long context and with a detailed answer takes tens of seconds. If the client timeout is shorter than the full generation time, the connection closes without receiving a single byte — for example, cURL error 28: Operation timed out, 0 bytes received. On short requests generation fits within the timeout and the effect does not appear.
In streaming mode (stream: true) the server sends status 200 and the headers right after processing begins, and events flow as tokens are generated, including the reasoning phase. For large requests and reasoning models use stream: true — the client timeout on receiving the first byte then does not fire.
Guidelines for the client timeout:
- Streaming mode (
stream: true) — set the timeout for receiving the first event with a margin for the model's reasoning. A reasoning model spends a few seconds before the first token; a value of 30 seconds or more covers this phase with a margin. - Synchronous mode (
stream: false) — the whole-request timeout must cover the full generation time, not just the network exchange. For a large request to a reasoning model this is tens of seconds. In addition, on a provider error or timeout a synchronous request is retried automatically, and then the final response or error arrives after a few minutes. So values of 9-15 seconds are not enough: set the whole-request timeout with a margin for such retries — a few minutes — or switch to streaming mode, where the timeout on receiving the first byte removes the problem.
Known specifics
There is no fallback model in the stream. On a provider failure a synchronous request is retried with the default model, while in streaming mode the client receives the error in the last event before data: [DONE].
Errors arrive inside the stream. Status 200 has already been sent when the platform learns of overload or of an interrupted generation, so a service event of the form data: { "error": { "code": "pool_exhausted", "retryAfter": 5 } } arrives before data: [DONE]. Read the stream to the end and check the error field.
An incomplete structured response also arrives as an error event. If the request carried response_format and the model did not produce complete JSON, data: { "error": { "code": "structured_output_truncated", "message": "...", "type": "invalid_request_error" } } arrives before data: [DONE]. This event has only three fields — code, message and type. The finishReason, param and suggestedMaxTokens fields carried by the synchronous 422 are not present here. The message text differs by finish reason. When generation is cut off with finish_reason: "length", the message suggests retrying with a raised max_tokens. When the stream ends without a finish reason, the message warns that the response may be incomplete. When the model finishes on its own with another reason, the message states that there is no parseable JSON and recommends a model without reasoning. See guaranteed JSON response.
An unparseable structured response arrives as its own event. If the model answered a response_format request with non-empty text that does not parse as a JSON object or array, data: { "error": { "code": "structured_output_invalid_json", "message": "...", "type": "invalid_request_error" } } arrives before data: [DONE]. The event carries the same three fields — code, message and type. The answer chunks have already been delivered by then, so discard what you collected; the stream status stays 200 by protocol. Retrying with a raised max_tokens does not change the result. Recovery from the internal reasoning_content channel is not performed here, unlike the empty-answer case: a document appended after chunks already delivered would merge with them in the client's buffer. See guaranteed JSON response.
Looping reasoning is cut off with a reasoning_repetition event. If the model, before starting its answer, keeps repeating the same sentences in reasoning_content, the platform stops the generation without waiting for the service deadline or the max_tokens limit and sends data: { "error": { "code": "reasoning_repetition", "message": "...", "type": "invalid_request_error" } } before data: [DONE]. The error is not transient: the same request at the same reasoning level will loop again — send it again with reasoning turned off (reasoning_effort: "none"); a larger max_tokens will not help. Reasoning chunks already delivered stay with the client. The platform catches repetition of the same sentences, not a run of different wordings, and looks only at the reasoning before the answer starts: a repetitive answer in content (for example, an array of identical objects) is not cut off. If the model sends the same text into both channels at once, what counts is how it reaches the client: for a reasoning model this text arrives in content and is delivered as the answer; if it arrives only as reasoning_content, its repetition is cut off the same way as reasoning. The exception is response_format: {"type": "json_object"} when the model sends the answer itself through the reasoning_content channel as a JSON document (the reasoning starts with { or [): the cut-off is then deferred while recovery is still possible, and recovery decides — a complete document arrives as the answer, and if nothing can be recovered, reasoning_repetition arrives. If the document is too long to be recovered, the repetition cuts the stream off at once. Without response_format there is no exception: an answer that arrives only in reasoning_content is cut off with a reasoning_repetition event when it loops — recovering the answer from the reasoning is done only for json_object.
A stream also has an overall service deadline. Beyond the idle timeout, a call now has a bound on its total time: if it does not fit, the stream ends with data: { "error": { "code": "ai_deadline_exceeded", "type": "rate_limit_exceeded", "retryable": true, "retryAfter": <seconds> } } before data: [DONE]. Unlike the idle timeout, this deadline fires even while tokens keep arriving — it is about the length of the whole answer, not about a gap in it. You can name a shorter budget of your own with the X-AI-Deadline-Ms request header; the details are on the Limits and overload page.
A stalled response is terminated by an idle timeout. If the model sends headers but then goes silent mid-response and delivers no new data for longer than the idle window, the platform terminates the call and sends data: { "error": { "code": "stream_idle_timeout", "type": "server_error", "retryable": true, "retryAfter": <seconds> } } before data: [DONE]. The error is retryable — retry the request, honoring retryAfter. A continuous flow of tokens (including reasoning tokens from thinking models) resets the idle timer and is never subject to this timeout.
The same event also ends a stream that never started. If the upstream accepted the request but sent no response headers within the allotted budget, the call ends with the same retryable stream_idle_timeout before data: [DONE] instead of silence. Models from every provider behave this way: stream setup takes a single attempt on one shared budget, so the wait never doubles.
A stream always ends with a finish_reason event. If the model finished its answer but sent no terminal event, the platform adds one before data: [DONE] itself: an empty delta with finish_reason: "tool_calls" when the answer contained tool calls, or finish_reason: "stop" otherwise. If the connection to the model dropped before a finish reason arrived, the answer may be incomplete: data: { "error": { "code": "stream_incomplete", "type": "server_error", "retryable": true, "retryAfter": <seconds> } } arrives before data: [DONE]. The error is retryable — retry the request, honoring retryAfter.