For AI agents: markdown of this page — /docs-content-en/ai/chat/streaming.md documentation index — /llms.txt

Streaming (Server-Sent Events)

With stream: true the model response arrives as a stream of events as tokens are generated, rather than as a single chunk at the end. The first byte reaches the client within seconds regardless of the full response length.

Response headers: Content-Type: text/event-stream, Cache-Control: no-cache, Connection: keep-alive. Each event is a data: {JSON}\n\n line, and the terminating event is data: [DONE]\n\n.

Event format

data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"delta":{"role":"assistant","content":"CRM"}}]}

data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"delta":{"content":" is"}}]}

data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"finish_reason":"stop","delta":{}}],"usage":{"prompt_tokens":10,"completion_tokens":15,"total_tokens":25}}

data: [DONE]

usage arrives in the last event before [DONE]. Accumulate delta.content from all events to assemble the full response text.

Stream handling example

javascript 
const response = await fetch('https://vibecode.bitrix24.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'X-Api-Key': 'YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'bitrix/bitrixgpt-5.5',
    messages: [{ role: 'user', content: 'Tell me about CRM' }],
    stream: true,
  }),
})

const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ''

while (true) {
  const { done, value } = await reader.read()
  if (done) break
  buffer += decoder.decode(value, { stream: true })
  const lines = buffer.split('\n')
  buffer = lines.pop() ?? ''

  for (const line of lines) {
    if (!line.startsWith('data: ')) continue
    const payload = line.slice(6)
    if (payload === '[DONE]') return
    const chunk = JSON.parse(payload)
    const delta = chunk.choices?.[0]?.delta?.content
    if (delta) process.stdout.write(delta)
  }
}

Choosing a mode under load

In synchronous mode (stream: false) the full model response is assembled on the server and sent to the client as a single response — the first byte arrives together with the last. For a model with a long reasoning phase, bitrix/bitrixgpt-5.5-thinking, on a long context and with a detailed answer, generation takes tens of seconds. If the client timeout is shorter than the full generation time, the connection closes without receiving a single byte — for example, cURL error 28: Operation timed out, 0 bytes received. On short requests generation fits within the timeout and the effect does not appear.

In streaming mode (stream: true) the server sends status 200 and the headers right after processing begins, and events flow as tokens are generated, including the reasoning phase. For large requests and reasoning models use stream: true — the client timeout on receiving the first byte then does not fire.

Guidelines for the client timeout:

  • Streaming mode (stream: true) — set the timeout for receiving the first event with a margin for the model's reasoning. A reasoning model spends a few seconds before the first token; a value of 30 seconds or more covers this phase with a margin.
  • Synchronous mode (stream: false) — the whole-request timeout must cover the full generation time, not just the network exchange. For a large request to a reasoning model this is tens of seconds. In addition, on a provider error or timeout a synchronous request is retried automatically, and then the final response or error arrives after a few minutes. So values of 9-15 seconds are not enough: set the whole-request timeout with a margin for such retries — a few minutes — or switch to streaming mode, where the timeout on receiving the first byte removes the problem.

Known specifics

There is no fallback model in the stream. On a provider failure a synchronous request is retried with the default model, while in streaming mode the client receives the error in the last event before data: [DONE].

Errors arrive inside the stream. Status 200 has already been sent when the platform learns of overload or of an interrupted generation, so a service event of the form data: { "error": { "code": "pool_exhausted", "retryAfter": 5 } } arrives before data: [DONE]. Read the stream to the end and check the error field.

An incomplete structured response also arrives as an error event. If the request carried response_format and the model did not produce a complete JSON, data: { "error": { "code": "structured_output_truncated", "message": "...", "type": "invalid_request_error" } } arrives before data: [DONE]. This event has only three fields — code, message and type. The finishReason, param and suggestedMaxTokens fields carried by the synchronous 422 are not present here. The message text differs by finish reason: generation was cut off with finish_reason: "length" — retrying with a raised max_tokens is suggested, the stream ended without a finish reason — the response may be incomplete, the model finished on its own with another reason — there is no parseable JSON and a model without reasoning is recommended. See guaranteed JSON response.

A stalled response is terminated by an idle timeout. If the model sends headers but then goes silent mid-response and delivers no new data for longer than the idle window, the platform terminates the call and sends data: { "error": { "code": "stream_idle_timeout", "type": "server_error", "retryable": true, "retryAfter": <seconds> } } before data: [DONE]. The error is retryable — retry the request, honoring retryAfter. A continuous flow of tokens (including reasoning tokens from thinking models) resets the idle timer and is never subject to this timeout.

See also