Request limits and retries

Calls to models are bounded by a two-tier bucket, and under peak load the platform may briefly refuse service. Both signals — 429 and 503 — mean "retry later", not "the request is invalid". Neither one consumes quota.

Two bucket tiers

• Per key — the default setting is 600 requests per minute before division across backend replicas. On each replica, the bucket is tied to the key, not to the IP address: a client behind NAT does not share it with other keys, and the same key from different addresses enters one bucket on that replica.
• Per user — the default setting is 1500 requests per minute in total across all keys of one user before division across replicas. This protects the platform when a single account hands out a dozen keys to agents and each one draws down its own limit.

A request is checked first against the key, then against the user. The first exhausted counter returns 429.

For individual keys — partner integrations, batch processors — the platform administrator can raise the per-key limit. The per-user limit continues to apply.

The `429` response

The rate_limit_exceeded code arrives by two paths: when the platform itself imposed the limit and when the model cluster itself throttled the request. Below is the platform-limit response; the second path is described at the end of the section.

In this example, the per-key limit is set to 600, three backend replicas are running, and the key has no individual setting. Each replica applies ceil(600 / 3) = 200 requests per minute.

{
  "error": {
    "type": "rate_limit_exceeded",
    "code": "rate_limit_exceeded",
    "message": "Rate limit exceeded (per-key). Limit: 200 per minute. Retry after 12 seconds.",
    "scope": "per-key",
    "limit": 200,
    "retryAfter": 12
  }
}

The error.limit and error.scope fields arrive only on the platform-limit path. The error.limit field contains the bucket limit on the replica that handled the request. A different replica count or an individual key setting produces a different value.

On this path the scope field takes one of two values:

• per-key — the limit of a specific key is exhausted. Reduce the number of concurrent requests for this key.
• per-user — the shared account limit is exhausted. Reduce the total load across all keys or ask the platform administrator to raise the limit.

The platform-limit 429 comes with headers:

• Retry-After: <seconds> — how long to wait before the next attempt.
• X-RateLimit-Scope: per-key | per-user — the tier that was hit. Useful for client logs and metrics. On the cluster-throttling path this header is absent.

The limit is checked after the key is validated, so requests with an invalid key (401) do not consume quota.

The second path — the model cluster itself throttled the request. The code is the same, rate_limit_exceeded, but the limit was not imposed by the platform: the body carries the providerStatusCode field, the scope and limit fields are absent, the X-RateLimit-Scope header is absent as well, and the pause comes from Retry-After (in streaming mode, from the retryAfter field of the error frame). Branch on the code and on the pause, and check that the scope field and the X-RateLimit-Scope header are present instead of treating them as guaranteed for this code.

Transient `503`: `pool_exhausted` and `db_transient`

Besides 429, which means the client quota was exceeded, the platform may return 503 with one of two codes. pool_exhausted — the internal database connection pool is temporarily exhausted, for example during a simultaneous peak across several clients. db_transient — a database transaction closed or expired before the operation finished, and the platform rolled it back whole. The causes differ, the reaction is the same: this is a "try again in a few seconds" signal, not "the infrastructure is broken". The response always contains a Retry-After header — an integer number of seconds from 3 to 7, with a random spread on the server side so that mass recovery does not hit the pool again.

In ordinary (non-streaming) mode, handle both codes the same way — off the 503 status together with Retry-After, not off a list of strings you happen to know: that way your client survives the arrival of a third transient code.

{
  "error": {
    "code": "pool_exhausted",
    "type": "server_error",
    "retryAfter": 5
  }
}

In streaming mode (stream: true) status 200 is already sent before the platform learns about the overload, so the signal arrives as the last stream event: data: { "error": { "code": "pool_exhausted", "retryAfter": <number> } }, followed by the usual data: [DONE]. Recognize code === "pool_exhausted" and retry the request after retryAfter seconds.

There is also a third overload signal — 429 ai_congested. It arrives when the AI cluster pool is overloaded. The request was not executed and there is no charge. You can tell it apart from a quota overrun by the X-AI-Admission: shed header, which is absent from a 429 rate_limit_exceeded response.

The `429 ai_pacing_limited` response

A fourth 429 response has nothing to do with platform load. It arrives when the Bitrix24 account administrator has enabled AI-quota pacing and the call trips the daily or weekly window. Its body comes in the { "success": false, "error": { ... } } envelope rather than the raw shape of the other errors on this page, and it carries the reason, overageDenied, and resetAt fields. The fields and the modes are covered on the Company AI quota page.

The `429 ai_provider_cooldown` response

A fifth 429 response arrives when the model cluster answers with errors repeatedly and the platform stops calling it, so that retries do not keep it from recovering. The pause clears itself; the Retry-After header carries what is left of it, in seconds. The request was not executed and there is no charge.

The pause is not a fixed length: the first one is about a minute, but if the cluster is still failing when it expires, the next pause gets longer (up to four minutes). Take the wait from the Retry-After header rather than from a constant in your own code.

You can tell it apart from the other 429 responses by its code and by the headers it lacks: it carries neither X-RateLimit-Scope (present on rate_limit_exceeded when the platform itself imposed the limit) nor X-AI-Admission (present on ai_congested). In streaming mode status 200 has already been sent, so the signal arrives as the last event of the stream — the same way pool_exhausted does, with a retryAfter field inside the error frame.

The `429 ai_deadline_exceeded` response

A sixth 429 response arrives when a request does not fit inside its service deadline — the longest the platform is willing to hold a call before refusing it honestly. Such a call used to simply time out with no body; now it has a declared bound and a readable refusal.

The body carries the code ai_deadline_exceeded, the type rate_limit_exceeded, and a retryAfter field. Two headers come with it: Retry-After — the pause before a retry, in seconds, and X-AI-Deadline-Ms — the budget this request actually had, in milliseconds.

You can specify a shorter budget of your own with the X-AI-Deadline-Ms request header (milliseconds). It only shortens the deadline: a value above the platform one is not granted, and where an administrator has switched the deadline off, the header does not switch it on. A non-numeric or non-positive value counts as absent and never rejects the call.

The deadline is measured from the moment the request arrived and covers the whole of its processing, including the automatic retry on a fallback model. In streaming mode status 200 has already been sent, so the refusal arrives as the last event of the stream — as pool_exhausted and stream_idle_timeout do, with a retryAfter field inside the error frame.

Telling it apart from stream_idle_timeout is easy: that one means "the cluster went quiet" — mid-answer, or even at the very start, never sending headers — while this one means "there is no time left for the whole request", and it fires even while chunks keep arriving.

Errors

The full list of common API errors — Errors.

Ready-made retry recipe

async function callWithBackoff(makeRequest, { maxAttempts = 5 } = {}) {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const res = await makeRequest()
    if (res.status === 429 || res.status === 503) {
      const retryAfterSec = Number(res.headers.get('retry-after')) || 0
      // Honor Retry-After and add a small random spread
      // so that parallel workers do not return in sync.
      const baseMs = retryAfterSec * 1000 || Math.min(1000 * 2 ** attempt, 30000)
      const jitterMs = Math.floor(Math.random() * 1000)
      await new Promise(r => setTimeout(r, baseMs + jitterMs))
      continue
    }
    return res
  }
  throw new Error('Retries exhausted')
}

What matters:

• Always honor Retry-After. It matches the expected recovery time. A shorter interval speeds nothing up and only adds load to the platform.
• Add your own random spread on top of the server one — a delay of 100-1000 ms breaks synchronous worker loops.
• Cap the number of attempts. Five is enough for most scenarios. Beyond that, propagate the error to the caller.
• On 5xx without Retry-After, for example on a provider-side failure, grow the pause exponentially: min(2^attempt × 1s, 30s) with a random spread.
• Do not run Promise.all over thousands of records. Without pauses batch processing will hit one of the bucket tiers.

See also

• Create a chat completion
• Streaming
• Limits and optimization
• Errors