## Run a command

`POST /v1/infra/servers/:id/exec`

Runs a shell command on a BLACKHOLE server through the tunnel agent. The full standard output (`stdout`), error stream (`stderr`), and return code are passed to the client. By default the response is JSON after the command finishes (recommended for AI agents and scripts). If you need line-by-line progress — pass `?stream=true` and read the SSE events `stdout`/`stderr`/`exit`. A failure during command execution arrives in streaming mode as an `error` event.

**For AI agents and MCP clients always use JSON mode (without `?stream=true`)** — they cannot parse SSE.

## Parameters

| Parameter | In | Type | Required | Default | Description |
|----------|---|-----|:-----:|-----------|----------|
| `id` | path | string (UUID) | yes | — | BLACKHOLE server ID, `status: running`, `blackholeStatus: CONNECTED` |
| `stream` | query | string | no | `false` | `true` — SSE mode; otherwise — JSON response |

## Request fields (body)

| Field | Type | Required | Description |
|------|-----|:-----:|----------|
| `command` | string | **yes** | Shell command. 1–10,000 characters |
| `timeout` | number | no | Execution timeout in seconds: 1–600. Default 300 |
| `workdir` | string | no | Working directory. Up to 500 characters |
| `env` | object | no | Environment variables: `{ "KEY": "value" }`. Only string values |

## Examples

### curl — personal key

```bash
curl -X POST "https://vibecode.bitrix24.com/v1/infra/servers/SERVER_ID/exec?stream=false" \
  -H "X-Api-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"command": "ls -la /opt/app", "timeout": 30}'
```

### curl — OAuth application

```bash
curl -X POST "https://vibecode.bitrix24.com/v1/infra/servers/SERVER_ID/exec?stream=false" \
  -H "X-Api-Key: YOUR_APP_KEY" \
  -H "Authorization: Bearer USER_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"command": "npm ci --production", "workdir": "/opt/app", "timeout": 180}'
```

### JavaScript — personal key

```javascript
const res = await fetch(
  `https://vibecode.bitrix24.com/v1/infra/servers/${serverId}/exec?stream=false`,
  {
    method: 'POST',
    headers: {
      'X-Api-Key': 'YOUR_API_KEY',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      command: 'node -v',
      timeout: 10,
    }),
  }
)
const { data } = await res.json()
console.log(`exit ${data.exitCode} in ${data.duration}s:\n${data.stdout}`)
```

### JavaScript — OAuth application

```javascript
// A command with environment variables and a working directory
const res = await fetch(
  `https://vibecode.bitrix24.com/v1/infra/servers/${serverId}/exec?stream=false`,
  {
    method: 'POST',
    headers: {
      'X-Api-Key': 'YOUR_APP_KEY',
      'Authorization': 'Bearer USER_SESSION_TOKEN',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      command: 'npx tsx scripts/seed.ts',
      workdir: '/opt/app',
      env: { DATABASE_URL: 'postgresql://localhost/mydb' },
      timeout: 60,
    }),
  }
)
```

## Response fields

| Field | Type | Description |
|------|-----|----------|
| `success` | boolean | `true` if the command ran (including a non-zero exitCode) |
| `data.exitCode` | number | The process's return code |
| `data.stdout` | string | The command's standard output |
| `data.stderr` | string | The command's error stream |
| `data.duration` | number | Execution time in seconds |
| `data.truncated` | boolean | `true` if `stdout`/`stderr` were truncated by size (5 MB per stream) |

## Response example

```json
{
  "success": true,
  "data": {
    "exitCode": 0,
    "stdout": "Linux epd65hdv07p8g89c0c06 6.8.0-107-generic #107-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 13 19:51:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux\n",
    "stderr": "",
    "duration": 5,
    "truncated": false
  }
}
```

## Error response example

409 — another operation is already running on the server. This is a failure before the command starts, so the HTTP status reflects the outcome:

```json
{
  "success": false,
  "error": {
    "code": "EXEC_BUSY",
    "message": "Another operation is running on this server",
    "retryable": true,
    "retryAfter": 10
  }
}
```

The response also carries a `Retry-After` HTTP header (in seconds, equal to the `retryAfter` field) — this is a short poll interval: keep retrying with it until the lock clears. The full upper bound of "when the lock auto-expires" is in `error.hint.autoExpiresInSeconds`.

## Errors

The endpoint's errors fall into two groups, and they are delivered differently. A failure before the command starts arrives with a real HTTP status. A failure during execution arrives in the response body.

### Errors before the command starts

| HTTP | Code | Description |
|------|-----|----------|
| 400 | `VALIDATION_ERROR` | The request schema is violated (empty command, invalid timeout, `workdir` longer than 500 characters) |
| 400 | `NOT_BLACKHOLE` | Server in OPEN mode — the Deploy API is unavailable |
| 400 | `COMMAND_TOO_LONG` | The command is longer than 10,000 characters. The response carries a `hint`: ship large payloads and scripts via [`/upload`](./upload.md), then run `bash /path/script.sh` |
| 401 | `MISSING_API_KEY` | The `X-Api-Key` header was not provided |
| 401 | `INVALID_API_KEY` | Invalid or expired API key |
| 404 | `NOT_FOUND` | The server does not exist, was deleted, or belongs to another API key |
| 409 | `EXEC_BUSY` | Another `/exec` or `/deploy` is already running on the server. The response carries a `Retry-After` header and `retryable: true` / `retryAfter` (seconds) fields — retry at that interval. Or release the lock via [`/lock`](./lock.md). If `EXEC_BUSY` persists after releasing the lock — unstick the channel via `POST /v1/infra/servers/:id/unstick` (see below) |
| 409 | `SERVER_NOT_READY` | The server is not ready for the operation: it is not running or the tunnel is not connected. The response carries a `hint` field with the reason and the next step. Wake the server or call [`/repair`](/docs/infra/lifecycle/repair) and repeat the request. The "listed as connected but the Gateway has no live tunnel" case on this route arrives as `502 TUNNEL_NOT_FOUND` (see below) |
| 429 | `RATE_LIMITED` | The limit of 10 operations per minute per server was exceeded |

### Errors during execution

> **The HTTP status does not reflect the command's outcome.** On a separate virtual machine (`kind: "STANDALONE"`) the connection is held open for the whole run, and the `200` status is sent before execution begins — so both a successful and a failed command arrive with the code `200`. The failure marker is the `success: false` field and the `error` object in the response body. **Check `success` in the body, not the HTTP status:** a client that branches on the status will treat a failed command as successful. For a galaxy app (`kind: "GALAXY_APP"`) the same error arrives with the status `502`.

| Code | Description |
|-----|----------|
| `EXEC_TIMEOUT` | The `timeout` was exceeded (or the standard 300 seconds). The agent terminates the whole process group forcibly (SIGKILL, no grace period). The response carries a `hint` object, its fields are described below |
| `EXEC_FAILED` | An execution error on the agent |
| `EXEC_BUSY` | The agent's own exec mutex is busy — it is already running another command. This differs from the platform lock (`409` before the start): it arrives during execution, in the body as `success: false` (on `STANDALONE` — with HTTP `200`, on `GALAXY_APP` — with `502`). The response carries a `hint` with the recovery path via [`/unstick`](#unsticking-a-stuck-channel) |

#### The `hint` fields on `EXEC_TIMEOUT`

| Field | Type | Description |
|-------|------|-------------|
| `hint.reason` | string | What happened: the command did not finish within the allotted time, and the whole process group was terminated immediately, with no grace period |
| `hint.recovery` | string | What to do: the upper bound for `timeout` is 600 seconds, so run longer operations as a background job (`systemd-run --unit=<name>`) and follow it via [`GET /v1/infra/servers/:id/logs`](./logs.md) with the `?service=<name>` parameter; for progress within the timeout pass `?stream=true` |
| `hint.recoveryAction` | string | A pointer to the documentation: `docs: /docs/infra/deploy/exec` |

The hint arrives both in the JSON response and in the SSE `error` event of streaming mode. An agent-side `EXEC_BUSY` carries a `hint` object of the same shape — there `recoveryAction` points to `POST /v1/infra/servers/:id/unstick`.

A tunnel communication failure arrives in the same shape — with its own code in the `error.code` field (for example `TUNNEL_NOT_FOUND` or `GATEWAY_TIMEOUT: …`). Some gateway codes carry details after a colon (`GATEWAY_UNREACHABLE: …`, `GATEWAY_TIMEOUT: …`), so match `error.code` by prefix, not by an exact comparison. In streaming mode (`?stream=true`) the same errors arrive as an SSE `error` event. The event data carries `code` and `message`, and for `EXEC_TIMEOUT` and an agent-side `EXEC_BUSY` a `hint` as well. There is no `success` field in it — the `error` event itself is the failure marker.

Full list of common API errors — [Errors](/docs/errors).

## Known specifics

- **The command runs through `/bin/sh`.** The `command` field is executed by the `/bin/sh` interpreter — a minimal POSIX shell (`dash`), not `bash`. Constructs specific to `bash` (`set -o pipefail`, `[[ … ]]`, arrays) are unavailable in it and fail. Run them explicitly — via `command: "bash -c 'set -o pipefail; …'"` or as a script file `command: "/bin/bash /opt/app/script.sh"`.
- **Server-level locking.** While an `/exec` or `/deploy` is running, a second such call returns 409 `EXEC_BUSY`. If the previous `/deploy` was interrupted (for example, failed at the `healthcheck` step or the client aborted the connection on a timeout), the server lock remains held until it expires automatically — for `/deploy` this is up to 15 minutes, for `/exec` — until the command's `timeout` expires plus 90 seconds. The lock can be released immediately by calling [`DELETE /v1/infra/servers/:id/lock`](./lock.md), after which `/exec` works again. There is no need to recreate the server.
- **The maximum size of `stdout`/`stderr` is 5 MB per stream.** If the command's output is larger, `truncated: true` arrives and the tail is cut off. The command finished anyway, and `exitCode` is correct. For large output, redirect to a file: `command: "my-cmd > /opt/app/output.log 2>&1"` and then read it via `/exec cat /opt/app/output.log`.
- **The `.env` from `/deploy` is not picked up automatically.** Systemd loads `.env` when the service starts, but not for one-off commands via `/exec`. If you need `DATABASE_URL` / API keys — pass them in the `env` field: `{ env: { DATABASE_URL: "..." } }`.
- **The two timeouts together give `timeout + 30` seconds.** The agent kills the process exactly at `timeout`; the Gateway waits another 30 seconds for the final events and only then returns `EXEC_TIMEOUT`.
- **Keeping the connection alive in JSON mode.** If the command runs longer than 15 seconds, the server periodically sends spaces in the response body — this prevents timeouts in nginx and intermediate proxies. The JSON parser ignores leading and trailing spaces, so the client simply reads valid JSON once the command completes.
- **Client timeout: make sure it is inter-byte, not total.** The keepalive spaces reset "silence" timeouts, so the classic Python socket timeouts (`urlopen(..., timeout=300)`, the `requests` read timeout) will not by themselves abort a long command in JSON mode. Two other things do abort it. First — clients and wrappers with a true wall-clock deadline for the whole request (for example, `aiohttp` with `ClientTimeout(total=…)` or an agent framework's own deadline): keepalive cannot help against those. Second — the server-side `timeout` of the command itself (300 seconds by default): once it expires, the agent kills the process and you get `EXEC_TIMEOUT` regardless of the client settings — do not mistake it for a client-side disconnect. In practice: in Python set split timeouts — `requests.post(url, json=body, headers=headers, timeout=(10, 60))` — 10 seconds for connect and 60 for the inter-byte pause; if the client does drop the connection, the server-side lock persists for up to `timeout + 90` seconds (`EXEC_BUSY` for repeated calls). Commands longer than a few minutes are better run as a background job — see "Background jobs" below.

## Unsticking a stuck channel

If `/exec` still returns `EXEC_BUSY` after [`DELETE /v1/infra/servers/:id/lock`](./lock.md) (or `/deploy` fails with code `DEPLOY_FAILED` and the message "Another command is running"), the agent-side exec mutex has leaked — a background process is holding the channel open, so the server lock is released but the channel stays busy. `POST /v1/infra/servers/:id/unstick` releases the platform-side lock AND bounces the agent tunnel, which on reconnect finishes the stuck command and frees the mutex. The server is not rebooted.

If a legitimate operation (a deploy, exec, or harden) is still running on the server when you call it, the endpoint returns `409 OPERATION_IN_PROGRESS` by default and leaves it alone — otherwise it would abort the running command. Only a genuinely stuck channel should be unstuck; if you are certain it is hung, retry with `?force=true`.

Response: `{ "success": true, "data": { "backendLockReleased": true, "agentBounced": true, "reconnected": true } }`.


Unsticking errors:

| HTTP | Code | Description |
|------|------|-------------|
| 404 | `SERVER_NOT_FOUND` | The server does not exist, was deleted, or belongs to a different API key. The same code arrives if the server record disappeared between the access check and the unsticking |
| 409 | `OPERATION_IN_PROGRESS` | A legitimate operation (a deploy, exec, or harden) is running on the server — the channel is not stuck. The `message` field names the operation. Retry with `?force=true` only if you are certain the channel really is hung |
| 409 | `CONFLICT` | Unsticking of this server is already in progress — wait for it to finish |
| 409 | `GALAXY_UNSTICK_UNSUPPORTED` | Unsticking is not supported for a galaxy host or a galaxy app: the host is shared, and bouncing its tunnel would abort the commands of neighbouring applications |
| 429 | `RATE_LIMITED` | The limit of 6 requests per minute per "API key + server" pair was exceeded |
| 502 | `GATEWAY_ERROR` | The platform-side lock was released, but the agent tunnel could not be bounced — the gateway is unreachable. Recovery is incomplete, retry the request |

## Choosing the mode

Three call modes and when each one fits:

| When | How | Monitoring |
|------|-----|------------|
| The command fits into 600 seconds, no progress needed | `{ "command": "...", "timeout": 600 }` without `?stream=true` — a blocking JSON response | `exitCode`, `stdout`, `stderr` in the response body |
| The command fits into `timeout`, live progress needed | the same request with `?stream=true` — SSE events `stdout` / `stderr` / `exit` | read the events as they arrive |
| The command runs longer than 600 seconds, or the channel must be freed | a background job `systemd-run --unit=<name>` — `/exec` returns immediately | [`GET /v1/infra/servers/:id/logs?service=<name>`](./logs.md) |

Do not set the client-side overall request deadline lower than the command's `timeout` — the client would cut the connection before the server can answer. Split timeouts in Python:

```python
import requests
r = requests.post(
    f"{VIBE_URL}/v1/infra/servers/{SERVER_ID}/exec",
    headers={"X-Api-Key": VIBE_API_KEY},
    json={"command": "npm ci --production", "workdir": "/opt/app", "timeout": 600},
    timeout=(10, 60),   # 10 seconds to connect, 60 for a pause between bytes
)
```

Raising the `/exec` `timeout` does not help the `install` step — deploy steps have their own 300-second limit, separate from the 600 of `/exec`.

⚠️ The `timeout` default is **300 seconds**, not 600. The 600 ceiling exists, but to use it the field must be passed explicitly. A command planned for nine minutes and started without `timeout` is killed at the fifth.

## Background jobs

The `/exec` `timeout` is capped at 600 seconds, and when it expires the agent terminates the whole process group forcibly (SIGKILL, no grace period). Run anything longer — package installation, a database dump restore, a heavy build — as a background job: `/exec` returns instantly, the channel is freed (a second call will not hit `EXEC_BUSY`), and the process keeps running independently of the HTTP connection.

### Recommended — a transient systemd unit

```json
{ "command": "systemctl reset-failed restore-db 2>/dev/null; systemd-run --unit=restore-db /bin/bash /opt/data/restore.sh", "timeout": 30 }
```

The job gets its own cgroup and lives independently of the exec session. Manage it with standard calls:

- status: `{ "command": "systemctl is-active restore-db" }` — `active` (running), `inactive` (finished successfully), `failed` (crashed).
- logs: journald picks up the unit's output automatically — read it via [`GET /v1/infra/servers/:id/logs?service=restore-db`](./logs.md).
- `systemctl reset-failed <name>` before a rerun — otherwise systemd refuses to create a unit with the name of the failed job.

### Lightweight — background with output redirect

```json
{ "command": "(cd /opt/app && npm run build > /tmp/build.log 2>&1 &)", "timeout": 10 }
```

Redirecting both streams to a file is mandatory — it releases the exec session's pipes. Progress: `{ "command": "tail -20 /tmp/build.log" }`.

### Anti-pattern — bare `nohup`

`nohup cmd &` **without a redirect** does not work: in a pipe (rather than a terminal) `nohup` does not redirect the output, the background process inherits the exec session's pipe, and the agent waits for it to close right up to the `timeout` — then forcibly terminates the whole process group, including your "background" process. Always add `> file 2>&1`.

Pass secrets (database passwords and the like) via the `env` field, not inside `command` — the agent logs executed commands (the first 200 characters). Also keep in mind that the `install`/`preStart` steps of [`/deploy`](./deploy.md) have their own 300-second timeout — move heavy installations into a one-off `/exec` or a background job, not into the install script.

## See also

- [Full deploy](./deploy.md) — `POST /v1/infra/servers/:id/deploy` — a chain of several `/exec` with a health check.
- [Release a stuck lock](./lock.md) — `DELETE /v1/infra/servers/:id/lock` — on `EXEC_BUSY` after an abort.
- [Service logs](./logs.md) — `GET /v1/infra/servers/:id/logs` — the system journal via `journalctl` plus monitoring of background jobs started via `systemd-run` (`?service=<unit name>`).
- [Upload a file](./upload.md) — `POST /v1/infra/servers/:id/upload` — write a file to the server.
- [Repair the tunnel](/docs/infra/lifecycle/repair) — on `SERVER_NOT_READY`.
- [A fast release cycle](./fast-cycle.md) — techniques that shorten the edit loop without a full deploy.