Operate

Wait until a server is ready, then read what it serves:

until curl -sf http://127.0.0.1:8000/health > /dev/null; do sleep 5; done
curl -s http://127.0.0.1:8000/health

Run against the CPU stand-in at v0.35.0 on 2026-10-26.

Figure · When the engine dies

A dead engine does not serve again: the server says so, exits, and your supervisor starts a new one.

1. Healthy/health answers 200; requests are served
2. A request failsthe server probes the engine, for 5 seconds or longer if the engine has been slow
3. The engine is deadevery request, and /health, answers 503 with the reason
4. Exit code 702 seconds later, at most 8 more for requests in flight: about 7 seconds after the failing request
5. Your supervisorstarts a new server, which loads the model again

An error caused by the request itself, such as a 422, does not probe the engine.

Source: docs/running.md "When the engine dies" at the pinned release.

1. Run it under a supervisor

When its engine dies, the server exits with code 70 and leaves the restart to whatever runs it.

Supervisor	Setting	The full file
Docker Compose	restart: unless-stopped	With Docker
systemd	Restart=on-failure, TimeoutStartSec=infinity	On a cloud GPU
Kubernetes	restartPolicy: Always, a liveness probe on /health	On a cloud GPU

• Give the first start time. It downloads the weights: allow 45 minutes, as the image’s own health check does.
• Keep the weights on a volume, so a restart does not download them again.
• Requests sent while the new server loads are refused, so retry them in your client.

2. Watch it

Watch	Where	What a bad value means
The status of /health	GET /health	anything but 200: alert; 503 means the engine has died
The exit code	your supervisor	70: the engine died and the server ended itself
Server time	the x-decisio-server-ms header	the server’s time for the request, without the network
Queueing	wait in the x-decisio-stages header	a growing wait is requests queueing behind the one engine
What answered	the x-decisio-tasks and x-decisio-adapter headers	which taught task answered, and the adapter if the server serves one (--adapter)
Certified tasks	GET /metrics	see Watch for drift

Ask one question and print what it cost:

curl -s -D - -o /dev/null http://127.0.0.1:8000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state": "Both charges are 49.00 EUR, same day. Please refund one.", "questions": {"urgent": {"type": "noul"}}}' \
  | grep -i '^x-decisio'

Run against the CPU stand-in at v0.35.0 on 2026-10-26. It printed, trimmed to the fields this page uses:

x-decisio-server-ms: 754.3
x-decisio-route: text
x-decisio-stages: prepare=0.7;warm=0.0;engine=753.8;wait=0.0

• The server runs one engine; concurrent requests queue behind it and are answered in turn, first come, first served, with no queue limit. Batch concurrent requests scores the waiting ones together instead.
• There are no engine metrics and no structured log: take latency and queueing from the headers, in your client or your proxy’s log.
• After a deploy, read /health for the model, its checkpoint and revision, and max_model_len with its source.

3. Know what happens when the engine dies

The figure above is the sequence: the server checks the engine, answers 503, finishes what is in flight, exits with code 70 about 7 seconds after the failing request, and your supervisor starts it again. Once the engine is dead, every request, and /health, answers at once with the reason: {"ok": false, "engine": "dead", "reason": "...", "exit_code": 70}

• A death between requests turns /health to 503 within a second, before any request arrives.
• An error caused by the request itself, such as a 422, is answered and does not touch the engine.

4. Keep tasks across restarts and replicas

• One server: start it with --tasks-dir on a volume that outlives the container, and --require-tasks N, as Teach a task shows. One directory belongs to one server.
• Replicas behind a load balancer: start every one with --task-store on object storage, such as s3://bucket/prefix. A task taught on one replica reaches the others within --task-store-poll-seconds, 5 by default, as the same fitted record, so every replica serves the same numbers.
• Two teachers at once: a second registration of the same task answers 409 instead of overwriting it; read the task again and send it again.
• Keep fitting on one replica: start the others with --task-store-readonly and route the task-writing routes to that one.
• If the store goes away: replicas keep serving what they have, and teaching answers 503 until it is back; /health says so under systemone.task_store.

5. Secure it

1. Keep it on a private network, behind your proxy. The server listens on 127.0.0.1 by default; do not expose it to the internet.
2. Put authentication, rate limits and request-size limits in the proxy. The server has none.
3. Restrict the routes that change it to trusted callers: POST /v1/tasks, POST /v1/tasks/import, DELETE /v1/tasks/{id} and the abstention routes.
4. Do not configure CORS on the proxy for this server: it would let a web page’s browser send requests to it.
5. Run one server per trust boundary, one for each group of callers that trust each other: its cache and its tasks are shared by every caller.
6. Leave --debug-readout off. It exposes the model’s raw readouts.
7. Keep to the latest release: it is the one that receives security fixes.

Report a vulnerability through the repository’s private vulnerability reporting, not a public issue: an acknowledgement comes within 5 working days, an assessment within 15.

What next

• Scale: many questions, long texts and the settings that move server time.
• Upgrade: move to a new release without losing your certificates.
• Troubleshoot: what you see, why, and what to do.