# Server flags

> The flags a production server sets first, then every flag with its value for each model, from decisio's own reference.

Source: https://opendecisio.com/docs/flags/

With no flag, the server serves Gemma 4 31B with the settings it was measured with; most deployments set only `--base`. A flag you give overrides the model’s value. In the container, `DECISIO_BASE` and `DECISIO_PORT` do the same as `--base` and `--port`.

 

## Flags to set first

  

| Flag | Why |
| --- | --- |
| `--base` | picks the model, its checkpoint at a pinned revision, and its measured settings |
| `--model`, `--revision` | serve a specific checkpoint, at a revision you pin |
| `--host`, `--port` | the server listens on `127.0.0.1:8000`; put a proxy in front |
| `--max-model-len` | pins one text length; without it the Gemma models take what the card holds, so cards that differ serve different lengths |
| `--gpu-memory-utilization` | the model’s share of the card; it changes how long a text the card holds, a lot |
| `--tasks-file` | loads exported [tasks](https://opendecisio.com/docs/glossary/#task) at start, so a restart keeps them |
| `--served-name` | the name `GET /v1/models` lists, kept stable across a change of model |
| `--kv-cache-dtype` | `fp8` holds a longer text, but suspends a certificate made without it |
| `--engine-process` | matters only with a second engine (`--image-model`, `--head-engine`, `--one-engine`) |
| `--backend mlx` | serves on a Mac, from an MLX build |
| `--debug-readout` | leave it off in production |

 

The text length and the cache type are part of what a certificate covers: change either, and certify your tasks again.

 

## Every flag

 

```
uv run python -m decisio.serve.vllm_engine [flags]
```

 

A shape to fill in, not a command to run.

 

[Server flags](https://opendecisio.com/docs/flags/#every-flag) is the complete table of the flags, their defaults and values, generated from the server's argument parser at release time; this page says what each base takes.

 

`--base` chooses the base model and brings its checkpoint at a pinned revision and its value for every setting marked "the base's" below; a flag given explicitly overrides it. With a `--model` and no `--base`, the base is detected from the checkpoint's `config.json`. A checkpoint whose `model_type` no base serves, or whose `config.json` cannot be read, is refused with the supported bases named, because a base's classes, token ranges and temperatures are fitted for its own model; `--base <key>` with the same `--model` is the explicit way to serve it as that base's profile, if it is one.

 

The [CPU stand-in](https://opendecisio.com/docs/glossary/#cpu-stand-in) (`--backend hf`) is the exception: it takes any small model and uses the Qwen profile. With neither, the vLLM server serves `gemma-4-31b`, the default base (the CPU stand-in and the MLX backend always name a checkpoint, and refuse to start without one). `--base` also takes a decisio model repository, `owner/name[@revision]` or a directory holding its `decision_config.json`: the repository's weights are the checkpoint, its file names the base, and the server refuses to start if the file differs from the installed profile ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "Model repositories"). The served defaults need no other flag: every default below is what was measured, base by base ([Evaluation card](https://opendecisio.com/docs/evaluation-card/) sections 1, 6.1 and 7.1). `gemma-4-31b` is served from its FP8 repository, `tachara-ai/decisio-gemma-4-31b`, and `--model google/gemma-4-31B-it` serves Google's weights instead, quantized to FP8 when they load (vLLM's `fp8_per_tensor` online quantization, which `quantization="fp8"` redirects to), a setting of its profile; `--base gemma-4-31b-bf16` loads them at bf16, which did not leave room for a 32,768-token context on a 96 GB card ([Evaluation card](https://opendecisio.com/docs/evaluation-card/) section 7; [Choosing a model](https://opendecisio.com/docs/bases/#6-the-31b-in-bf16)). `gemma-4-31b-bf16` is the 31B in bf16, a second precision beside `gemma-4-31b`: it takes every value of the `gemma-4-31b` column below except `--served-name` (`decisio-gemma-4-31b-it-bf16-letters`) and the context ceiling (65,536), none of them measured at bf16 (its temperatures are the FP8 31B's, fitted on its reads), serves [`google/gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it) at its pinned revision with no quantization, and is chosen with `--base` only: a 31B checkpoint without `--base` is still the default 31B. `gemma-4-12b-fp8` is the 12B in FP8, a second precision beside `gemma-4-12b` (bf16, the default and the one every 12B record is on; [Choosing a model](https://opendecisio.com/docs/bases/#4-the-12b-in-fp8)): it takes every value of the `gemma-4-12b` column below except `--served-name` (`decisio-gemma-4-12b-it-fp8-letters`), none of them measured on FP8 yet (its temperatures are the bf16 12B's until they are refitted), quantizes [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it) to FP8 when it loads, as the 31B's profile does, and loads a stored conversion (a `config.json` that declares FP8) as stored; `--backend mlx` refuses it, and `--base gemma-4-12b` refuses an FP8 checkpoint. `--help` prints the same list with each flag's description.

 

### Every flag

 

Where the columns differ, the value is the base's.

 

#### Model and engine

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--base` |  |  | the default | the base, or a decisio model repository (`owner/name[@revision]`, or a directory); without it, detected from `--model`, or the default if no `--model` either |
| `--model` | [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8) | [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it) | `tachara-ai/decisio-gemma-4-31b` (its FP8 repository; [`google/gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it) serves Google's weights instead) | the checkpoint, a local directory or a Hugging Face repository id |
| `--revision` | `95a723d08a9490559dae23d0cff1d9466213d989` | `707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7` | `eaf1e4fa30a351ab87d07803ca60f1fa5606f882` (the repository's; `842da3794eaa0b77d5f08bae87a17459d91ff475` for [`google/gemma-4-31B-it`](https://huggingface.co/google/gemma-4-31B-it)) | the checkpoint's revision on the Hub; the pin applies whenever `--model` is the base's own repository |
| `--backend` | `vllm` | `vllm` | `vllm` | `vllm`; `mlx` for Apple silicon (the three bases, [Get started](https://opendecisio.com/docs/#1-pick-your-machine)); `hf` for the CPU stand-in, not for measurement: it serves a base's own checkpoint (`--base` and `--model` that belong together), not another model under a base's profile (Gemma 4 E4B under `--base gemma-4-31b` carries the 31B's soft cap, prompt and temperatures and is not supported; a NaN it produces is answered 500 with the model and engine named, [HTTP API](https://opendecisio.com/docs/api/)) |
| `--model-class` | `hidden-readout` | `hidden-readout` | `hidden-readout` | `hidden-readout`: decisio's text class that also returns the hidden state at the answer position, for the [intent head](https://opendecisio.com/docs/glossary/#intent-head); `text-only`: the same without it (the default with `--head-engine`); `view`: a directory built by `decisio.serve.make_text_only`, loaded as it is, needing no plugin |
| `--mode` | `separate` | `separate` | `separate` | `separate`: one prompt per [question](https://opendecisio.com/docs/glossary/#question), the state shared through the prefix cache; `packed`: questions packed into one pooling request (the compact layout only) |
| `--pack` | 16 | 16 | 16 | questions per pack in packed mode |
| `--engine` | `{"compilation_config": {"max_cudagraph_capture_size": 4096}}` | the same | the same | extra `LLM(...)` keyword arguments, as JSON |
| `--adapter` | none | none | none | `name=path` of a vLLM-format LoRA adapter; repeatable. A request uses it only when its body names it (`"adapter": "<name>"`). An adapter whose tensors are named for the other layout than the served class (`base_model.model.model.language_model.layers.` from a multimodal checkpoint, against `base_model.model.model.layers.` on a text-only base, or the reverse) is served from a mapped copy, made before the engine is built, with the tensor bytes unchanged; `/health` reports the mapping in `adapter_names`. An adapter in neither layout is refused at once, before the weights load |
| `--batch-invariant` | off | off | off | `on` or `off`: vLLM's batch-invariant kernels (`VLLM_BATCH_INVARIANT=1`, set before the engine starts); without it a tuned repository's `defaults.batch_invariant` decides. Refused when the environment says the opposite. In `/health` and the serving-stack fingerprint ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "Batch invariance") |
| `--emulate-precision-casts` | off | off | on (FP8) | `on`, `off` or `auto`: torch inductor's `emulate_precision_casts` (`TORCHINDUCTOR_EMULATE_PRECISION_CASTS=1`, set before the engine starts), which narrows the difference between two cold starts. `auto` is on for an FP8 base (the 12B FP8, the 31B served FP8, a tuned repository of either) and off for bf16. Refused when the environment says the opposite. In `/health` and the serving-stack fingerprint ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "Precision casts") |
| `--default-adapter` | none | none | none | the name of a loaded `--adapter` that answers a request naming none, at its own temperatures; `"adapter": "none"` asks for the base, another name gets that adapter. In `/health` and the serving-stack fingerprint ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "A default adapter") |
| `--adapter-temperatures` | none | none | none | `name=file` of a `decisio.adapter/1` file of that adapter's temperatures per question type (`choice`, `noul`, `score`) and, for choice, optionally per band of [option](https://opendecisio.com/docs/glossary/#option) count (`choice_bands`); repeatable. A request that names the adapter is served at them; a type the file does not list keeps the base's. It replaces the `decisio_adapter.json` beside the adapter's weights, which is read when the flag is not given. An unusable file, or a name `--adapter` did not load, is refused before the engine is built. `/health` reports them in `adapter_temperatures`, and the adapter's weights hash and temperatures are part of the serving-stack fingerprint ([running.md](https://opendecisio.com/docs/deploy/adapter/)) |
| `--gpu-memory-utilization` | 0.9 (0.47 with `--image-model`) | 0.9 | 0.9 | the text engine's share of the card. With `--image-model` the two engines share the card, and the default is the measured pair's 0.47; a pair that cannot fit (over 1.0 together, or over 0.96 with a default in it) is refused at start, before either engine is loaded |
| `--max-model-len` | 32768 | the largest that fits the card, 32768 to 65536 | the largest that fits the card, 32768 to 98304 | the longest prompt in tokens. Without it the vLLM server on a Gemma base serves the largest multiple of 8192 that fits the card's KV cache, up to the base's ceiling and not below 32768; given, that length, refused above the ceiling. The Qwen base, `--image-model`, a second engine, `--engine-process separate` and the other backends stay at 32768 ([Requirements and limits](https://opendecisio.com/docs/limits/#2-how-long-a-text-and-how-many-options)) |
| `--kv-cache-dtype` | `auto` | `auto` | `auto` | `auto` (the model's own) or `fp8`, which holds twice the tokens in the KV cache and changes the numerics: a certificate is made under one dtype and suspended under the other |
| `--allow-deep-gemm` | off | off | off | start even when `VLLM_USE_DEEP_GEMM` is set to something other than 0 |
| `--served-name` | `decisio-qwen3.6-35b-a3b-letters` | `decisio-gemma-4-12b-it-letters` | `decisio-gemma-4-31b-it-letters` | the name `GET /v1/models` lists |
| `--host`, `--port` | `127.0.0.1`, 8000 | the same | the same | the listen address; put a reverse proxy in front to expose it |

 

#### The prompt

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--prompt-tail` | `spaced` | `spaced` | `spaced` | `spaced`: a blank line before and after the lettered options, then one line asking for the chosen option's letter alone; `compact`: the compact layout, for tasks registered under it |
| `--answer-slot` | `prefill` | `template` | `template` | where the label is read: after "Answer:" prefilled in the assistant turn, or at the chat template's own first assistant position |
| `--layout` | `plain` | `header` | `header` | `plain`: the state then the question; `header`: the request's questions are listed before the state, the default of the Gemma bases. Held at `plain` while tasks fitted under plain apply, or an adapter or tuned repository recorded no layout ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "The [header layout](https://opendecisio.com/docs/glossary/#prompt-layout)"). A state cached for one set of questions is not reused for another under `header`; tasks and certificates are per layout |
| `--flavor` | none | none | none | a named configuration of the Gemma bases: `calibrated` (calibrated [probabilities](https://opendecisio.com/docs/glossary/#probability)), `commit` (commits yes/no answers outside the uncertain middle band) or `read-enhanced` (reads short requests twice). The server is named `Decisio-31b-<flavor>`, `Decisio-12b-<flavor>` or `Decisio-12b-fp8-<flavor>`, on the header layout; a flag you give wins and is recorded as an override ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "Flavors") |
| `--read-twice-under` | off | off | off | N counted input tokens (the header, the state and each question's text, as `usage.input_tokens` counts): a request read once of at most N has each question's user turn written twice. Never the default: it roughly doubles the input tokens of the requests it applies to, and `usage.input_tokens` counts both copies ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "Read twice"). Measured choice N = 2,000 |
| `--read-twice-max-questions` | none | none | none | with `--read-twice-under`: a request with more than R questions is read once; R comes from a timing run on your card |
| `--temperatures-file` | none | none | none | the base's temperatures per type and its choice bands from a `decisio.adapter/1` file (`adapter_temperatures fit`); a `--temperature-<type>` flag beats it, it beats a tuned repository's defaults |
| `--label-variants` | `single` | `summed` | `summed` | the tokens read per label: the one form the slot reads, or every single-token form summed |
| `--system-prompt` | off | on | on | a system turn before each question |
| `--noul-rendering` | `letters-keys` | `letters` | `letters` | how a [yes/no question](https://opendecisio.com/docs/glossary/#yesno-question) is asked: `letters-keys`, a two-option letter choice, the false side first, the sides named; `letters`, the same with each side shown as its description; `words`, read from the yes and no tokens |
| `--describe-options` | on | on | on | show an option that has a description as its description alone; `--no-describe-options` shows `key: description` |
| `--hide-index-keys` | on | on | on | never show enumerated keys (`option_0`, `option_1`, ...) |
| `--desnake-labels` | on | on | on | show bare snake_case labels as words |

 

#### Temperatures and outputs

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--temperature` | 1.506 | 3.592 | 5.252 | the temperature on the text route's plain readout, `softmax(log p / T)`; 1 switches it off; never changes the most probable option |
| `--temperature-choice` | 1.370 | the global one | 4.672 | the temperature for choice questions |
| `--temperature-noul` | the global one | the global one | the global one | the temperature for yes/no questions |
| `--temperature-score` | the global one | the global one | the global one | the temperature for score questions |
| `--noul-commit`, `--no-noul-commit` | off | off | off | report an uncommitted yes/no probability at the band's edge (below); a tuned repository's `defaults` state their own setting, and either flag overrides it |
| `--think`, `--no-think` | off | off | off | think when unsure ([running.md](https://opendecisio.com/docs/think/)): a question below its type's [threshold](https://opendecisio.com/docs/glossary/#threshold) times `--think-factor` is read again after the served model has written up to `--think-budget` tokens; a request's `think` field decides for that request, a tuned repository's `defaults.think` turns it on, and `--no-think` turns that off |
| `--think-factor`, `--think-budget` | 1, 512 | 1, 512 | 1, 512 | the factor on the thresholds (choice 0.75, yes/no 0.92, score 0.70) and the most reasoning tokens per question; a tuned repository's defaults state their own |
| `--over-limit` | refuse | refuse | refuse | a state over the engine's prompt ceiling: `refuse` answers 422, `truncate` cuts the state to its start and answers (never certified; [running.md](https://opendecisio.com/docs/limits/#2-how-long-a-text-and-how-many-options)); a request's `over_limit` field decides for that request, and a tuned repository's `defaults` state their own setting |
| `--over-limit-tokens` | the context less 1 | the context less 1 | the context less 1 | the prompt ceiling a truncating server cuts to; needed on the MLX build, which has no context length to read |
| `--orders` | 1 | 1 | 1 | 2: two-order averaging, each question also read in a second option order |
| `--branch-log` | none | none | none | a JSONL file for the two orders' disagreement per question |

 

#### Several questions and padding

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--multi-question` | `sequential` | `sequential` | `warm` with the engine in the server's process, else `sequential` | how a request's questions are scored (below) |
| `--register-boundary` | not used (no registration needed) | `before` | `before` | when a single question on a new state registers its boundary for the next question: `after`, once the response is out; `before`, ahead of the question; `off`, never ([Get started](https://opendecisio.com/docs/#1-pick-your-machine)) |
| `--concurrency` | `serial` | `serial` | `serial` | `serial`: one request at a time through the engine, an answer a function of its request alone; `batched`: the requests waiting for the engine scored together in one call, more throughput under concurrent load (measured on the FP8 12B, new 1,000-token states, 8 and 16 clients; measure on your own traffic), an answer can move slightly with its traffic ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "Concurrency") |
| `--batch-max-requests` | 16 | 16 | 16 | with `--concurrency batched`, the most requests in one engine call |
| `--engine-process` | `in` (`separate` with a second engine) | `in` (`separate` with a second engine) | `in` | where vLLM's engine runs: `in`, the server's process; `separate`, a process of its own (vLLM's arrangement) (below) |
| `--pad-policy` | `always` | `always` | `always` | `always`: pad every state; `shared`: pad only requests with more than one question; `row`: pad a single-question request's whole row (below); `none`: never pad (the default with `--backend mlx`) |
| `--pad-to` | `block` | `none` | `none` | what a state is padded to: the KV cache block, a token count, or nothing |
| `--pad-where` | `front` | `front` | `front` | where the padding goes: `front`, before the chat template; `user`, at the start of the user turn; `between`, between the state and the question |

 

#### Tasks and abstention

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--tasks` | on | on | on | apply registered per-task calibration and intent heads (`POST /v1/tasks`) |
| `--tasks-file` | none | none | none | a JSON list of tasks to load at start, as `GET /v1/tasks?full=1` returns them |
| `--head-engine` | off | off | off | read the intent head's hidden state from a second copy of the model in vLLM's pooling mode instead of from the serving engine: faster head questions, at a second weight copy; not with `--image-model` |
| `--head-gpu-memory-utilization` | 0.47 | 0.47 | 0.47 | the second engine's share of the card |
| `--abstention` | on | on | on | apply registered [abstention](https://opendecisio.com/docs/glossary/#abstain) thresholds (`POST /v1/abstention/tasks`) |
| `--abstention-tasks` | none | none | none | a JSON list of abstention tasks to load at start |
| `--abstain-option` | none | none | none | offer this extra option (for example "can't tell") on requests that use imajev's extension, reported as `unknown_probability` and `abstained`; untrained |

 

#### Images

  

| Flag | `qwen3.6-35b-a3b` | `gemma-4-12b` | `gemma-4-31b` | What it does |
| --- | --- | --- | --- | --- |
| `--image-model` | none | none | none | the full multimodal checkpoint, as a second engine serving only requests that carry images; measured with the Qwen base only |
| `--max-images` | 128 | 128 | 128 | with `--image-model`: the most images one request may carry (128 images of 512 by 512 pixels fill the context, so for photos the context ends a request; lower it to save multimodal profiling memory at start-up, raise it for smaller images) |
| `--image-gpu-memory-utilization` | 0.49 | 0.49 | 0.49 | the image engine's share of the card, beside the text engine's 0.47; a pair with a default in it may total at most 0.96, a pair given in full at most 1.0 |
| `--one-engine` | off | off | off | load only the image engine and serve text requests on it too, for a card that cannot hold both; changes the text route's class |

 

#### The Mac (`--backend mlx`)

  

| Flag | Default | What it does |
| --- | --- | --- |
| `--prefix-cache-mb` | the base's: 2048 (Qwen and `gemma-4-31b`), 7400 (`gemma-4-12b`) | the cross-request prefix cache's budget in MiB (2^20 bytes); 0 turns it off |
| `--mlx-memory-limit-gb` | what the base needs, at most two thirds of the Mac's memory | the MLX memory limit in GiB; 0 leaves MLX's own (about 80 percent of the memory); a base that does not fit is refused ([Memory on a Mac](https://opendecisio.com/docs/deploy/mac/#3-mind-the-memory)) |
| `--mlx-cache-limit-gb` | 2 | the MLX cache limit in GiB: freed buffers kept for reuse |
| `--tokenizer` | [`Qwen/Qwen3.6-35B-A3B-FP8`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8) | the tokenizer the prompts are built with, so they are the vLLM path's byte for byte |

 

#### Verification

  

| Flag | Default | What it does |
| --- | --- | --- |
| `--debug-readout` | off | honour the `x-decisio-debug` request header (the raw readout in the response); for reads and diagnostics, not for latency measurement or production (about 130 ms more per single-question request on a 12B) |

 

### What the behaviour flags do

 

#### Several questions in one request (`--multi-question`)

 

With `sequential`, the served default, a request with several questions first prefills the state once, then scores each question in its own engine call, reading the state from the prefix cache, so every answer equals the same question sent alone. With `warm`, for bulk scoring, the questions are scored in one batch after the same prefill: faster (four questions on a new state take about half the time of `sequential` on the Qwen base and about two thirds on the Gemma bases), but a batched answer differs from the same question sent alone, and when two options are close the choice can change ([Evaluation card](https://opendecisio.com/docs/evaluation-card/) section 4). With the engine in its own process (`--engine-process separate`) a warm answer can also move between repeats, because that process does not always put a request's questions in one engine step (vllm-project/vllm#59764); with the engine in the server's process (`--engine-process in`) an identical four-question warm request repeated exactly on a server that had answered it before (272 of 272 requests, 5 repeats each, vLLM 0.30.0); a single-question request repeated on such a server can differ slightly (5 of 200 requests, up to 0.031, on vLLM 0.30.0 and 0.31.0 alike; [Evaluation card](https://opendecisio.com/docs/evaluation-card/) sections 4 and 7.5). `--engine-process` resolves to `in` for a single-engine server and to `separate` when a second engine is configured (`--image-model`, `--head-engine`, `--one-engine`), with which `in` is refused; an explicit choice is honoured, and the server prints the arrangement and why at start-up and reports it in `/health` as `engine_process`.

 

On the Gemma 4 31B base `warm` is the default when the engine runs in the server's process: against `sequential` it changed no choice on the suite, JevBench and the travel requests, by at most 0.0073 in a probability ([`runs/2026-10-05_engine-death-gates/`](https://github.com/aminry/decisio/tree/v0.35.0/runs/2026-10-05_engine-death-gates)); the other bases keep `sequential`. With `batch`, every question goes in one engine call that prefills the state itself, with no warm-up. Measurements: [Evaluation card](https://opendecisio.com/docs/evaluation-card/) section 4, [`runs/2026-10-02_multi-question-and-rendering/`](https://github.com/aminry/decisio/tree/v0.35.0/runs/2026-10-02_multi-question-and-rendering), and the README's example request in [`runs/2026-10-01_docker-first-gpu-start/repeat_variability/`](https://github.com/aminry/decisio/tree/v0.35.0/runs/2026-10-01_docker-first-gpu-start/repeat_variability).

 

#### Row padding (`--pad-policy row`)

 

By default every state is padded to end on the KV cache block, so a single question on a state the server has not seen is prefilled in two engine steps, the state and then the question. With `row`, a single-question request is padded so that its whole row, state and question, ends on the block boundary, and vLLM prefills it in one engine step; requests with several questions are padded as by default. It is faster on a new state, and accuracy and calibration stayed within noise on the suite, JevBench, the four Decision Index benchmarks and the intent heads.

 

Its cost is repeatability: a repeated identical request can move slightly on long prompts, where the default padding returns the same probabilities every time, so it is off by default. The demos ran with it ([Demos](https://opendecisio.com/demos/)). Measurements: [`runs/2026-10-02_pad-policy-row/`](https://github.com/aminry/decisio/tree/v0.35.0/runs/2026-10-02_pad-policy-row); not measured on the Gemma base.

 

#### Determinism

 

**Qwen base.** On a running server a question returns the same probabilities every time, alone or among other questions: each question is scored in its own engine call. Across server restarts, single-question answers have matched an earlier record on the whole suite in most starts measured, and moved slightly without changing a choice in the others.

 

**Gemma 4 31B (the default) and Gemma 4 12B.** Neither repeats bit for bit: their logits come out in bf16 after the soft cap, and a state read for the first time and read again from the prefix cache can land a step apart; several questions in one request stay within the bound in the README's settings table. For the 31B, on one machine, two fresh starts gave the same 200 single-question answers on all 200 items, but asking the same 200 again on the server that had just answered them changed 5 (largest 0.031, no top answer changed); on other machines the same requests differed from an earlier machine's record on every item (median 4.3e-3, largest 0.379, 4 top answers changed).

 

So the 31B repeats exactly on a freshly started server on one machine, and nothing is promised across machines ([Evaluation card](https://opendecisio.com/docs/evaluation-card/) section 7.5). The 12B returns the same choice with probabilities that can move slightly ([Evaluation card](https://opendecisio.com/docs/evaluation-card/) section 6.5). Any change of rounding moves this model's answers, the batch, the padding or where a long row is split, so every bit-identity claim holds for a fixed configuration and request form.

 

Every claim here is for the default `--concurrency serial`; with `batched`, an answer also depends on the other requests scored in its engine call ([Get started](https://opendecisio.com/docs/#1-pick-your-machine), "Concurrency"). Measurements: [Evaluation card](https://opendecisio.com/docs/evaluation-card/) sections 4 and 6.5.

 

#### Committed yes/no answers (`--noul-commit`)

 

The rule commits a yes/no answer: a probability strictly between 0.20 and 0.80 is reported at the band's edge on its own side, 0.80 above 0.5 and 0.20 at or below it. It is useful when a downstream scorer treats middle probabilities as no answer.

 

The answer never changes; its probability does, and calibration pays for it: log loss and ECE rise. Measurements: [`runs/2026-10-03_noul-commit/`](https://github.com/aminry/decisio/tree/v0.35.0/runs/2026-10-03_noul-commit).

 

#### The compact layout (`--prompt-tail compact`)

 

The compact layout has no blank lines around the options and ends with "Answer with the letter only.". Tasks registered under one layout are not applied under the other, so a task registered under the compact layout needs `--prompt-tail compact` or registering again. Packed mode reads the compact layout only, and the image route keeps it.

 

## What next

 

- [Choosing a model](https://opendecisio.com/docs/bases/): what `--base` picks.
- [Scale](https://opendecisio.com/docs/scale/#5-set-the-flags-that-move-server-time): the flags that move server time, and what each costs.
- [Operate](https://opendecisio.com/docs/operating/): running a server in production.