Skip to content
Decisio

DocsHow-to guides

3 min read · updated

Think when unsure

Have the server read an unsure question again after the model writes its reasoning, and spend the extra time only on those questions.

The questions decisio is unsure of are the ones a single read most often gets wrong. With thinking on, each of them is read again after the served model writes its reasoning, to get more of them right. The extra time goes to those questions only: a sure question is still answered from one read, in milliseconds.

Figure · Thinking when unsure

Only an unsure question is read again, after the model writes its reasoning.

A question with more than 16 options is never unsure: it stays at one read.

Source: docs/running.md ("Thinking when unsure") and docs/cli.md of the pinned release.

It follows the procedure of Chon (Apache-2.0). No measurement of its accuracy or its time is published yet.

1. Turn it on

Section titled “1. Turn it on”

Start the server with --think:

pip install "decisio[serve]==0.35.0"
python -m decisio.serve.vllm_engine --think

Shown, not run: it needs Linux with one NVIDIA card, which this check does not have.

  • Thinking is off by default, on every base: a request is one read, as fast and as repeatable as before.
  • --no-think turns it off, including when a tuned model’s repository turns it on in its defaults.think.
  • A request decides for itself with "think": true or "think": false; without the field, the server’s setting applies.

2. Ask with it

Section titled “2. Ask with it”
curl -s http://127.0.0.1:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Our invoice shows two seats, but we removed the second user last month.",
  "think": true,
  "questions": {"queue": {"type": "choice", "instructions": "Which team should handle this ticket?",
    "criteria": {"billing": "Invoices and charges", "access": "Logins and seats", "bug": "The product misbehaves"}}}
}' | python3 -c 'import json, sys; r = json.load(sys.stdin); print(json.dumps({"thinking": r.get("thinking"), "thinking_tokens": r["usage"].get("thinking_tokens")}, indent=1))'

Shown, not run: it needs a server that thinks; the CPU stand-in cannot write reasoning, only the vLLM engine on a card can.

  • A question thinks when its top probability is under its type’s threshold: 0.75 for a choice, 0.92 for yes/no, 0.70 for a score.
  • --think-factor scales the three thresholds, 1 by default; a lower factor thinks on fewer questions.
  • --think-budget caps the reasoning at 512 tokens per question by default, from 16 to 4,096. It is set on the server only.
  • --think-types choice,score limits thinking to the named question types; without it, every type may think.
  • All the unsure questions of a request go to the model in one call.

3. Read what it did

Section titled “3. Read what it did”
  • thinking in the response says how many questions thought, the reasoning tokens and the time, and for each question whether it thought, or why not.
  • usage.thinking_tokens counts the reasoning tokens, and usage.output_tokens includes them.
  • The headers x-decisio-think-ms and x-decisio-think-tokens carry the time and the tokens.
  • A question whose reasoning comes back unusable is answered from its one read, marked thinking: unavailable with the reason; the request does not fail.

The server’s setting, and how many questions have thought, are in /health:

curl -s http://127.0.0.1:8000/health \
  | python3 -c 'import json, sys; print(json.dumps(json.load(sys.stdin)["systemone"]["think"], indent=1))'

Run against the CPU stand-in at v0.35.0 on 2026-10-26. It printed:

{
 "default": false,
 "factor": 1.0,
 "budget": 512,
 "requests": 0,
 "questions": 0,
 "unavailable": 0,
 "tokens": 0,
 "ms": 0.0
}

4. Give thought answers their own temperature

Section titled “4. Give thought answers their own temperature”

A thought answer mixes two reads and comes out sharper than a single read, so a temperature fitted on single reads can leave it over-confident. On a tuned model, score answers that thought needed about twice the single-pass temperature.

  • --think-temperature-choice, --think-temperature-noul and --think-temperature-score set a temperature per type for the answers that thought, and for those only. A tuned model’s repository or an adapter’s file can carry them; a flag wins.
  • Which questions think does not change: the unsure test still uses the single-pass temperature.
  • A type with no think temperature keeps the single-pass one, so a server without them answers as before.
  • /health shows them under systemone.think_temperatures, and a certificate made under other ones is suspended (think_mode_changed).
  • python -m decisio.serve.adapter_temperatures fit-think fits them from labelled answers that thought.

5. Know what stays at one read

Section titled “5. Know what stays at one read”
  • A question with more than 16 options.
  • A question a registered task answers: its correction and certificate were fitted on one read.
  • Every question of a request with images, an abstention option or two orders.
  • A question whose type is not in --think-types.
  • Every question of a request whose text was cut to fit (--over-limit truncate): thinking on a text that long can take longer than a caller waits.

The response says why a question did not think. Where it runs, how it shares the server and what it does to certificates are in Requirements and limits.

What next

Section titled “What next”