LLM

LLM primitives

The LLM is the reasoning engine, swappable like a database driver. Four operations, gated by the llm capability (auto-granted in run, required under serve):

llm.syn

-- Doc example: LLM operations. The four ops return TEXT and need a real provider,
-- so the doctest verifies the offline/online check; the ops are shown in `summarize`.
intent: "doc example: LLM operations"
require llm

task summarize(report)
    when llm_available()
        give reason about report with context = "be concise"   -- needs a provider
    otherwise
        give "(LLM offline)"

print("summarize(\"a report\") → " + summarize("a report"))    -- offline → (LLM offline)

test "llm_available() is a bool — branch on it instead of guessing"
    assert_eq(type_of(llm_available()), "bool")

-- llm_stream(prompt, context, on_chunk): on_chunk receives each fragment as it is
-- produced; the call returns the full text. Offline it returns the placeholder
-- WITHOUT ever invoking on_chunk (placeholders are not answers).
task on_tok(t)
    print("chunk: " + t)

test "llm_stream: offline → placeholder, on_chunk untouched; online → text"
    when llm_available()
        assert_eq(type_of(llm_stream("Reply with one word: pong", "", on_tok)), "text")
    otherwise
        assert_eq(llm_stream("q", "", on_tok), "[no llm provider]")

-- Offline, each op returns a descriptive placeholder — text, never an answer. They are NOT
-- inputs: lineage() records only what a real model returned.
test "offline placeholders: text you can tell apart, and never recorded as an input"
    when not llm_available()
        let ticket be "my order arrived broken"
        assert_eq(reason about ticket, "[reasoning about: my order arrived broken]")
        assert_eq(decide between ["refund", "replace"] given ticket, "[decision pending]")
        assert_eq(analyze ticket for "tone", "[analysis of: tone]")
        assert_eq(generate "reply email" given ticket with tone = "empathetic", "[generated: reply email]")
        assert_eq(llm_step("q", [], ""), {"kind": "final", "text": "[no llm provider]", "tokens": 0})
        assert_eq(llm_usage(), 0)
        assert_eq(lineage(), [])

The four operations§

require llm

let analysis be analyze sales_data for "trends and anomalies"
let action   be decide between ["refund", "replace", "escalate"] given ticket
let email    be generate "response email" given complaint with tone = "empathetic"
let insight  be reason about problem with context = background_data

They take a subject: reason about X, decide between [...] given X, analyze X for "...", generate "..." given X. (A bare reason "literal" also works.)

Validated decisions§

A decide between [...] result must be exactly one of the options. Synsema enforces this in layers: on network providers the choice is forced at the API level (an internal tool whose schema restricts the answer to your options — zero retries in the common case); the answer is then normalized (case, punctuation, whole-word containment: "The answer is BLUE" → BLUE); if it still doesn't match, one retry with feedback; and only then it warns (one-time stderr notice) and returns the raw response — the chain never breaks, but never silently.

Offline mode§

Without a provider configured, the ops return descriptive placeholders (e.g. decide → "[decision pending]"), so programs stay runnable — the chain never breaks. It's never silent, though: the first LLM op that falls back prints a one-time stderr notice pointing you to synsema llm status for the exact diagnosis (which variable is missing, and where). The placeholders are [reasoning about: …], [decision pending] (which is not one of your options — code that branches on a decide must handle it), [analysis of: …] and [generated: …]; llm_step and llm_stream return [no llm provider]. Branch on llm_available() instead of guessing:

require llm

task summarize(report)
    let s be "(LLM offline)"
    when llm_available()
        set s to reason about report
    give s

Inside parallel_map and inside a spawned agent (engine v0.6.45+), reason, decide, analyze and generate use the same provider as the code that started them. Before, under synsema run each parallel_map item and each agent ran in a fresh interpreter without providers and got the offline placeholders, while the same code on the main thread got real answers. Under serve, agents were already wired. The process budget below holds inside them too.

Metering & budget — `llm_usage()`, `SYNSEMA_LLM_BUDGET`§

Every real provider call is metered. llm_usage() → number of LLM tokens (input + output) consumed by this process so far — introspection, no capability (like llm_available()); 0 offline or before any call. One counter per process, shared under serve too. It is monotonic: time windows are your program's policy, not runtime state.

The host can set a hard token budget with SYNSEMA_LLM_BUDGET (a positive integer, resolved process environ > .env; invalid or 0 → no budget, with a warning). At the ceiling, every LLM op degrades to the marker text [llm budget exceeded: used N of M tokens] — no error, no network call, one stderr notice per process. An LLM op never breaks the agent chain (same philosophy as offline placeholders), so code that must stop on exhaustion branches on the marker or on llm_usage():

require llm

task ask_once(question)
    let before be llm_usage()
    let answer be reason about question
    when contains(answer, "llm budget exceeded")
        give "(budget exhausted — not retrying)"
    print("this call cost " + text(llm_usage() - before) + " tokens")
    give answer

Per identity, under serve (engine v0.6.20+). SYNSEMA_LLM_BUDGET_PER_IDENTITY="alice=200000,agent-7=1500000" caps the tokens each authenticated identity may consume in this process — the identity your auth with task returns (id, sub or keyid; the same one spend and rate limits use), on every kind of route, stream and socket included. The counter is process-wide (one for all workers and stream threads) and lives beside the global one. When an identity crosses its line, its ops return [llm budget exceeded for identity alice: used N of M tokens] — the same marker contract, one stderr notice per identity — while other identities and anonymous requests continue. Requests without an identity (no auth, run, cron) are bound only by SYNSEMA_LLM_BUDGET. Think in real numbers: an ordinary task spends 20 000 tokens or more per call, so identity ceilings are hundreds of thousands. The four operations did not change — this is a host ceiling around the provider, like SYNSEMA_SPEND_CEILING; a platform that needs the count to survive a restart keeps its own ledger and reads llm_usage().

Model answers are inputs — lineage (v0.6.29+)§

The engine records the answer a real model returns to reason, analyze and generate as an input of the program: lineage() lists it with source llm, what = <op> prompt sha256-salted:<hex> (reason prompt …, analyze prompt …: a salted commitment to the prompt, never the prompt itself), the sha256 of the text the model returned and encoding: "text"; receipt() publishes it among its inputs. In v0.6.30 a decide answer is not recorded, and an offline placeholder is never an input. What that list is for, and how to reveal a prompt later, is in Observability § Lineage.

Private data to a model§

Engine v0.6.46+, with --labels. Until v0.6.45 any private value in reason, analyze, decide, generate, judge, llm_step or llm_stream was a label_violation. Now it can reach a model the host allows, and the answer stays private. Two sides, both required:

1. The program declares whose data it may send: require llm("customers"). A bare require llm still means public data only. require llm("P") also covers the calls with public data (before v0.6.46 it broke every LLM operation). 2. The host allows it per principal and per provider, in its ceiling:

Ceiling item	Allows data private to `customers` to go to
`llm=customers@api.anthropic.com`	the provider whose host is `api.anthropic.com`
`llm=customers@127.0.0.1`	an endpoint set with `SYNSEMA_LLM_BASE_URL=http://127.0.0.1:8080/v1` (the host, without the port)
`llm=customers@local`	a model in the process (`SYNSEMA_LLM_PROVIDER=local`); an unconfigured, offline provider counts as `local` too
`llm=customers@*`	any provider — written on purpose
`llm=customers`, `llm=*`, `llm`	no private data: an item without `@provider` never allows it
`llm=customers@embedder`	in `@synsema/wasm`, the embedder's `llm` hook

The host the check compares is the provider's: anthropic → api.anthropic.com, openai → api.openai.com, deepseek → api.deepseek.com, minimax → api.minimax.io, local → local; with SYNSEMA_LLM_BASE_URL set, the host of that URL. The program never names the provider — the host picks it, and the check follows the provider of each call. An item with a scope also allows the calls with public data, to whichever provider is configured.

synsema run --labels --cap-set "stdout,llm=customers@api.anthropic.com,declassify=customers" app.syn

require llm("customers")
let note be private("Ana Perez, order 3141", "customers")
let summary be generate "a one-line summary" given note
print(summary)                                           -- private(customers)
print(declassify(summary, "the summary goes back to the same customer"))

Refused, and nothing is sent, when the ceiling has no matching item, the program has no require llm("customers"), or there is no ceiling at all:

label_violation: reason would send data private to (customers) to the model; the host allows that per principal and provider in its ceiling (`--cap-set llm=<principal>@<provider host>`, `@local` for a model in the process, `@embedder` in wasm) and the program declares `require llm("<principal>")`. Otherwise declassify(<that value>, "<why the model may see it>") the part the model may see

Under a private branch the call is refused before anything else (reason called under private control flow …) and, since v0.6.46, writes no "granted llm" audit entry first.

Allowed, the prompt carries the data itself (the few cases that got through before v0.6.46 sent the text private(customers)), and:

The answer carries the label of what went in. reason, analyze, decide and generate give a private text. llm_step gives the whole map private, so choosing a tool by step["name"] needs a declassify, which has its own ceiling item — see Tool calling.
llm_stream: each chunk carries the label, and on_chunk runs under the label of the answer, like the body of an each over a private list (how many chunks there were is private too). A public effect in it — print, assigning a public variable — is a label_violation. Collect into a private variable instead:

require llm("customers")
let note be private("Ana", "customers")
let parts be private([], "customers")
task on_chunk(t)
    set parts to parts + [t]
let full be llm_stream("Greet the customer", note, on_chunk)

Audit: each call writes llm("customers@api.anthropic.com") with source: "llm-data" and reason = the operation; a refused one is written with granted: false and reason: "<op>: not in the host ceiling or not required". The principal in it is the program's declared set.
Lineage: the answer to a private prompt is hashed with the prompt's salt and its length is left out — what = generate prompt sha256-salted:<hex> response sha256-salted, bytes: 0.
The judge has its own item, judge=customers@…, and its own require judge("customers") — see Judge. A decide answered by the judge (SYNSEMA_JUDGE_DECIDE=1) is checked against judge=.

What this does not give. The provider sees the data: this allows a provider the host trusts; it does not protect the data from it. What the model answers is private but not trusted — an injected prompt can make it say anything — so a declassify, with its own ceiling, is what stands between the answer and a sink. And the size of the prompt is visible on the network, as in any https request.

Streaming (`llm_stream`)§

llm_stream(prompt, context, on_chunk) generates with the configured provider, invoking the task on_chunk with each text fragment as it is produced, and returns the full text. Same llm gate. Network providers stream the API's SSE text-deltas as real chunks; the embedded local provider streams token by token. (With SYNSEMA_LLM_HTTP_STREAM=0 a network provider falls back to one chunk — the whole response.)

require llm

task on_tok(t)
    print(t)

let full be llm_stream("Explain CSP in one line", "", on_tok)

Offline it returns "[no llm provider]" without invoking on_chunk. Under a serve SSE route it composes with send — define the emitting task inside the stream block (send is a statement, not a function, and only parses there):

serve on 8080
    route "GET /chat"
        stream
            task emit(tok)
                send tok
            let full be llm_stream("Count to ten in words", "", emit)
            send full as "full"

If on_chunk fails (e.g. the send of a disconnected client), generation stops early and the error propagates — recoverable with try/recover.

Configure a provider in Provider config; let the model pick tools in Tool calling.