--- title: "Models & providers" description: "Providers and models declared in pi-models.json, the local engines behind them, and cost tracking." --- # Models & providers Providers and models are declared in `pi-models.json`, which is symlinked to `~/.pi/agent/models.json` by `modules/ai.nix`. Inside `pi`, run `/models` to see the available models and `/model` to switch. ## Default provider The default provider and model are set in `modules/ai.nix` `settings.json`. As of this writing the default is **Melious** with **`deepseek-v4.1-flash`**. Melious and Novita expose a broad catalog (GLM, DeepSeek, Qwen, MiniMax, Kimi, Mistral, GPT-OSS), most configured with zero cost pending provider pricing. ## Scaleway Scaleway models are served from `https://api.scaleway.ai/v1` and configured under the `scaleway` provider. Current models: `mistral-medium-3.5-128b`, `gpt-oss-120b`, and `pixtral-12b-2409` (multimodal). Credentials live in `secrets/scaleway.yaml` (`scaleway_access_key`, `scaleway_secret_key`), exported as `SCW_ACCESS_KEY` / `SCW_SECRET_KEY` by `modules/shell.nix`. See [SCALEWAY_USAGE.mdx](model-providers/SCALEWAY_USAGE.mdx) for detailed setup and usage instructions. ## Keera Keera serves Apertus v1.5 from `https://gateway.keera.ch/api/v1` and is configured under the `keera` provider. Current models: `apertus-v1-5-70b`, `apertus-v1-5-8b`, and the `-thinking` variant of each, all advertising a 262,144-token context window. The `-thinking` variants reason before answering but do not accept a tool-calling request: with Pi's default tool set the gateway refuses with `400 "auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set`, so run them with tools turned off (`pi --no-tools`, which disables built-in and extension tools together). Verified on 2026-10-18: `apertus-v1-5-70b` completed a real bash tool call, `apertus-v1-5-8b` answered a plain prompt, and `apertus-v1-5-8b-thinking` answered a plain prompt with tools disabled. Credentials live in `secrets/keera.yaml` (`keera_api_key`), exported as `KEERA_API_KEY` by `modules/shell.nix`. The gateway advertises neither per-token pricing nor an output limit, so cost is configured as zero and `maxTokens` is set to 32,768 as a conservative placeholder. List the served models at any time with: ```sh curl -sS https://gateway.keera.ch/api/v1/models -H "Authorization: Bearer $KEERA_API_KEY" | jq -r '.data[].id' ``` ## Wintermolt [`wintermolt`](../pkgs/wintermolt/default.nix) reaches every OpenAI-compatible provider through its generic `forai` backend. The build tracks upstream main at c505541, one commit past the merge of [lupin4/wintermolt#4](https://github.com/lupin4/wintermolt/pull/4) (a101daa), which made `WINTERMOLT_BACKEND` a startup choice that honours a named `WINTERMOLT_MODEL`. The merge also turns a provider error frame inside a `200` stream into a failed request. Hosted gateways report quota and model errors that way, and wintermolt otherwise printed an empty answer and exited `0`. One hunk is still carried in [`patches/wintermolt/`](../patches/wintermolt/): - A configured `WINTERMOLT_FORAI_URL` makes the `forai` backend an OpenAI-compatible HTTP client, with its bearer token from `WINTERMOLT_FORAI_API_KEY` defaulting to empty so localhost stays keyless. ### `wm`: run any catalog model `wm` takes one `/` argument, resolves the endpoint and the bearer token from `pi-models.json`, and passes every other argument to wintermolt: ```sh # one-shot prompt on a chosen provider/model wm melious/deepseek-v4.1-flash -e "summarize README.md" # a bare model id means melious wm deepseek-v4.1-flash -e "summarize README.md" # ids that contain a slash stay intact when the first segment is not a provider wm openrouter/z-ai/glm-5.2 -e "say hi" wm local-BaseRT/basecompute/Qwen3-8B -e "say hi" # interactive REPL on GLM 5.3 wm melious/glm-5.3 # every / in the catalog wm --list ``` The argument splits on the FIRST `/` only, and only when that leading segment names a provider in the catalog. Otherwise the whole argument is a Melious model id, so ids such as `z-ai/glm-5.2` are not mistaken for a provider. `wm-melious ` is kept as a compatibility alias for `wm melious/`, and `wm-melious --list` still prints bare Melious ids. The endpoint comes from the provider's `baseUrl` in `pi-models.json` with a trailing `/v1` stripped, because the client appends `/v1/chat/completions` itself. Keera's `.../api/v1` base still works. The token comes from the provider's own shell variable, as the catalog's `apiKey` declares it: | Provider | Token | | --- | --- | | `melious` | `MELIOUS_API_KEY` (falls back to `WINTERMOLT_FORAI_API_KEY`) | | `keera` | `KEERA_API_KEY` | | `openrouter` | `OPENROUTER_API_KEY` | | `novita` | `NOVITA_API_KEY` | | `scaleway` | `SCW_SECRET_KEY` | | `ollama`, `local-oMLX`, `local-BaseRT`, `local-Camelid`, `google-antigravity` | keyless | `modules/shell.nix` exports those variables from the matching sops secrets, so no key enters the Nix store or the repo. A keyless provider explicitly clears the bearer, so the Melious token exported for the home environment is never sent to a local engine. `modules/shell.nix` also keeps `WINTERMOLT_FORAI_URL` / `WINTERMOLT_FORAI_API_KEY` as the default for a bare `wintermolt` run. `wm` always overrides both per invocation. Those exports live in the interactive zsh init, so a non-interactive shell has none of them. `ssh host 'bash -lc "wm melious/..."'` therefore refuses with `wm: melious needs $MELIOUS_API_KEY, which is not set` on both platforms, while the same command from an interactive zsh runs. That is a property of the shell setup, not of `wm`; run it from an interactive shell. Exercised for real on 2026-10-21 (see [Verification matrix](#verification-matrix)): - **melious**: 6 of 23 ids answered. - **scaleway**: 3 of 3 ids answered. - **openrouter**: `z-ai/glm-5.2` answered, which also proves a model id containing a slash survives the split. - **keera**: the gateway answered `HTTP 401`. - **novita**: the gateway answered `HTTP 403`. - **local-Camelid**: `Llama-3.2-3B-Instruct-Q8_0.gguf` answered `PONG` (25 s, keyless), and `tinyllama-1.1b-chat-v1.0.Q8_0.gguf` was refused with `HTTP 422` because its chat template has no tools branch. - **local-oMLX**: `Ternary-Bonsai-8B-mlx-2bit` answered `hello`, from a cold engine that `wm` started and stopped itself (see [local engine activation](#local-engine-activation)). - **ollama**: keyless, connection refused while its engine was stopped. Wired through the catalog and the usage above but not exercised: the `local-BaseRT` engine, `ollama` with a model that is actually served, and `google-antigravity`. `google-antigravity` is wired keyless only and needs the rotator to inject credentials; that plumbing is not implemented here. ### Local engine activation The four local engines are off at login because a loaded model pins unified memory, and pi starts one on demand through `engine-gate` (see [engine-gate](#engine-gate-on-demand-activation)). `wm` runs without pi, so it repeats that activation with the same mechanics (`launchctl kickstart`, `launchctl kill SIGTERM`, and the same `/v1/models` readiness probe): | Provider | Engine | Port | `wm` activation | | --- | --- | --- | --- | | `local-oMLX` | `org.nix-community.home.omlx` | 8000 | start, wait, stop | | `local-Camelid` | `org.nix-community.home.camelid` | 8781 | start, wait, load the GGUF, stop | | `local-BaseRT` | `org.nix-community.home.basert` | 8453 | start, wait, stop | | `ollama` | `org.nix-community.home.ollama` | 11434 | start, wait, stop | The second column carries the launchd label; on Linux the same row maps to the systemd user unit of the same name (`camelid`), and the other three rows have no unit there. Non-local providers never touch an engine. An engine that was already running when `wm` was invoked is left running. Only an engine `wm` started itself is stopped again when the run ends, because `keepAlive = false` leaves it down and the memory free. Each step is bounded by an environment variable and reports the engine's own log on timeout instead of only a connection error: | Variable | Default | Bounds | | --- | --- | --- | | `WM_ENGINE_TIMEOUT` | 120 s | the wait for the engine to answer *and* to be able to generate | | `WM_LOAD_TIMEOUT` | 180 s | the Camelid `POST /api/models/load` | | `WM_STOP_TIMEOUT` | 10 s | the wait for a stopped engine to release its port | Readiness is not just "the port answers". Camelid reports `generation_ready: false` in `/health` for the 10-35 s its cold start spends auto-loading a GGUF and building its Metal kernels. `wm` polls `/health` when the engine publishes it and `/v1/models` otherwise. Camelid also tracks a single model, so `wm` loads the requested GGUF with `replace: true`, which swaps out whatever the engine auto-loaded. It skips that load when `/v1/models` already lists the model by `id` *or* `filename`. A model Camelid auto-loaded appears under its own display id (e.g. `Llama 3.2 3B Instruct`) while requests resolve by either. Two limits are worth knowing: - `launchctl` is macOS-only, so on Linux there is a systemd path for Camelid only (`systemctl --user start|stop camelid`) and no way to start the other three engines. `wm` refuses instead of trying, before it prints any starting line, on `spectral-cockpit` (Rocky Linux 8.10, x86_64, no GPU, 2026-10-21). `wm local-oMLX/...`, `local-BaseRT/...` and `ollama/...` exit 1 with `wm: has no systemd user unit in this configuration`, naming `modules/linux-services.nix` as the only source of a unit, then `wm: could not be started (http://127.0.0.1:)`. `wm local-Camelid/...` exits 1 with `wm: the 'camelid' systemd user unit is not installed on this host`, naming `services.camelid` in that module as the way to install it and `systemctl --user start camelid` as the manual alternative, then `wm: Camelid could not be started (http://127.0.0.1:8781)`. The `camelid` unit only exists in a home that includes `modules/linux-services.nix` (`philipp-linux`). The lighter `spectral-cockpit` home is `spectral-home` plus `common-home` and has none, which is why the unit is missing there. Starting Camelid **through** the unit is therefore still unexercised: no Linux host with that home was available, and only the refusal paths above were run. - On a cold Camelid start `wm` also pays for the model the engine auto-loads before it can load the requested one, so a cold run costs 40-70 s even though the answer itself takes seconds. An engine that is already running skips that. Three behaviours matter when a run fails: - A `local-*` model no longer fails with a bare connection error: `wm` starts the engine, and if the engine does not answer in time it prints the tail of `/tmp/.err.log`. An engine that is running but still warming up does not fail fast either. A request inside Camelid's `generation_ready: false` window ends as `[deepseek curl error] Timeout was reached` followed by `[fallback]`. That is why `wm` waits for `generation_ready` before it starts wintermolt. - wintermolt always sends its tool definitions, so a model whose chat template has no tools branch is refused by the engine. Hosted gateways answer `The model '' does not support function calling`; Camelid answers `422 unsupported_chat_template` (`the loaded chat template has no certified tools branch; failing closed`). - A failed request falls back to Ollama and then OpenAI, so `[fallback] ...` on stderr means the provider you asked for did not answer. Any id a gateway serves works, not only the ones in `pi-models.json`: ```sh # current Melious gateway catalog curl -sS https://api.melious.ai/v1/models -H "Authorization: Bearer $MELIOUS_API_KEY" | jq -r '.data[].id' ``` ### Verification matrix `scripts/wintermolt-matrix.sh` sends one non-interactive prompt to every model of one provider, all at once, and prints pass/fail, latency and the provider's own error text per model. It counts `[fallback]` as a failure, so an Ollama answer cannot pass for the provider under test. The provider argument defaults to `melious`; any other catalog provider can be spot-checked the same way. A local provider works too, but `wm` starts and stops the engine per model, so a Camelid sweep pays one cold start (40-70 s) per entry. ```sh # re-verify the whole Melious catalog scripts/wintermolt-matrix.sh # spot-check another provider scripts/wintermolt-matrix.sh keera ``` Melious, 2026-10-21: 6 of 23 ids completed a round trip (`glm-5.3-flash`, `glm-5.3`, `glm-5.1`, `deepseek-v4-flash-0731`, `deepseek-v4.1-flash`, `deepseek-v4.1-flash-1017`). Twelve ids were refused with `Insufficient credits for request` because the Melious balance was negative (`Available: -0.001576 EUR`); they are untested against a funded account rather than broken. Five ids are not served by the gateway at all, so `pi-models.json` carries stale entries for them. `minimax-m2.7`, `minimax-m2.5`, `mistral-medium-3.5-128b`, `qwen3-next-80b-a3b-thinking` and `holo2-30b-a3b` all answer `Model not found`. Scaleway, same day: 3 of 3 ids passed (`mistral-medium-3.5-128b`, `gpt-oss-120b`, `pixtral-12b-2409`). Camelid, same day, one call per model because the engine serves one model at a time: `Llama-3.2-3B-Instruct-Q8_0.gguf` answered `PONG` in 25 s. `tinyllama-1.1b-chat-v1.0.Q8_0.gguf` returned `HTTP 422` (`no certified tools branch`). Both local engines were then re-run **cold** (agent confirmed `state = not running`, port closed, no listener) through the built `wm` wrapper, which is what exercises the activation path: | Command | Total | Engine | Result | | --- | --- | --- | --- | | `wm local-oMLX/Ternary-Bonsai-8B-mlx-2bit -e "Reply with exactly the word: hello"` | 18 s | `wm: oMLX ready on :8000 after 3s` | model answered `hello` | | `wm local-Camelid/Llama-3.2-3B-Instruct-Q8_0.gguf -e "Reply with exactly the word: PONG"` | 39 s | `wm: Camelid ready on :8781 after 24s` | model answered `"PONG"`, no load needed (Camelid had auto-loaded that GGUF) | | `wm local-Camelid/tinyllama-1.1b-chat-v1.0.Q8_0.gguf -e "Reply with exactly the word: hello"` | 66 s | `wm: Camelid ready on :8781 after 34s`, then `wm: Camelid loaded tinyllama-...gguf` | engine started and the GGUF loaded, but wintermolt's request was refused: `HTTP 422` | Each run ended with `wm: stopped after 0s` and the agent back at `state = not running`. On the cold oMLX run the engine log confirms the model was loaded and later freed (`Unloaded model: Ternary-Bonsai-8B-mlx-2bit, freed=2.15GB`). The TinyLlama refusal is the model, not the wiring. Against the same running engine and the same loaded GGUF, a plain `/v1/chat/completions` request **without** `tools` answers (`Reply: Hello!`). Adding one `tools` entry returns `422 unsupported_chat_template`: `the loaded chat template has no certified tools branch; failing closed`. wintermolt always sends its tool definitions, so no request of its reaches a model whose template has none. The smallest oMLX model (`Ternary-Bonsai-8B-mlx-2bit`) does accept them and is the smallest local model that completes the loop. `keera/apertus-v1-5-8b` returns `HTTP 401` and `novita/zai-org/glm-5.2` returns `HTTP 403`, so their keys need checking before those providers can be called done. `ollama` fails with `Could not connect to server` because its engine was stopped; the keyless path itself is exercised by that run (an empty bearer, with `WINTERMOLT_FORAI_URL` correctly reduced to `http://localhost:11434`). A second host carried the same branch: `spectral-cockpit`, Rocky Linux 8.10, x86_64, no GPU, 2026-10-21. `wm --list` printed the same 46 ids, and `wm melious/deepseek-v4.1-flash -e "reply with exactly: PONG"` answered `PONG` with exit 0 (`[backend] Switched to forAI (deepseek-v4.1-flash at https://api.melious.ai)`). The hosted path is therefore exercised on Linux as well as macOS; the local-engine paths there end in the refusals recorded under [local engine activation](#local-engine-activation). ## Local models Local models run through three engines, each with an `engine-gate` extension that kickstarts the matching engine when a local model is selected and stops it when a cloud model is active. - **Ollama** (`ollama` provider, `http://localhost:11434/v1`): runs a `launchd` daemon and auto-pulls models on `home-manager switch`. Currently exposes `gemma4:e4b`, `qwen2.5-coder:7b`, and `qwen3.6:35b-a3b`. - **oMLX** (`local-oMLX`, `http://localhost:8000`): macOS MLX engine serving `apertus-v1.5-8b-mlx-4bit`, `Ornith-1.5-9B-MLX-4bit`, and `Ternary-Bonsai-8B-mlx-2bit`. Apertus v1.5 uses a custom `apertus1p5` architecture that baseRT cannot convert, so it runs on oMLX. The models are downloaded on `home-manager switch` via the `pullOmlxModel` activation. - **baseRT** (`local-BaseRT`, `http://localhost:8453`): serves `basecompute/Qwen3-8B`, a pre-converted catalog model (no local conversion needed). Resident footprint is roughly 7.6 GB and insensitive to context size, so a wide 32K context costs no extra RAM. - **Camelid** (`local-Camelid`, `http://127.0.0.1:8781`): Rust-native GGUF backend serving `Llama-3.2-3B-Instruct-Q8_0.gguf` and `tinyllama-1.1b-chat-v1.0.Q8_0.gguf` from `~/.camelid/models/`. Runs on macOS (Metal GPU lane, launchd agent) and on Linux (CUDA with dynamic driver loading when an NVIDIA GPU is present, otherwise CPU kernels; systemd user unit, started with `systemctl --user start camelid`). The GGUFs are downloaded on `home-manager switch` by the `pullCamelidModels` activation in `modules/ai.nix` (each file is fetched once if absent, then reused). Because the engine resolves the pi-models model id against that directory, the local filename must match the model id exactly. ### engine-gate on-demand activation All three `local-*` engines are off by default (`runAtLoad = false`, `keepAlive = false`) and are started/stopped on demand by the `engine-gate` extension. The `engine-gate` extension is launchctl-based and therefore macOS-only. On Linux the Camelid unit is started manually with `systemctl --user start camelid`, or by `wm local-Camelid/...`, which drives that unit itself. Both only work in a home that includes `modules/linux-services.nix`; elsewhere the unit is not installed and `wm` refuses with that reason. `wm` repeats the same activation for a plain CLI run; see [local engine activation](#local-engine-activation). A `local-` provider that the loaded engine-gate does not cover is reported as `UNCOVERED` (an error notification) rather than silently ignored — that guard is what makes a stale extension copy (e.g. a long-running session daemon from before a new engine was added) obvious, since otherwise the request fails with an opaque `ECONNREFUSED` connection error. ### Qwen3.6 35B A3B `Qwen/Qwen3.6-35B-A3B` is downloaded on `home-manager switch` (if not already cached) by the `pullHuggingFaceModels` activation in `modules/ai.nix`. It is exposed as an Ollama model (`qwen3.6:35b-a3b`) and as a Melious model (`qwen3.6-35b-a3b`). On Ollama the context window is 262,144 tokens with an 8,192-token per-request cap, and local usage is zero cost. Note: the initial download is large (roughly 70 GB) and can take significant time and bandwidth. ## Harness² Harness² (`harness2`, installed from [`pkgs/harness2`](../pkgs/harness2/default.nix)) is a separate, external tool. It does not read `pi-models.json`. It takes its executor, improver, and judge model IDs from a benchmark-local `.env` (Vertex AI). See [harness2.mdx](harness2.mdx) for the entry points and the credentials and tools a benchmark run needs. ## Cost tracking `pi-extensions/cost-tracker/` tracks Anthropic spend and `pi-extensions/cost-tracker-scaleway/` tracks Scaleway spend. Both log one JSON line per assistant turn and show live footer status in pi. See [extensions.mdx](extensions.mdx) for the `/cost` and `/cost-scaleway` command reference. Pricing is currently set to $0 in most model definitions until the respective providers publish definitive per-token rates.