Search Documentation

Search for pages and headings in the documentation

Model Compatibility Matrix

pi-models.json and the omlxModels / camelidGgufs pull lists in modules/ai.nix declare which local models are configured.

This file is the counterpart that records which of them are verified to load and generate on the engine actually serving them, plus the rules for admitting new ones.

Keep the two apart: models.mdx answers “what is wired up”, this file answers “may this row honestly be served yet”.

Rules

  • Exact-row rule. One row per (exact model artifact, engine, version). A row states what was actually run on that row: model id, engine + engine version, and the load/generation evidence with its date. Nothing inherits: a sibling size, quant, or base-arch row — even from the same upstream family — proves nothing about another row.
  • Admission rule. A model may only be added to omlxModels, camelidGgufs, pullOllamaModels, or pi-models.json when its row here shows a PASS. An engine refusing a model gets a REFUSED row instead, so the same dead combination is never retried. A REFUSED row can only be overturned by a new row with new evidence (e.g. an engine upgrade).
  • Claim boundary. Each row lists what it does not claim. A PASS row claims load + basic generation on the recorded engine version only. It never claims parity against a reference runtime, throughput, portability across machines, or fitness for a specific pi workload unless separately recorded.

Matrix

Model Engine Status Evidence Boundary
apertus-v1.5-8b-mlx-4bit oMLX 0.6.4 Supported long-standing lineup entry no parity/throughput claim
Ornith-1.5-9B-MLX-4bit oMLX 0.6.4 Supported long-standing lineup entry no parity/throughput claim
Ternary-Bonsai-8B-mlx-2bit oMLX 0.6.4 Supported greedy “ok” receipt, 2026-10-15 no parity/throughput claim
Llama-3.2-3B-Instruct-Q8_0.gguf Camelid 0.7.8 Supported long-standing lineup entry no parity/throughput claim
tinyllama-1.1b-chat-v1.0.Q8_0.gguf Camelid 0.7.8 Supported long-standing lineup entry no parity/throughput claim
basecompute/Qwen3-8B baseRT Supported long-standing lineup entry no parity/throughput claim
gemma4:e4b + 2 Ollama models Ollama Receipt pending auto-pull configured; no receipt yet unverified until a row lands
Ternary-Bonsai-2-4B-mlx-2bit oMLX 0.6.4 REFUSED see below do not retry on this version
Ternary-Bonsai-2-4B-Q2_0.gguf Camelid 0.7.8 REFUSED see below do not retry on this version

REFUSED details (2026-10-15)

  • oMLX 0.6.4 / mlx-vlm 0.6.3, prism-ml/Ternary-Bonsai-2-4B-mlx-2bit: Model type prism_hadamard_qwen35 not supported on VLM and LLM lanes. The Bonsai-2 pack ships its own loader (runtime/, quickstart.py); stock mlx-vlm/oMLX cannot host it. This closed PR #299 after a live load test.
  • Camelid 0.7.8, Ternary-Bonsai-2-4B-Q2_0.gguf (prism-ml GGUF): runnable-smoke rejects the combo qwen35/Q2_0G64/Bpe — the arch and the custom ternary quant are not anchored (oracle-qualified combos only).

Recording a row

When a candidate model comes up, run the load/generation check against the exact engine version first, record the result here, and only then wire or drop the config. PASS rows carry the date and what was generated; REFUSED rows carry the exact error.