Harness² (Harness Squared)

Harness² is Google Research’s framework for recursive agent harnessing for an open world (README.md upstream). A harness is the system around a model that lets it act as an agent – prompts, tools, memory, skills, orchestration – and harnessing is the process of optimizing that system (README.md). Harness² optimizes the harness around an executor with the model weights held fixed; it is not a model or an inference engine. It ships as the harness2 Python package, installed here through pkgs/harness2/default.nix.

Where the harness sits

Harness² drives an executor and edits the harness around it (README.md, harness2/substrate/). The default executor is OpenCode; --substrate cx selects Codex instead, reached through a Responses API endpoint supplied by the optional harness2-bridge (docs/SETUP.md). Recursion happens at two levels (README.md):

• Task-level recursion runs a contrastive Propose-Probe-Compose step on the current task: propose harness edits, probe them on the same task, and compose a refined harness from the behavioral contrasts.
• Harnessing-level recursion keeps an improvement harness that accumulates adopted harnesses and evidence-backed lessons from each task.

--mode parallel explores k candidates and composes k refinements; --mode sequential carries the refined harness through k successive steps (docs/EXPERIMENTS.md).

Entry points

The Nix package installs all seven console scripts declared in pyproject.toml:

Running it

The credential-free demo is the quickest check that the installation works:

harness2-demo --output runs/demo --k 2

harness2-demo runs both recursion modes against a local accounting task with real Python executions and writes runs/demo/demo.json (harness2/cli/demo.py). A benchmark run follows the upstream sequence, for example JobBench (docs/BENCHMARKS.md):

harness2-setup jb
harness2-check --bench jb
harness2 --bench jb --k 1 --mode parallel
harness2-judge --bench jb --k 1 --mode parallel
harness2-report --bench jb --k 1 --mode parallel

Keep the benchmark, executor, subset, --k, --mode, and seed identical across the run, judge, and report commands (docs/EXPERIMENTS.md). --dry-run on harness2 and harness2-setup previews task selection and setup commands without model calls or downloads.

Working directory

harness2/config.py resolves the working directory’s layout, not a fixed install prefix:

• .env is loaded from the current directory (load_dotenv).
• dependencies/ and runs/ are created in the current directory; HARNESS2_RUNS_DIR overrides the output root.
• The benchmark patches and runner overlays come from third_party/, which this package bundles inside the installed harness2 package, so harness2-setup finds them from any directory.

Run the benchmark commands from a dedicated directory to keep .env, dependencies/, and runs/ together.

Runtime prerequisites

The demo and the CLI run on macOS or Linux with no credentials. A model-backed benchmark run needs more (docs/SETUP.md, docs/BENCHMARKS.md):

harness2-check validates the installed dependencies, credentials, and configuration without calling a model, but it does not test model availability (docs/SETUP.md). Model-backed runs incur API costs; start with --only-domains and --max-tasks (docs/EXPERIMENTS.md).

harness2-bridge needs the upstream bridge extra (litellm, fastapi, google-auth), which this package does not include; install it separately if the Codex executor is required.

Benchmarks

The full LAB task stream has 1,143 tasks, so select a subset for the first run (docs/EXPERIMENTS.md).

Web access during evaluation

Executor network access is unrestricted upstream, and JobBench publishes each task’s RUBRICS.json on Hugging Face, so an executor can retrieve its own grading criteria (docs/EXPERIMENTS.md, docs/BENCHMARKS.md). Block huggingface.co, hf.co, and *.hf.co (and github.com / raw.githubusercontent.com for benchmark repositories) at the network level during a run when the comparison must be fair. Codex runs with its sandbox disabled, so a network-level restriction is required rather than a tool setting.

Licence

Harness² itself is Apache-2.0 (LICENSE). Code under third_party/ and the benchmark datasets keep their upstream licences (third_party/README.md, dependencies/README.md). WorkBuddy-Bench is under Tencent’s licence, which carries a territorial limitation; read it before downloading or using that benchmark (docs/BENCHMARKS.md).