---
title: "Harness² (Harness Squared)"
description: "Google Research's recursive agent-harnessing framework: entry points, benchmark setup, and runtime prerequisites."
---

# Harness² (Harness Squared)

Harness² is Google Research's framework for *recursive agent harnessing for an open world* (`README.md` upstream).
A **harness** is the system around a model that lets it act as an agent -- prompts, tools, memory, skills, orchestration -- and **harnessing** is the process of optimizing that system (`README.md`).
Harness² optimizes the harness around an executor with the model weights held fixed; it is not a model or an inference engine.
It ships as the `harness2` Python package, installed here through [`pkgs/harness2/default.nix`](../pkgs/harness2/default.nix).

## Where the harness sits

Harness² drives an **executor** and edits the harness around it (`README.md`, `harness2/substrate/`).
The default executor is OpenCode; `--substrate cx` selects Codex instead, reached through a Responses API endpoint supplied by the optional `harness2-bridge` (`docs/SETUP.md`).
Recursion happens at two levels (`README.md`):

- **Task-level recursion** runs a contrastive Propose-Probe-Compose step on the current task: propose harness edits, probe them on the same task, and compose a refined harness from the behavioral contrasts.
- **Harnessing-level recursion** keeps an *improvement harness* that accumulates adopted harnesses and evidence-backed lessons from each task.

`--mode parallel` explores `k` candidates and composes `k` refinements; `--mode sequential` carries the refined harness through `k` successive steps (`docs/EXPERIMENTS.md`).

## Entry points

The Nix package installs all seven console scripts declared in `pyproject.toml`:

| Command | Purpose |
|---|---|
| `harness2` | Run one refinement sweep (`harness2/cli/run_sweep.py`) |
| `harness2-setup` | Install a benchmark's pinned tools, patches, and data (`harness2/cli/setup.py`) |
| `harness2-check` | Check installed dependencies, credentials, and configuration without calling a model (`harness2/cli/check.py`) |
| `harness2-judge` | Grade a run's deliverables (`harness2/cli/judge.py`) |
| `harness2-report` | Report a graded run (`harness2/cli/report.py`) |
| `harness2-demo` | Run the real recursion loop with local deterministic editors, no model credentials (`harness2/cli/demo.py`) |
| `harness2-bridge` | Local LiteLLM Responses bridge for the Codex executor (`harness2/cli/bridge.py`) |

## Running it

The credential-free demo is the quickest check that the installation works:

```sh
harness2-demo --output runs/demo --k 2
```

`harness2-demo` runs both recursion modes against a local accounting task with real Python executions and writes `runs/demo/demo.json` (`harness2/cli/demo.py`).
A benchmark run follows the upstream sequence, for example JobBench (`docs/BENCHMARKS.md`):

```sh
harness2-setup jb
harness2-check --bench jb
harness2 --bench jb --k 1 --mode parallel
harness2-judge --bench jb --k 1 --mode parallel
harness2-report --bench jb --k 1 --mode parallel
```

Keep the benchmark, executor, subset, `--k`, `--mode`, and seed identical across the run, judge, and report commands (`docs/EXPERIMENTS.md`).
`--dry-run` on `harness2` and `harness2-setup` previews task selection and setup commands without model calls or downloads.

### Working directory

`harness2/config.py` resolves the working directory's layout, not a fixed install prefix:

- `.env` is loaded from the current directory (`load_dotenv`).
- `dependencies/` and `runs/` are created in the current directory; `HARNESS2_RUNS_DIR` overrides the output root.
- The benchmark patches and runner overlays come from `third_party/`, which this package bundles inside the installed `harness2` package, so `harness2-setup` finds them from any directory.

Run the benchmark commands from a dedicated directory to keep `.env`, `dependencies/`, and `runs/` together.

## Runtime prerequisites

The demo and the CLI run on macOS or Linux with no credentials.
A model-backed benchmark run needs more (`docs/SETUP.md`, `docs/BENCHMARKS.md`):

| Requirement | Needed for |
|---|---|
| Linux and Python 3.12 | The documented benchmark environment; this package uses Python 3.12 |
| Node.js 20+ and `npm` | `harness2-setup` installs a pinned Bun runtime and the OpenCode CLI |
| `uv` | `harness2-setup` creates each benchmark's own `.venv` |
| `git` and `curl` | Pinned upstream checkouts and data downloads |
| Google Cloud CLI and Application Default Credentials | Vertex AI executor, improver, and judge models (`GOOGLE_CLOUD_PROJECT`, `gcloud auth application-default login`) |
| Docker Engine | WorkBuddy-Bench only (`docker info` must succeed) |
| Pandoc and Poppler | LAB only |
| Network to `github.com`, `registry.npmjs.org`, Hugging Face, a Docker registry | `harness2-setup` downloads |

`harness2-check` validates the installed dependencies, credentials, and configuration without calling a model, but it does not test model availability (`docs/SETUP.md`).
Model-backed runs incur API costs; start with `--only-domains` and `--max-tasks` (`docs/EXPERIMENTS.md`).

`harness2-bridge` needs the upstream `bridge` extra (`litellm`, `fastapi`, `google-auth`), which this package does not include; install it separately if the Codex executor is required.

## Benchmarks

| Benchmark | Setup | Notes |
|---|---|---|
| JobBench | `harness2-setup jb` | Public, ungated Hugging Face data; no Docker |
| LAB | `harness2-setup lab` | Legal tasks; needs Pandoc and Poppler |
| WorkBuddy-Bench | `harness2-setup wb --subset office` | Needs Docker and a `google-vertex/gemini...` solver plus `gemini...` judge |

The full LAB task stream has 1,143 tasks, so select a subset for the first run (`docs/EXPERIMENTS.md`).

## Web access during evaluation

Executor network access is unrestricted upstream, and JobBench publishes each task's `RUBRICS.json` on Hugging Face, so an executor can retrieve its own grading criteria (`docs/EXPERIMENTS.md`, `docs/BENCHMARKS.md`).
Block `huggingface.co`, `hf.co`, and `*.hf.co` (and `github.com` / `raw.githubusercontent.com` for benchmark repositories) at the network level during a run when the comparison must be fair.
Codex runs with its sandbox disabled, so a network-level restriction is required rather than a tool setting.

## Licence

Harness² itself is Apache-2.0 (`LICENSE`).
Code under `third_party/` and the benchmark datasets keep their upstream licences (`third_party/README.md`, `dependencies/README.md`).
WorkBuddy-Bench is under Tencent's licence, which carries a territorial limitation; read it before downloading or using that benchmark (`docs/BENCHMARKS.md`).