This week we discussed Harness2: Recursive Agent Harnessing for an Open World, from Google Cloud AI Research and Northwestern University. The setting is the agent harness: the system prompt, guardrails, memory, skills, tool guidance, scripts, subagents and plugins that sit around a frozen model and make it act as an agent. Most of us use one generic harness for everything.
Harness2 argues for the opposite: build a harness for each task, at test time, without any score to tell you whether an edit worked, and keep the harnesses that worked, one lesson attached to each, so the next task in the same line of work can borrow from them. The authors report one improvement step adding up to 11.5 points on WorkBuddy-Bench, 4.5 on JobBench and 11.8 percentage points on Harvey's Legal Agent Benchmark.
Two words carry the whole paper, so it helps to pin them down. A task is one request with its source material, done in a single agent session capped at 200 model turns and an hour. On LAB, Harvey's legal benchmark, that is something like producing a redline of an agreement; on JobBench, a benchmark of professional work, reconciling the schedule, venue, labour and budget files and writing the memo. A domain is the group a task belongs to: a practice area such as arbitration or antitrust on LAB, an occupation on JobBench, a category such as developer or QA on WorkBuddy-Bench. Tasks in a domain run one after another, and what the system learns stays inside the domain. An arbitration task can draw on earlier arbitration tasks and nothing else.
Two loops, four runs per task
Two agents are involved, and in the main results they run on different models, which the paper's own figure doesn't make obvious. The executor, Gemini 3.5 Flash or GLM-5 inside OpenCode or Codex, is the agent that does the task. It never touches a harness. The improver, Claude Opus 4.8 in the main results, runs as a coding agent, in a fresh session for each stage of the two loops below. It reads the executor's runs and writes harness files, but never does the task itself. The harness being edited isn't OpenCode or Codex itself, whose agent loop stays fixed, but the files they load: instructions, memory, skills, subagents, scripts and plugins. In the starting harness, H0, these files are empty apart from the benchmark's instructions.
The inner loop, steps 1 to 5 in the diagram below, is what the paper calls a Propose–Probe–Compose step (see the Task-Level Recursion section). The executor first does the task with H0. This is the reference run. Two editor sessions read it, along with every earlier entry in the domain's library, and each write a candidate harness: an invariant one that fixes how the agent works (tool calls, procedure, checking its output), and a domain one that adds what a practitioner would know, usually as a skill. The executor then reruns the task with each candidate under the same conditions. These are the probes. A composer compares each probe with the reference run and keeps, merges or drops the edits; it can also keep just one candidate, or neither. The executor runs the task one last time with the result to produce the deliverable.
That comparison is what replaces a score. It tells the composer whether an edit changed the behaviour it was aimed at, not whether the answer got better.
That is four executions of the task in the basic version. The paper also tries spending more compute in two ways: running several candidates side by side and composing from all of them (parallel), or feeding the final harness back in as the new starting point and going round again (sequential). Its "+ Recursion" results use the parallel version, with five candidates per editor and five composed finals; a selector then picks one deliverable blind, without seeing which harness produced it.
We spent a while on who is allowed to edit what. Our reading on the call was that the invariant editor can touch anything, while the domain editor sticks mostly to skills and guardrails. The editor prompts in the appendix match this: the domain prompt tells it to stay away from scripts, hooks and procedure, because "a parallel candidate owns procedure". The line runs both ways, though. The invariant prompt tells its editor to leave out professional rules and domain content, and to "stay focused on how the agent works".
The outer loop, step 6 in the diagram, runs once the task is done. A reflector reads the whole record, gives each edit a verdict (helped, hurt, no observed effect, or confounded) and writes at most one lesson. If it finds no lesson it can support, nothing is stored. Otherwise the task gets an entry in a harness improvement library: the harness that was used, kept as real files, plus a note on why it was chosen, what each edit did and the one lesson worth carrying over. On every later task in the same domain, the whole library is copied into the editors' and the composer's workspace, and they copy what fits into the new harness. Nothing is inherited automatically, and they never see an earlier task's runs, only its finished harness and the note.
In practice this means that once a later task's reference run with the bare baseline H0 is done, its editors can look through every harness that earlier tasks in its domain ended up with, pick out the pieces that seem to fit, e.g. a skill here, a guardrail line there, and carry them into its own harness after checking them against its own documents. The library grows by up to one entry per task, so the later a task comes in the stream, the more there is to pick from. As such, the paper's claim is that later tasks do better on average, which its ablation and reuse analyses support.
The paper calls this library the improvement harness, R_t. The improver's prompts and procedure never change, so the library is the only part of the whole setup that evolves from one task to the next, which is why the paper's figure draws it as a database.
What the paper doesn't really cover is the interface between the agents and this memory. The library is a folder with an index, and the editors browse it like any other file in their workspace. In the released code the index is a Markdown file that lists each entry with a one-line inventory of its components and the cue of its lesson, and the editor is told to read it first and open what looks relevant. Retrieval, then, is the model skimming a list of one-line cues. Whether that holds up as the library grows isn't studied; the authors list better retrieval as future work. That could have been the most interesting part.
Why does every task start from scratch?
What bothered us most at first is that the base harness, H0, never changes. Every new task starts from it again. If the invariant editor found a bug in how the harness formats tool calls, why not fix it once for everyone?
The intuition we built is that this mirrors catastrophic forgetting in fine-tuning. Specialise a model on one task and it tends to lose what it knew about others. A harness tuned hard for one task is the same kind of risk for the next one, and the paper says as much: inheriting a highly tailored harness causes noisy improvement as tasks shift.
On the call we read the library as holding meta-learnings, lessons about learning rather than about the task. Our analogy was sport. Learning tennis won't teach you ping-pong, but the way you learned tennis should help you learn ping-pong faster.
Having read the code, we think we read too much into it. What the library holds sits one step above the task, not two. It keeps the harnesses that worked, as files, and one rule per task of the form "when this kind of situation comes up, put this in that component". The prompt's own example, quoted below, is a rule about legal drafting. That is knowledge about how to equip the agent for a line of work, kept per domain, and the only sense in which it is meta is that it concerns the harness rather than the answer. Back in tennis terms, what you learned building your return game against one opponent is kept for the next tennis opponent, and a ping-pong match never gets to see it.
Learning without a score
The reflector prompt at the end of the appendix is the most interesting part of the paper for us. There is no score anywhere in the loop: a guard refuses to hand any improver session a directory that contains a grader verdict or rubric. The reflector is told to treat every candidate as evidence, since an edit that appeared only in a rejected candidate may be the one that improved behaviour.
It then writes at most one lesson, pitched at the right level of abstraction. The prompt's own example:
Too generic is useless ('be thorough'); too specific never fires again ('the fee rose to 250 on Jan 1'). Aim between: 'when an instrument performs several sequential acts and one of them terminates the actor's own authority, order them so the terminating act comes last'.
The prompt also tells the reflector to prefer the verdict "confounded" whenever a candidate changed several things at once and no run isolates one of them, and to narrow or drop a lesson that rests only on confounded rows.
We liked the idea, but the whole thing felt fuzzy. The reflector is asked to judge whether an edit helped without ever being told the outcome. In effect the paper assumes the editor model is smart enough to work out what makes a harness better.
Is it really self-improvement?
This is where we pushed back hardest. The paper frames itself as recursive self-improvement, but the executors are Gemini 3.5 Flash and GLM-5, while every harness is written by Claude Opus 4.8. A much stronger model watches a weaker one work and rewrites its harness, and the weaker model then does better.
Our worry is that this is partly distillation: the stronger model's knowledge ends up in skills and guardrails the weaker model can follow. Some of us went further and said that with another model in the loop, it is simply not self-improvement. Using the same model in both roles would have impressed us much more, because it would show the approach can steer a model into a part of its own capability it wasn't using.
After the call, we found the paper runs this experiment in its Analysis section. With the same model as executor and editor, all five models tested, from GLM-5 to Claude Opus 5, improve on every benchmark: Opus 5 gains 3.3, 5.5 and 3.0 points on WorkBuddy-Bench, JobBench and LAB, and GLM-5 gains 13.7 points on LAB. That weakens our distillation worry, although the headline results still use Opus 4.8 as the editor. The released code goes the same way: the improver model defaults to the executor's model, and a stronger editor is an opt-in setting.
Lots of moving parts, little explanation
The other criticism was about the paper's method more broadly. There are many moving parts, the prompts are long and clearly AI-generated, and the paper shows that the system works without really explaining why. The ablation in the Analysis section shows that removing the library, the invariant/domain split or the composition step each costs 1.6 to 3.7 points, which tells us they matter but not what each one actually does.
We did come up with one possible reason it works. Building a harness is mostly a coding task, and models are now good at coding, at reading traces and at pulling high-level heuristics out of them. That is our guess, though, not something the paper shows.
One harness per task
The idea of a harness per task also took some getting used to. Before answering a question, the agent goes through a whole process of rebuilding its own harness. It seemed like a lot.
Two things made it seem more reasonable. A harness is small compared with most projects, so editing it may cost less than it sounds. And although the paper's tasks are an hour's work at most, nothing forces the unit to be that small. A task could be a whole project: an app you are building has its own idiosyncrasies, and a harness tuned to them makes more sense than a generic coding harness.
The expensive part is the task itself. It runs at least four times before you get a result: the reference, the two probes and the final run. For a question that means four answers. For a whole app, it means building the app four times, and twenty times with the parallel setup behind the "+ Recursion" results. Running the probes in parallel saves time but not compute, as the Limitations section admits. For long tasks, that bill adds up quickly.
The takeaway
Harness2 makes a simple bet:
Don't carry a tuned harness from task to task. Start fresh each time, and keep the harnesses that worked, one lesson attached to each, for the next task in the same line of work.
On the call, the meta-learning framing clicked for us as learning how to learn. The code shows something more modest: a store, per domain, of harnesses that worked, each with one rule attached. A harness per task still felt strange to some of us. The reflector prompt is a useful example of how to extract a lesson at the right level of abstraction. We would have hoped the model had delved more into why improvements were thought occur.
The main intuition we took away is that the traces are the asset. Whatever loop you build on top, it can only learn from the runs you kept.