The Harness Beat the Model: What Google's Self-Improving Agents Actually Proved
Synergised Consulting Ltd
Last Updated: 19 October 2026
Google Cloud AI Research has open-sourced a framework called Harness², in which an AI agent rewrites its own operating setup, and in the published experiments, a weaker model with a self-improved setup beat a stronger model with a standard one. That single result is worth an owner's attention, because most buying decisions around AI tools still assume the opposite: that quality comes from the model alone.
The framing in the paper is simple. A model is the thing you rent or download. A harness is everything wrapped around it that lets it act as an agent: the system prompt, the tools it can call, its memory, its skills, its guardrails. Two businesses can run the identical model and get completely different results, because the harness is where the work actually gets specified.
What Harness² Actually Does
Harness², released by Google Cloud AI Research under the Apache 2.0 licence, is a framework for improving the harness automatically. The model's weights stay frozen; the system around it is what changes. According to the Harness² paper [1], the framework exposes eight editable components around the model: a system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, and a middleware plugin.
The improvement loop works like this. The agent proposes changes to its own harness, tests competing versions on the same task under matched conditions, and keeps the changes that measurably helped, with no human-graded answer key in the loop. It then carries the lessons from that process into the next task. It is self-improvement with the evidence attached, not self-improvement on vibes.
To an owner who has run a discovery workshop, none of this is a foreign idea. The loop is a disciplined version of what a good operator does with any process: try a change, compare it against the previous way, keep what measurably improved, write down why. Harness² simply applies that discipline to the agent's own setup, and does it without anyone re-orchestrating the model.
The Result: A Weaker Model Won
Here is the headline number, and it is the reason this development matters beyond research circles. According to the paper [1], GLM-5, a model that starts weaker than Google's own Gemini 3.5 Flash, was given one automated improvement step to its harness. On WorkBuddy-Bench, a benchmark of office, web and code tasks, the improved GLM-5 scored 75.7. Gemini 3.5 Flash with its standard harness scored 72.4.
The weaker model, with a better harness, beat the stronger model with a default one. Across all three benchmarks the paper tested, professional work, legal work, and office/web/code tasks, a single improvement step gained up to 11.5 points, and the weaker model gained more than the stronger one did.
Read that as a buyer, not as a researcher. The reflex when results disappoint is to upgrade the model, which usually means paying more per task forever. The Harness² result says some of that gap sits in the setup, not the model, and the setup is the part you own, can inspect, and can fix without changing vendors.
What It Looks Like in Practice
An illustration, so this does not stay abstract. A business runs an agent that drafts supplier-reconciliation summaries each Friday. The model is fine; the output keeps missing the discount terms buried in PDF attachments, so someone re-checks every summary by hand. Under a Harness²-style loop, the agent proposes three changes to its own instructions and tooling, one adds an explicit extraction step for attachment text, one restructures the prompt, one adds a checklist subagent. It runs all three variants against the same stack of real documents, compares behaviour, and keeps the extraction change because the evidence supports it. Nobody retrained anything. The setup changed, and the score with it.
The eight components matter here too. Skills and scripts, subagents and tool guidance are exactly the layers a small business's agent setup already has, whether or not anyone wrote them down. Harness² makes editing them systematic; the underlying point is older and simpler, those layers exist, they decide the output, and most businesses have never inventoried them.
What It Is Not
The repository is honest about its limits, and the limits matter more than usual here. The project carries Google's own disclaimer that it is "not an officially supported Google product" [2]: a research release, not a supported platform.
Two operational caveats sit in the project's own documentation. First, executors run with network access unrestricted, and the Codex executor runs with its sandbox disabled [2]: an agent authorised to rewrite its own setup is also an agent whose guardrails are part of what gets rewritten, which is a governance question before it is a performance one. Second, the default evaluation judge requires a Gemini or Vertex endpoint [2], so running it yourself means API accounts, keys, and per-run costs, not a free lunch that improves your agent while you sleep.
And a business's own tasks rarely come with the clean, bounded task streams the benchmarks use. Self-improvement loops inherit whatever the measurement can see; if nobody can score a task, the loop has nothing to hold onto.
What an Owner Takes From This
Two things, and neither requires running the framework.
First, the score gap between model choices is partly a harness gap. When an agent's output disappoints, the questions worth asking before a model upgrade are: what is the system prompt, what tools does it actually have, what does its memory hold, where are its guardrails written down. In our own agent work, the difference between a mediocre and a useful agent has rarely been the model; it has been how precisely the work was specified around it. Documentation of those layers is cheap; model upgrades are not.
Second, improvement needs a measurement to improve against. Harness² only works because each change is tested and kept or discarded on evidence. A business that wants better agent output can borrow the discipline without the framework: keep a dated record of what the agent was asked, what it produced, what changed in the setup, and whether the output improved. That record: a written inventory plus a change log. is what turns "we should get a better model" into "our extraction step misses attachment text; here is the fix and here is the measured difference it made."
Sources
- [1] Harness 2: Recursive Agent Harnessing for an Open World, Google Cloud AI Research, arXiv 2610.14224
- [2] google-research/harness2 repository, GitHub, Apache 2.0
