cd ~/
Agentic AIAgent ArchitectureInfrastructureDeveloper ToolsMulti-Agent Systems

25 Ranks From the Harness Alone

LangChain jumped 25 ranks on TerminalBench by changing only the harness. The model never changed.

Evyatar Bluzer
3 min read

For months I have spent more time on guardrails, feedback loops, and tool plumbing than on prompts, without a good name for that work. April fixed the naming problem.

The short version: the harness - everything in an agent system except the model - is now the biggest performance lever available. LangChain's Deep Agents team gained 14 points and 25 ranks on TerminalBench 2.0 without changing the model at all.

Agent = Model + Harness

The term crystallized this month. OpenAI shipped its Agents SDK update on April 15 with a model-native harness - sandbox execution, filesystem tools, snapshot and rehydration - built from the same scaffolding that powers Codex. Martin Fowler published a detailed synthesis the same week, and an awesome-harness-engineering repo appeared on GitHub. The concept now has a name and a community forming around it.

Harness means everything in an agent that is not the model itself: the tools it can call, the guardrails that constrain it, the feedback loops that help it self-correct, and the observability layer that lets humans monitor behavior.

The LangChain Proof Point

LangChain's Deep Agents team took GPT-5.2-Codex from outside the top 30 at 52.8% to rank 5 at 66.5% on TerminalBench 2.0, and the model never changed. What they swapped was the harness: self-verification loops, loop detection middleware, better context engineering. A 14-point jump from infrastructure alone is the kind of result that should change how teams allocate engineering effort.

What Goes Into a Harness?

The emerging taxonomy breaks a harness into seven categories:

  • Guides - feedforward controls that steer before the agent acts
  • Sensors - feedback controls that observe after the agent acts and trigger self-correction
  • Rails - hard constraints that prevent catastrophic actions
  • Scaffolds - the execution environment, sandbox, filesystem access
  • Exemplars - few-shot examples and demonstrations
  • Mirrors - self-reflection and evaluation mechanisms
  • Throttles - rate limiting, cost controls, token budgets
Model plus harnessAn outer box labeled agent holds a full-width model box on top and, below it, a highlighted harness box containing seven smaller cells in two rows. Agent = Model + HarnessModelGPT-5.2-Codex, never changedHarness: everything in the agent that is not the modelswapped on its own, GPT-5.2-Codex went from 52.8% to 66.5% on TerminalBench 2.0Guidessteer beforethe agent actsExemplarsfew-shotdemonstrationsRailshard limits oncatastrophic actionsScaffoldssandbox, filesystem,executionSensorsobserve after the act,self-correctMirrorsself-reflection,evaluationThrottlesrate limits, cost,token budgets
An agent is one model plus a harness of seven named parts; LangChain's Deep Agents team swapped only the harness and took GPT-5.2-Codex from 52.8% to 66.5% on TerminalBench 2.0 with the model box unchanged.

Fowler frames it as three interlocking systems: context engineering, architectural constraints, and entropy management. Different cut, same conclusion: the model is necessary but not sufficient. The rails category is where the kill switch no agent is allowed to see lives.

The Vocabulary Catches Up

Fowler identifies three human postures - humans outside, in, or on the loop - and argues that maintaining the harness, rather than reviewing individual outputs, is the only approach that scales with agent throughput. I agree with him. The CLAUDE.md files, AGENTS.md conventions, and skill files an agent can edit for itself that I touch every day are harness engineering, which means plenty of us were practicing the discipline before it had a name. Now that it has one, I expect the job listings within a quarter.

Comments