An AI harness is everything around the AI model that is not the model itself. That means the instructions, tools, permissions, memory, checks and loop that turn a model into a working agent. It matters because early evidence suggests that, among top models, the harness can change results as much as or more than the model does.
Key takeaways
- Agent = model + harness. The harness is the code, settings and logic around the model.
- A preprint reports that swapping the harness moved one model’s score by 13 points, while swapping models inside one harness moved scores by 2.5 to 5 points.
- Good harnesses use approval gates, progress logs and checks before a task counts as “done”.
- When you compare AI tools, test the whole tool on your own tasks, not just the model name.
What is a harness?
LangChain’s Vivek Trivedy defines the harness as “every piece of code, configuration, and execution logic that isn’t the model itself”. His summary formula is “Agent = Model + Harness”. LangChain lists these parts: instructions, tools and skills, infrastructure such as a filesystem or sandbox, orchestration such as subagents and model routing, and hooks that handle things like context compaction and lint checks.
A 2026 arXiv study of the source code of eleven coding agents gives a similar definition. It describes the harness as the runtime that links the model to its environment. That runtime covers the control loop, tools, context management, safety controls, orchestration and extension points. The authors say the discipline of “harness engineering” was named in early 2026.
Anthropic does not give a formal definition. Its long-running agents article calls the Claude Agent SDK a “general-purpose agent harness”. In that article the harness runs the agent loop, gives the model tools and manages context through compaction.
What is inside a harness
- Instructions. What the agent should do and how.
- Tools. The things it can use, such as search, files, a browser or your booking system. MCP servers are one way to add tools.
- Permissions. The Agent SDK documentation says you can control which tools run automatically and which need approval.
- A sandbox. A safe place to run work, either hosted or self-managed.
- Orchestration. Handing work to subagents or routing between models.
- Checks. Hooks that test work before it is accepted.
Lessons from long-running agents
In November 2025, Anthropic’s Justin Young wrote that “consistent progress across multiple context windows remains an open problem”. An agent that works for hours loses track of what it did earlier.
Anthropic’s fix had two parts. An initialiser agent sets up the environment on the first run. Then a progress file, called claude-progress.txt, records what each session did, so the next one can pick up from there. Anthropic also saw a clear failure, Claude’s “tendency to mark a feature as complete without proper testing”. The harness therefore needs a test step before anything is called finished.
In June 2026, Anthropic described three failure modes of single-context work: stopping early, favouring its own output, and drifting from the goal after compaction. It says separate subagents with their own context counter these. It also notes that such workflows often use more tokens, so they suit complex, high-value jobs.
OpenAI’s harness engineering experiment
OpenAI published its own account in February 2026. It says a small team spent about five months building a product of roughly one million lines of code with no manually written code. OpenAI reports the team grew from three to seven engineers and merged around 1,500 pull requests. It estimates the work took about one tenth of the time of hand-coding. These figures are OpenAI’s own and have not been independently checked.
The lesson OpenAI draws is that the engineer’s job becomes designing environments, specifying intent and building feedback loops. Its summary line is “Humans steer. Agents execute.”
Same model, different harness
A May 2026 preprint, “Stop Comparing LLM Agents Without Disclosing the Harness”, argues that for long tasks the harness can matter more than the model. A preprint has not been peer reviewed. Treat the numbers as early evidence.
- The authors cite a leaderboard (Morph) showing Claude Opus 4.5 scoring 45.9% on SWE-bench Pro in a standard scaffold and 55.4% in Claude Code.
- In their own test on a 100-task subset of SWE-bench Verified, switching harness moved GLM-5.1 from 52.5% to 65.5%.
- Switching model inside one harness moved scores by only 2.5 to 5.0 points. In 6 of 9 model comparisons, the ranking reversed across harnesses.
The sample is small: three models, three harnesses, 100 tasks and two runs per cell. It is also a single paper. The direction is still useful to you as a buyer.
Context engineering sits inside the harness
Anthropic defines context engineering as “curating and maintaining the optimal set of tokens” during inference. It covers what goes into the model’s view: tools, outside data and message history. Our reading is that this is one job inside the harness. The harness also runs the loop, permissions, sandbox and checks. No source states it in exactly those words.
Two meanings of “harness”
Be careful with the word. In 2025 and 2026 it usually means the runtime around an agent. Older usage means a testing rig. EleutherAI’s lm-evaluation-harness, for example, is “a unified framework to test generative language models” with more than 60 academic benchmarks. They are different things. One runs your agent. The other scores a model.
What this means for your business
- Add approval gates. Let an agent read freely, but require a person to approve sending emails, payments and deletions.
- Keep a progress log. For long jobs such as building a site or a set of reports, a written log and checklist let each session continue where the last stopped.
- Require checks before “done”. Make testing a step, not a hope.
- Start simple. Anthropic advises finding “the simplest solution possible” and adding complexity only when needed.
- Evaluate tools on your own tasks. Two products using the same model can perform very differently. Try your real jobs in each.
If you want an AI agent built with sensible limits, logs and checks, talk to us.
Sources
- The Anatomy of an Agent Harness, LangChain, 10 March 2026
- Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents, arXiv (preprint), v1 submitted 15 July 2026
- Effective harnesses for long-running agents, Anthropic (Justin Young), 26 November 2025
- A harness for every task: dynamic workflows in Claude Code, Claude Blog, 2 June 2026
- Harness engineering, OpenAI, 11 February 2026
- Agent SDK overview, Claude Code documentation, undated
- lm-evaluation-harness, EleutherAI, GitHub, undated
- Stop Comparing LLM Agents Without Disclosing the Harness, arXiv (preprint), 7 May 2026
- Building effective agents, Anthropic, 19 December 2024
- Effective context engineering for AI agents, Anthropic, 29 September 2025