{"id":23471,"date":"2026-08-22T07:30:52","date_gmt":"2026-08-22T07:30:52","guid":{"rendered":"https:\/\/scannn.com\/the-evolution-of-the-agent-harness\/"},"modified":"2026-08-22T07:30:52","modified_gmt":"2026-08-22T07:30:52","slug":"the-evolution-of-the-agent-harness","status":"publish","type":"post","link":"https:\/\/scannn.com\/lv\/the-evolution-of-the-agent-harness\/","title":{"rendered":"The Evolution of the Agent Harness"},"content":{"rendered":"\n
\n
\n
<\/p>\n
<\/picture><\/div>\n

<\/a><\/figure>\n<\/div>\n

Sometime around Christmas 2025, AI engineers noticed a change in agents. They started to work! It\u2019s hard to pin down exactly why. Maybe we finally had holiday downtime to try the newest agents with the newest models. Maybe the models had crossed some <\/span>capability threshold<\/span><\/strong>. Maybe <\/span>the wrappers<\/span><\/strong> around the models had matured.<\/span><\/p>\n

What I\u2019ll argue in this post is that it was the confluence of the last two. <\/span>The model and the harness improving together<\/span><\/strong> and then their curves of improvement crossing at the right moment. And that dynamic helps to explain what comes next: models keep absorbing the harness into their weights, engineers keep deleting what got absorbed, and <\/span>what remains is a harness for human attention rather than for the model.<\/span><\/strong><\/p>\n

Lukasz Kaiser, one of the people who invented the Transformer, said on <\/span>\u201cUnsupervised Learning\u201d<\/span><\/a> in June:<\/span><\/p>\n

\n

\u201cThe change last winter, last Christmas \u2014 it\u2019s a little hard to pin down. I mean, the harness changed and a little post-training changed and then new pre-trained models came\u2026 but it felt like a <\/span>big jump<\/span><\/strong> which is not that easy to pin down what did it.\u201d<\/span><\/p>\n<\/blockquote>\n

The answer to <\/span>\u201cWhat happened?\u201d<\/span><\/strong> isn\u2019t solely in the model weights. It\u2019s in the system that grew up around the weights.<\/span><\/p>\n

The answer is in the agent harness.<\/span><\/strong><\/p>\n

Think back to November 2022, when ChatGPT was the most advanced AI tool. The only capability at its disposal was next-token prediction and some Reinforcement Learning from Human Feedback (RLHF) that allowed it to act like a helpful assistant. No tools, no search, and no reasoning.<\/span><\/p>\n

The original ChatGPT was confined to its training data and the prompt you sent it. No more, no less. <\/span>It was a brain in a vat.<\/span><\/strong><\/p>\n

The agent harness is a way for the LLM to <\/span>break free from that confinement<\/span><\/strong> and interact with real digital information space.<\/span><\/p>\n

An agent harness is everything besides the model weights that makes the agent work.<\/span><\/strong> The environment, tools, context and guardrails that surround the model. Without the harness the model is a brain in a vat. It can take an epistemic action, but needs the harness to actuate that decision in real digital space.<\/span><\/p>\n

The harness is like giving the mind of the model a body.<\/span><\/strong> With the harness, the model can perceive (context), act (tools), persist information (memory and compaction), and enforce its boundaries (permissions and guardrails).<\/span><\/p>\n

\n
<\/p>\n
\"\"<\/picture><\/div>\n

<\/a><\/figure>\n<\/div>\n

Two curves run through the path of model \/ harness evolution. <\/span>What the harness asks of the model, and what the model can deliver in practice.<\/span><\/strong><\/p>\n

The gap between these two curves is equal to the effectiveness of an agent, and the closing of that gap is what I\u2019ll argue led to the tangible improvement in agents that Lukasz Kaiser referenced.<\/span><\/p>\n

Here\u2019s how the gap closes, in stages:<\/span><\/p>\n

    \n
  1. \n

    ReAct, \u201cThe Harness on Paper\u201d<\/span><\/strong> (October 2022): <\/span>ReAct<\/span><\/a> is a prompting technique to get models to reason through prompting. It\u2019s the agentic loop on paper, external to the model weights. It defines the idea of an \u201cagent loop\u201d where a model reasons -> acts -> observes -> repeats. Again, <\/span>the ReAct loop exists only as a prompting method.<\/span><\/strong> Prompting is the only reasoning method that exists at this time and no one calls it a \u201charness.\u201d Toolformer (Meta, Feb. 2023), that same winter, hints that tool use could be trained in rather than prompted. It\u2019s a bit like Alan Turing\u2019s idea of the computer before it was instantiated in a physical substrate. A powerful idea that is only later made manifest. (ReAct predates ChatGPT by a month \u2014 October 2022 vs. November 2022). Both the curves are near zero at this point. The gap is small because <\/span>we are just getting started<\/span>.<\/span><\/p>\n<\/li>\n

  2. \n

    AutoGPT\/BabyAGI, \u201cPremature Autonomy\u201d <\/span><\/strong>(Spring 2023): With AutoGPT\/BabyAGI, the harness curve sprints ahead of the model capability curve. Both hand the model full autonomy, asking the model to act as an \u201cautonomous employee,\u201d but the models at this point are still little more than brittle next-token predictors. <\/span>A loop doesn\u2019t add capability to a model.<\/span><\/strong> A loop amplifies the capability a model has, and below some threshold the loop amplifies errors rather than reliability. Consider the power of compounding in the negative: 95% reliability per-step over a 20-step task results in a ~36% average success rate. The harness hands the model an assignment it has no realistic chance of completing. <\/span>This is where the gap is at its widest<\/span><\/strong> and the next 18 months are a reaction and attempt to close that gap.<\/span><\/p>\n<\/li>\n

  3. \n

    Cursor\/Copilot, \u201cRetreat to Human in the Loop\u201d <\/span><\/strong>(2023 – 2024): The first AI-powered IDEs recognize the failure-mode of giving the model too much autonomy. They close the gap by pulling the harness curve down below the model curve. <\/span>Don\u2019t give the model the loop directly; give the human the loop<\/span><\/strong> and empower the human to orchestrate the loop while the model speeds the human up. The first version of Devin tries to hand the autonomy back to the model. A <\/span>test from the team at Answer.AI<\/span><\/a> shows that is still premature, with a ~15% success rate. It\u2019s evidence that the move from the IDEs to retreat from full autonomy is not cowardly, but the correct move. However, while the prevailing tactic is to pull the harness down below the model, <\/span>models continue to improve<\/span><\/strong>. Near the end of 2024, with the introduction of o1 \u2014 the first reasoning model \u2014 for the first time the gap <\/span>inverts<\/span><\/strong><\/em> and we begin to see the first signs of a model capability overhang.<\/span><\/p>\n<\/li>\n

  4. \n

    Claude Code, \u201cThe Curves Cross\u201d <\/span><\/strong>(February 2025): The inversion at the end of 2024 sets up an opportunity that someone has to seize: if the model is now ahead of the harness, then a harness intentionally riding the brakes of the model is leaving capability on the table. <\/span>Claude Code is the first coding agent built to seize that opportunity.<\/span><\/strong> It abandons the IDE for the terminal, gives the model bash and file read\/write access, and replaces the need for human approval on every change with <\/span>permission rules<\/span><\/strong>. The model is handed the loop again, and this time it understands the assignment. <\/span>Boris Cherny and team build Claude Code with the next model\u2019s capabilities in mind, not the current one.<\/span><\/strong> It is such a hit not because it\u2019s the first product to give the model autonomy, but because it\u2019s the first product to do so at the right <\/span>time.<\/span><\/strong><\/em> That time is <\/span>the crossover point where the model has gotten reliable enough to succeed with autonomy<\/span><\/strong>. Claude Code grows to roughly $1B ARR within six months, all because Anthropic seized the opportunity available when the curves begin to meet.<\/span><\/p>\n<\/li>\n<\/ol>\n

    What happens next is that the curves don\u2019t just meet, they begin to <\/span>braid <\/span><\/strong><\/em>together.<\/span><\/p>\n

    Today the harness matters, and in a way we can measure.<\/span><\/strong> <\/span>Harness-Bench<\/span><\/a> ran the same model over the same 106 tasks in different harnesses, and scores ranged from 52.4 to 76.2: a 23.8-point spread with zero change to the model. <\/span>Half the agent is the harness.<\/span><\/strong><\/p>\n

    \n
    <\/p>\n
    \"\"<\/picture><\/div>\n

    <\/a><\/figure>\n<\/div>\n

    OpenAI achieved a similar result<\/span><\/a> on ARC-AGI-3 with harness changes. Adding only retained reasoning and compaction, GPT-5.6 Sol\u2019s ARC-AGI-3 score tripled from 13.3% to 38.3%.<\/span><\/p>\n

    \n
    <\/p>\n
    \"\"<\/picture><\/div>\n

    <\/a><\/figure>\n<\/div>\n

    What\u2019s happening under the hood is that Reinforcement Learning (RL) has moved <\/span>inside<\/span><\/strong><\/em> the harness. From OpenAI\u2019s codex-1 release announcement in May 2025: \u201ccodex-1 was trained using reinforcement learning on real-world coding tasks in a variety of environments.\u201d<\/span><\/p>\n

    The two curves join and start to braid as one unified system.<\/span><\/strong><\/p>\n

    This is the dream of Toolformer manifesting in reality. Rather than a tool prompted from the outside, now tool calling is trained from within the <\/span>environment<\/span><\/strong><\/em> of the model.<\/span><\/p>\n

    Then, as the models are trained in the environment of the harness, <\/span>they start to absorb the harness capabilities into the model weights<\/span><\/strong>, learning how to auto-compact with knowledge of their own context window, for example.<\/span><\/p>\n

    GPT-5.1-Codex-Max launch:<\/span><\/p>\n

    \n

    \u201cThe first model natively trained to operate across multiple context windows through compaction.\u201d<\/span><\/p>\n<\/blockquote>\n

    Once the models absorb the harness capabilities, the harness can shed the scaffold.<\/span><\/strong> It\u2019s production by reduction. Thariq Shihipar from Anthropic said that the team <\/span>recently deleted 80%<\/span><\/a> of Claude Code\u2019s system prompt.<\/span><\/p>\n

    The measure of the pace of agent harness evolution is <\/span>how much of the harness you get to delete<\/span><\/strong>, while retaining the same capability level. This is the future we need to build towards as AI engineers.<\/span><\/p>\n

    This, then, is the loop of model \/ harness evolution: <\/span>train<\/span><\/strong> -> <\/span>absorb<\/span><\/strong> -> <\/span>shed<\/span><\/strong> -> <\/span>repeat<\/span><\/strong>. The model climbs to the next thing it can\u2019t do yet.<\/span><\/p>\n

    The jump that Kaiser pointed out is hard to pin down because it\u2019s not a discrete event. A pre-training leap is noticeable because you can articulate it in a model card. A model \/ harness co-evolution jump is less so, because there\u2019s no documentation of the evolution process. That\u2019s the answer to the jump last Winter: <\/span>it happened in the space between the model and harness working together.<\/span><\/strong><\/p>\n

    We need to ask: if every harness capability will eventually get absorbed into the model, what does that leave us with?<\/span><\/p>\n

    Keep deleting everything that the model can absorb. Imagine what your agent looks like at the conclusion of that process. What are you left with in your hand when you\u2019ve deleted everything?<\/span><\/p>\n

    What do the model weights absorb next? Multi-agent orchestration, tool selection, memory…to name a few possibilities. Researchers are building <\/span>self-improving harnesses<\/span><\/a> that can themselves be trained in a similar way to models.<\/span><\/p>\n

    What\u2019s left at the end of this deletion and absorption process are <\/span>the human-centric agent capabilities<\/span><\/strong>. Things like <\/span>permissions, identity, trust and legibility<\/span><\/strong>. A model that absorbs permissions into itself has dissolved permissions. Absorption doesn\u2019t end the harness. Absorption inverts the harness.<\/span><\/p>\n

    The harness becomes the agent\u2019s interface to the human that operates it.<\/span><\/strong><\/em><\/p>\n

    The harness was born as the human interface to the model. We grew from chatbox to IDE to the terminal. If the model absorbs the computer-facing capabilities, the next stage of evolution becomes one layer of abstraction up. The harness becomes the model\u2019s interface to our human attention.<\/span><\/p>\n

    It becomes the <\/span>attention-interface.<\/span><\/strong><\/p>\n

    Ryan Lopopolo said on the \u201c<\/span>Extreme Harness Engineering for Token Billionaires<\/span><\/a>\u201d episode of Latent Space:<\/span><\/p>\n

    \n

    \u201cThe only fundamentally scarce thing is the synchronous human attention of my team.\u201d<\/span><\/p>\n<\/blockquote>\n

    Tokens became abundant and reliable, yet <\/span>we remain bottlenecked on scarce human attention.<\/span><\/strong><\/p>\n

    We see sparks of this already, with Anthropic\u2019s long-running agent progress files and agentic approval queues.<\/span><\/p>\n

    The gap between the model and harness curve doesn\u2019t disappear when the model absorbs the harness. It migrates across the human boundary and creates a new pair of curves with a new gap. <\/span>The new gap<\/span><\/strong> is the space between what the agent asks of the <\/span>human<\/span><\/em>, and <\/span>what the human is able to answer<\/span><\/strong>.<\/span><\/p>\n

    I predict that within a year, <\/span>every company building agentic AI will ship a human attention policy surface<\/span><\/strong> in the way that every agentic AI company shipped <\/span>AGENTS.md<\/span><\/a>.<\/span><\/p>\n

    AGENTS.md tells the agent how to work with your codebase. <\/span>The attention-interface will tell the agent how to work with <\/span><\/strong>you<\/span><\/strong><\/em>.<\/span><\/strong> It will govern when it\u2019s allowed to interrupt you, when it should keep working, which decisions it can make alone and which decisions need your approval. And like everything else in the agentic system, <\/span>it will become a learnable component of the system that can learn with more data.<\/span><\/strong> Every correction becomes useful data.<\/span><\/p>\n

    The model began as a brain in a vat. The harness gave the brain a body, then the body started to dissolve into the brain. What\u2019s left for us to build is the thing no future model will ever absorb. The interface to the one true scarce resource: <\/span>human attention.<\/span><\/strong><\/em><\/p>\n<\/div>\n

    Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    Sometime around Christmas 2025, AI engineers noticed a change in agents. They started to work! It\u2019s hard to pin down exactly why. Maybe we finally had holiday downtime to try the newest agents with the newest models. Maybe the models had crossed some capability threshold. Maybe the wrappers around the models had matured. What I\u2019ll […]<\/p>\n","protected":false},"author":16,"featured_media":23472,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[143],"tags":[],"class_list":["post-23471","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"_links":{"self":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/23471","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/comments?post=23471"}],"version-history":[{"count":0,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/23471\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media\/23472"}],"wp:attachment":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media?parent=23471"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/categories?post=23471"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/tags?post=23471"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}