Kensink Labs
★ Harness hubTechnologies & InfrastructureAgent infrastructure
THE HARNESS · WHERE A MODEL BECOMES AN AGENT

The harness. The layer that turns a model into work.

A model on its own produces text. A harness is the program around it that assembles the context, hands it tools, runs the loop, decides what it is allowed to execute, and remembers what happened. Everyone argues about models. The harness is quietly where most of the difference is made. This is a field guide to how one is built, and a close read of three that matter: Hermes, OpenClaw, and Y Combinator’s qm.

Agent harnessMCPSandboxingTypeScriptPython
Build an agent that ships →All technologies →
THE SWING
27.4 pts, one model
THE STACK
8 layers
COVERED HERE
Hermes · OpenClaw · qm
AS OF
8 August 2026
[WHY THIS PAGE EXISTS]

Same model. Same task. Twenty-seven points apart.

In June 2026 a group of academic and industry researchers built Claw-SWE-Bench: 350 coding tasks across eight languages, run through five different harnesses with the model, the prompt, the budget, and the tool access all held fixed. The only variable was the harness.

On Qwen 3.6-flash, the best harness resolved 231 of 350 tasks. The worst resolved 135. That is 66.0% against 38.6%, a spread of 27.4 points from software choices alone, which the authors note is comparable to or larger than moving up a model tier.

One paper, one benchmark, not yet peer reviewed, and the frontier models were only run under a single harness so there is no top-model comparison in it. Treat it as a strong signal rather than a settled law. It still matches what we see when we put the same model behind two different loops.

Read Claw-SWE-Bench (arXiv:2606.12344)
[THE SHORT VERSION]

Everything around the model that makes it useful.

Where the word comes from

The older term is agent scaffolding, used by METR in autonomy evaluations since 2023. Harness arrived from benchmark jargon, where it meant the code that runs an agent against a task and grades it, and got pulled forward to mean the agent program itself during 2026. Anthropic’s own docs now describe Claude Code as the agentic harness around Claude. The shorthand that stuck:

agent = model + harness

The model is a function: text in, text out. It has no memory, no hands, and no way to check its own work. The harness is the program that gives it all three. It decides what the model sees, what it can reach for, what happens when it asks to run something dangerous, and what survives to the next turn. When people say an agent got better, they usually mean the harness did.

A harness gives you
  • A loop that already runs
  • A tool surface with schemas and validation
  • A policy layer between intent and execution
  • Somewhere isolated for commands to run
  • State that outlives the conversation
A harness is not
  • A framework: you configure it, you do not assemble it
  • A model: swap the provider, the loop stays
  • A guarantee: a weak policy gate is still a weak policy gate
  • A product: shipping agentic features means writing your own loop
The one-line test

If you have to write the loop, it is a framework. If the loop is already running and you are configuring it, it is a harness.

[THE MECHANISM]

Every harness is the same loop underneath.

Assemble the context, call the model, take the tool call it asks for, check it against policy, run it somewhere isolated, feed the result back, repeat. The loop exits when the model stops asking for tools. Everything else in this page is a variation on these six stations.

01Assemble context02Model call03Tool call04Policy gate05Execute in sandbox06ObserveCOMPACT WHEN THE WINDOW FILLSNO TOOL REQUESTED → EXIT THE LOOP, ANSWER
01Assemble contextRunning

The harness builds the prompt for this turn: system instructions, the project's AGENTS.md or CLAUDE.md, conversation history, tool schemas, and any files it retrieved. This assembly step, not the model, is where most harness quality lives.

[THE ANATOMY]

Eight layers sit between an instruction and a command.

The loop is the motion. This is the machinery. Harnesses differ far less in their loop than in how they build these eight layers, which is why two agents on the same model can behave so differently. Click any layer to see how Hermes, OpenClaw, and qm each solve it.

A turn, top to bottomRequest ↓

Click a layer to see how each harness implements it

[THE FIELD]

Three harnesses, three different bets.

These are not three versions of the same product. One is built for a power user, one for a person who wants an assistant on call, one for a company. Reading them side by side is the fastest way to understand what a harness actually decides.

Hermes

Nous ResearchThe agent that grows with you
Stars
227k
License
MIT
Latest
v0.20.0
Language
Python

A maximalist personal agent from the lab behind the Hermes model series. It runs as a terminal TUI, bridges out to six messaging platforms, and ships a desktop app in preview. The bet it makes is breadth: over 70 tools across roughly 28 toolsets, seven execution backends, and sub-agents it can spawn for parallel work.

What stands out
  • Self-improvement is the headline: it turns successful task trajectories into reusable skills and refines them with use.
  • Seven sandbox backends (local, Docker, SSH, Daytona, Modal, Singularity, Vercel Sandbox), two of which hibernate when idle.
  • Model-agnostic with provider fallback and credential pools. Defaults to OpenRouter.
  • Ships hermes claw migrate, which imports an OpenClaw install wholesale. It is positioned as a successor.
!Go in knowing

Nous has published no first-party benchmark numbers, and a GitHub issue asking for them has sat open and unanswered since May 2026. A Chinese startup, EvoMap, alleges the self-improvement system is an architecture-level copy of their Evolver engine; the maintainers blanked one issue's title and body, deleted comments, and blocked users rather than answering it. Judge that history on its own terms, but know it is there.

Name check

Not to be confused with the Hermes LLM series (same lab, but a model), Meta's Hermes JavaScript engine, or the logistics company.

View the repository

OpenClaw

OpenClaw FoundationThe always-on assistant you text
Stars
~385k
License
MIT
Latest
2026.7.1-2
Language
TypeScript

One of the fastest-growing repositories in GitHub history. It runs as a single Node.js gateway process on your own machine and you operate it from WhatsApp, Telegram, Signal, Discord, Slack, or iMessage. Its coding core is Pi, a deliberately minimal agent by Mario Zechner with four tools and a famously short system prompt.

What stands out
  • Memory, sessions, and skills are plain Markdown and YAML on disk. Greppable, Git-versionable, no vendor database.
  • A heartbeat scheduler wakes the agent on a timer so it acts without a fresh prompt. Its most distinctive and most risk-laden feature.
  • It can drive other harnesses: Claude Code, Codex, and OpenCode run as managed background sessions with worktree isolation and PR follow-through.
  • MCP works through mcporter, a first-party bridge. Native MCP support is still an open request.
!Go in knowing

The security record is the story. SecurityScorecard found 40,214 internet-exposed instances in February 2026, 63% of them vulnerable and 12,812 exploitable for remote code execution, leaking API keys and chat histories; Bitdefender later counted 135,000. Cisco found community skills on the ClawHub marketplace performing data exfiltration. China restricted its use on government and state-enterprise machines in March 2026. A maintainer put it plainly: if you cannot understand how to run a command line, this is far too dangerous a project to use safely.

Name check

Renamed twice: Clawdbot (November 2025), then Moltbot after an Anthropic trademark objection in January 2026, then OpenClaw three days later. It is unrelated to the OpenCode project despite being able to run it.

View the repository

qm

Y CombinatorMultiplayer agent harness for work
Stars
12.3k
License
MIT
Latest
v0.1.4
Language
TypeScript

The harness Y Combinator runs itself, open-sourced in late July 2026 and days old at the time of writing. It is the only one of the three designed for a company rather than a person. Multiplayer does not mean several models debating: it means every employee gets a private scope and every Slack channel, group message, and project gets a shared one.

What stands out
  • A scope is the unit of everything: its own memory, files, keychain, permissions, crons, web apps, and a durable sandbox where installed tools stay installed.
  • Harness-agnostic by design. Pi, OpenCode, Codex, and Claude Code all drive the same core, so changing vendor does not mean replacing the platform.
  • Three security postures set org-wide (Strict pauses on every tool call, Auto screens external data, Dangerous does neither) and narrower scopes can only tighten, never loosen.
  • You deploy it into your own Fly.io or AWS account. There is no hosted qm.
!Go in knowing

It is genuinely new, at v0.1.4, and its own reviewers call it unsuitable for unattended high-impact actions or as a hardened multi-tenant boundary. It also does not accept code pull requests: outside contributions go in as written proposals in adrs/ and the maintainers implement them, a policy that dominated its Hacker News launch thread.

Name check

Short for Quartermaster. Not related to the qm command in Proxmox or any queue manager of the same name.

View the repository

Stars, versions, and issue counts as of 8 August 2026. These projects move weekly.

[SIDE BY SIDE]

The same ten questions, asked of all three.

Where the rows disagree is where the design philosophies actually differ. Note the last three in particular: they are the ones that decide whether a harness fits a person or an organisation.

Question
Hermes
OpenClaw
qm
Built for
One power user
One person, always on
A whole company
You reach it via
Terminal, 6 chat platforms, desktop
6 messaging apps
Slack and a web UI
Written in
Python
TypeScript / Node
TypeScript / Node
State lives in
Pluggable memory providers
Markdown + YAML on disk
Postgres
Tool surface
70+ tools, ~28 toolsets
4 core tools + plugins
Small fixed set + execute
Isolation
7 backends incl. serverless
Optional sandbox mode
Durable per-scope MicroVM
Approval model
Dangerous-command detector
Pairing + allowlists
3 org postures + hard denials
Runs other harnesses
No
Yes (Claude Code, Codex, OpenCode)
Yes (Pi, OpenCode, Codex, Claude Code)
Multi-user
No
No
Yes, by design
First released
July 2025
November 2025 (as Clawdbot)
July 2026
[THE EVIDENCE]

What happens when you change only the harness.

Claw-SWE-Bench, June 2026. 350 tasks, eight languages, five harnesses, one model (Qwen 3.6-flash), identical prompts and budgets. Cost is total API spend across the full run, which is the column most comparisons leave out.

Harness
Pass@1
Resolved
Cost
OpenClaw
66.0%
231/350
$71.50
hermes-agent
62.6%
219/350
$103.30
ZeroClaw
58.3%
204/350
$49.30
NanoBot
47.4%
166/350
$133.10
GenericAgent
38.6%
135/350
$14.50
The spread
27.4 points

Between the best and worst harness on an identical model, prompt, and budget.

The cost trap
9× spend

The most expensive run in the paper's other backbone was also near the bottom for accuracy. Spending more is not the fix.

The caveat
One paper

Not peer reviewed, one benchmark construction, and frontier models were only run under a single harness.

[THE REST OF THE MAP]

The three above are not the whole field.

Hermes, OpenClaw, and qm are the ones worth reading closely because they make unusually different bets. But most teams will meet one of these first, and it is worth knowing what else is out there before committing.

Claude CodeAnthropicclosed

The reference implementation, and the one whose docs did most to popularise the word. Terminal, IDE, and cloud.

Codex CLIOpenAIopen source

Runs a separate Guardian model that approves or denies tool calls independently of the model doing the work.

CursorAnysphereclosed

IDE-native, and now ships its own coding model alongside third-party providers.

AiderPaul Gauthieropen source

The long-running terminal pair programmer. Git-native, provider-agnostic, still the pick for disciplined commits.

AmpSourcegraphclosed

Built on code-graph search, aimed at large codebases, and routes between model tiers by task difficulty.

OpenHandsAll Hands AIopen source

Sandboxed and autonomous: browse, execute, edit. Often the pick for unattended headless runs.

GooseBlockopen source

General purpose rather than code-only. Governance moved to the Linux Foundation in 2026.

ClineCline Botopen source

Started as a VS Code extension and grew a standalone CLI and SDK. Several forks trace back to it.

DevinCognitionclosed

The most hands-off of the group: delegate a task, it works in an isolated VM and opens a pull request.

SWE-agentPrinceton NLPopen source

The academic origin of the agent-computer interface idea. Its mini variant is about a hundred lines of Python.

CrushCharmopen source

TUI-first, from the team behind Bubble Tea. Renamed to avoid a collision with the unrelated OpenCode.

Antigravity CLIGoogleclosed

Replaced the open-source Gemini CLI for individual accounts in June 2026. Multi-agent, and closed.

Open source here describes the agent program itself, not an SDK published alongside it. Claude Code’s CLI is closed while its Agent SDK is open, so it reads as closed above.

[THE PART TEAMS SKIP]

A harness is a remote code execution engine you invited in.

That is not a criticism, it is the job description. The value comes from letting a model run commands on your behalf. The risk comes from exactly the same place, and it becomes real the moment three conditions overlap. Simon Willison named this combination the lethal trifecta in June 2025 and the framing has held up.

01

Access to private data

Your repository, your inbox, your customer records. The thing that makes the agent useful is the same thing that makes it worth attacking.

02

Exposure to untrusted content

A GitHub issue, a web page, a forwarded email, a link preview. Anything the agent reads can carry instructions it cannot reliably tell apart from yours.

03

A way to communicate out

An HTTP request, a commit, a reply. Without an exit path the first two are survivable. With one, extraction is a single successful injection away.

Hold all three at once and any text your agent reads becomes a potential instruction to exfiltrate what it can see. This is not theoretical. In February 2026 SecurityScorecard found more than 40,000 OpenClaw instances exposed on the public internet, 63% of them vulnerable, leaking API keys and full chat histories; a later Bitdefender count put the number at 135,000. Cisco found community skills on its marketplace quietly exfiltrating data. The lesson generalises well beyond one project: the default configuration of a powerful harness is rarely the safe one.

✓Do
  • Design the policy gate and the sandbox before the prompt
  • Give the agent its own credentials, scoped to the minimum
  • Run in a disposable workspace: container, VM, or git worktree
  • Keep a human gate on anything that writes to a system of record
  • Log every tool call, its result, and who approved it
  • Pin the harness version and read the changelog before upgrading
✕Don’t
  • Expose a harness control port to the internet
  • Install marketplace skills you have not read
  • Let one agent hold both production credentials and untrusted input
  • Give an unattended scheduler write access to anything that matters
  • Assume a sandbox flag is on because the docs mention it
  • Treat a benchmark score as evidence it will work on your codebase
[WHERE THIS IS GOING]

Six things that changed in the last year.

This is a fast-moving layer and most of what is written about it is already stale. These are the shifts we think actually matter, with the last one being the least discussed and probably the most useful.

01

Parallelism through worktrees

The cleanest way to run several agents on one repository is to give each its own git worktree, so they cannot collide on the same files. An orchestrator fans work out and collects the branches. This is now the default answer to running more than one agent at once.

02

AGENTS.md became a real standard

A plain Markdown file of build commands, test procedures, and architectural boundaries, written for agents rather than people. It started as an open spec in 2025, went to the Linux Foundation in December 2025, and is now read natively by most of the field. This project keeps the same thing in a CLAUDE.md.

03

Two interop layers, not one

MCP standardises agent-to-tool. Google's A2A, now under the Linux Foundation, standardises agent-to-agent discovery and delegation. Treating them as competitors is a category error: most serious deployments will end up speaking both.

04

Agents moved into CI

Headless runs triggered by a push, a PR comment, or a cron, with scoped permissions that stop the agent touching workflow files or bypassing branch protection. The agent reviews the diff before a human opens it.

05

Harnesses that rewrite themselves

The newest and least proven thread. The Ralph loop re-runs an agent against a fixed spec with a fresh context every pass, converging over many cheap stateless attempts. A June 2026 paper reports double-digit gains on Terminal-Bench from letting the harness modify itself with the model held constant. Interesting, not yet something we would build on.

06

Good harnesses get smaller

Anthropic's engineering team makes the point well: every component in a harness encodes an assumption about what the model cannot do alone. They deleted a whole sprint-decomposition layer once a newer model stopped needing it. Harness complexity should shrink as models improve, and most teams never go back and check.

[HOW WE BUILD IT]

How Kensink Labs works with harnesses.

We use these tools daily and we build agentic features into products for clients. Those are two different jobs and we keep them separate.

01

Decide whether the agent is the tool or the product

If it helps your team work, adopt a harness and configure it hard. If it ships to your customers, write the loop yourself. This one decision sets everything downstream, and getting it backwards is the most expensive mistake in this space.

02

Design the blast radius first

Before a single prompt, we settle what the agent can reach, which credentials it holds, where it executes, and what needs a human. A harness with a weak policy gate is not a starting point we will build on.

03

Build the context layer deliberately

Most agent quality is decided before the model is called: what goes into the prompt, what gets retrieved, what gets compacted, and what the project conventions file says. This is the layer we spend the most time on because it pays back the most.

04

Evaluate on your work, not a leaderboard

We build a small eval set from your actual tasks and measure the harness against it. A 350-instance public benchmark tells you something real about harness design. It tells you nothing about your codebase.

[COMMON QUESTIONS]

Questions we get asked.

What is an agent harness, in one sentence?
It is the program that wraps a model and turns it into something that does work: it assembles the context, offers a set of tools, runs the loop of call-and-observe, decides what the agent is allowed to execute, and holds onto state between turns. The model supplies the reasoning. The harness supplies everything else.
How is a harness different from a framework like LangChain?
A framework is a library you build an application with. A harness is a running application you point at a task. LangChain gives you parts; Claude Code, Hermes, and OpenClaw give you an agent that already works and that you configure. The line blurs at the edges, but the test is simple: if you have to write the loop, it is a framework. If the loop is already running, it is a harness.
Does the harness really matter more than the model?
Not more, but by less of a margin than most teams assume. The one controlled comparison we have (Claw-SWE-Bench, June 2026) held the model fixed and swapped only the harness, and saw Pass@1 move by up to 27.4 points. That is a swing on the order of a model-tier upgrade. It is a single, not-yet-peer-reviewed paper on one benchmark, so treat it as a strong signal rather than a settled law, but the direction matches what we see in production.
Which of these three should we actually run?
They are not really competing for the same job. Hermes is the maximalist personal agent, heavy on tools and self-improvement. OpenClaw is the always-on assistant you talk to from a messaging app, and it carries a serious security history you have to design around. qm is the only one of the three built from the start for a company rather than a person, with per-user and per-room scoping. If you want the loop inside your own product rather than an assistant, none of them is the answer: you write the loop.
Is it safe to point one of these at a production system?
Not by default, and mostly not at all without work. The moment an agent can read untrusted text, use tools, and reach the network, you have the conditions for prompt injection to become real damage. Our position is that the policy gate and the sandbox are the parts you design first, before the prompt: least privilege on credentials, an explicit allowlist of commands, a disposable workspace, and a human in the loop for anything that writes to a system of record.
Do we need a harness at all, or should we build our own loop?
If you want an agent that helps your team work, take an existing harness. If you want agentic behaviour inside a product you ship to customers, write the loop yourself. It is a few hundred lines, and you need the control over context, cost, and failure handling that a general-purpose harness deliberately abstracts away. We have done both and the deciding question is always whether the agent is the tool or the product.
Share
View .md
[RELATED]

Worth a look next.

LLM
LLM agents
Read more
LLM
LLM observability
Read more
Services
AI agent development
Read more
BUILD THE LOOP PROPERLY

The model is the easy part.

Picking a provider takes an afternoon. Deciding what the agent can reach, where it runs, what needs a human, and how you will know it worked is the actual engineering. We design that layer, prove it against evals built from your own tasks, and hand you the source. No framework lock-in, no vendor lock-in.
Start a conversation →Read the K-Framework