{"id":22941,"date":"2010-01-25T05:11:44","date_gmt":"2010-01-25T05:11:44","guid":{"rendered":"https:\/\/scannn.com\/tigerless-labs-autoharness-autoharness-a-self-learning-skill-layer-for-claude-code-distills-skills-from-your-real-sessions-updates-them-as-you-work-and-prunes-the-ones-that-stop\/"},"modified":"2010-01-25T05:11:44","modified_gmt":"2010-01-25T05:11:44","slug":"tigerless-labs-autoharness-autoharness-a-self-learning-skill-layer-for-claude-code-distills-skills-from-your-real-sessions-updates-them-as-you-work-and-prunes-the-ones-that-stop","status":"publish","type":"post","link":"https:\/\/scannn.com\/lv\/tigerless-labs-autoharness-autoharness-a-self-learning-skill-layer-for-claude-code-distills-skills-from-your-real-sessions-updates-them-as-you-work-and-prunes-the-ones-that-stop\/","title":{"rendered":"tigerless-labs\/autoharness: Autoharness \u2014 a self-learning skill layer for Claude Code \u2014 distills skills from your real sessions, updates them as you work, and prunes the ones that stop getting used. No daemon, no benchmark. \u00b7 GitHub"},"content":{"rendered":"\n
\n

Self-Learning Skills for Claude Code<\/strong><\/p>\n

\n <\/a> \"python\"<\/a> \"platform\"<\/a> \"license<\/a>\n<\/p>\n

autoharness is a self-learning skill layer for Claude Code.<\/strong> It learns<\/strong> skills from your real
\nsessions, merges<\/strong> same-scenario ones instead of stacking near-duplicates, updates<\/strong> them in use,
\nand prunes<\/strong> any that stop getting used \u2014 so the layer stays clean on its own<\/strong>, touching only
\nthe skills it wrote itself<\/strong>.<\/p>\n

Same model, different harness \u2014 42% \u2192 78% on CORE-Bench (HAL<\/a>).
\nThe harness does much of the work (swyx’s Big Model vs Big Harness<\/strong>), yet it’s still rebuilt by
\nhand every model generation. autoharness bets one slice of it \u2014 the skill layer \u2014 can maintain itself.<\/p>\n

<\/p>\n\n\n\n\n\n<\/tr>\n<\/thead>\n\n\n\n\n\n\n
Learns from real work<\/strong><\/td>\nEach episode is distilled into a skill from the session you were already having \u2014 no separate data-collection or replay loop.<\/td>\n<\/tr>\n
Groups, doesn’t just pile up<\/strong><\/td>\nA new episode doesn’t always add a skill \u2014 the reflector compares it against what’s there and folds same-scenario skills into one, so the layer consolidates by category instead of accreting near-duplicates.<\/td>\n<\/tr>\n
Validated in use, not on a benchmark<\/strong><\/td>\nA skill survives by being adhered to in later turns (usage rate), not a held-out score. No oracle on the active path, and no tokens spent on a dedicated eval.<\/td>\n<\/tr>\n
Only its own skills<\/strong><\/td>\nTouches only the skills it generated through this plugin \u2014 everything else, whether you wrote it or installed it, is left completely alone.<\/td>\n<\/tr>\n
Evidence kept for later<\/strong><\/td>\nEvery create\/update logs its scenario and decision to a per-skill ledger \u2014 the raw material to build a benchmark from real usage if you ever want one.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/markdown-accessiblity-table><\/p>\n

Requires python3<\/code> on your PATH<\/strong> \u2014 autoharness runs entirely as Python (zero third-party
\ndependencies); its hooks and MCP server won’t fire without it.<\/p>\n

Type these in the Claude Code input box.<\/p>\n

\n
\/plugin marketplace add tigerless-labs\/autoharness\n\/plugin install autoharness@autoharness\n<\/code><\/pre>\n<\/div>\n
Then run \/reload-plugins<\/code> (or restart Claude Code).<\/p>\n
Zero config. It now watches your sessions and lands learned skills into .claude\/skills\/<\/code> in the
\nbackground. Cadence and lifecycle thresholds are tunable \u2014 see Configuration<\/a>.<\/p>\n
Update from a terminal \u2014 refresh the catalog, then update with the full plugin@marketplace<\/code>
\nid<\/strong>, then restart:<\/p>\n
\n
claude plugin marketplace update autoharness       \nclaude plugin update autoharness@autoharness\n<\/code><\/pre>\n<\/div>\n
Then restart Claude Code<\/strong> to apply \u2014 a version bump is a fresh cached copy, not a hot reload.<\/p>\n
The refresh is first on purpose: without it, update<\/code> checks a stale local catalog and may report
\nalready at the latest version<\/code> when a newer release actually shipped.<\/p>\n
Third-party marketplaces have auto-update off<\/strong> by default. To make future releases hands-off,
\nenable it once: \/plugin<\/code> \u2192 Marketplaces<\/strong> \u2192 autoharness<\/strong> \u2192 Enable auto-update<\/strong>. The
\ninstalled copy is cached by the version<\/code> in plugin.json<\/code>; a release reaches users only when that
\nfield is bumped.<\/p>\n
\n
claude plugin uninstall autoharness@autoharness     \nclaude plugin marketplace remove autoharness       \n<\/code><\/pre>\n<\/div>\n
Uninstalling only stops it from running \u2014 the skills it landed and its own state live outside<\/strong> the
\nplugin and stay on disk. To clear those too, delete its state dir (~\/.claude\/autoharness\/<\/code> global,
\n<repo>\/.claude\/autoharness\/<\/code> per project) and the self-authored skills under .claude\/skills\/<\/code> (each
\ncarries a self-authored<\/code> ledger marker, so they’re easy to tell from yours). Your own skills are
\nnever touched.<\/p>\n
Every knob is an AUTOHARNESS_*<\/code> environment variable with a built-in default \u2014 nothing to
\nconfigure unless you want to change the pace.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n\n
Variable<\/th>\nDefault<\/th>\nWhat it does<\/th>\n<\/tr>\n<\/thead>\n
AUTOHARNESS_REFLECT_EVERY_N<\/code><\/td>\n10<\/code><\/td>\nReflection cadence: a background reflection run fires every N host turns, and each run receives that full N-turn window. Lower = learns faster, spawns more child sessions.<\/td>\n<\/tr>\n
AUTOHARNESS_DIGEST_EXCHANGES<\/code><\/td>\n20<\/code><\/td>\nHow many exchanges before<\/em> the episode window are compressed into the reflector’s prior-context digest (text + tool names only).<\/td>\n<\/tr>\n
AUTOHARNESS_MATURITY_PROJECT<\/code><\/td>\n100<\/code><\/td>\nProbation gate, project layer: after this many requests have arrived in its layer since a skill landed, it faces graduation review \u2014 never used across the whole probation \u2192 archived; used at least once \u2192 graduates into the mature pool. Until then it’s recalled as usual but can’t be archived.<\/td>\n<\/tr>\n
AUTOHARNESS_MATURITY_GLOBAL<\/code><\/td>\n300<\/code><\/td>\nSame gate for the global layer \u2014 higher because a global skill loads in every project.<\/td>\n<\/tr>\n
AUTOHARNESS_CAPACITY_PROJECT<\/code><\/td>\n50<\/code><\/td>\nCap on mature<\/em> skills in the project layer. For graduates, capacity contention is the only death: nothing is archived until the mature pool exceeds this, then the lowest usage rates go first.<\/td>\n<\/tr>\n
AUTOHARNESS_CAPACITY_GLOBAL<\/code><\/td>\n20<\/code><\/td>\nSame cap for the global layer \u2014 smaller because its blast radius is every project.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
<\/markdown-accessiblity-table><\/p>\n
Set them in the environment Claude Code launches with \u2014 either the shell
\n(export AUTOHARNESS_REFLECT_EVERY_N=3<\/code>) or the env<\/code> map in .claude\/settings.json<\/code>:<\/p>\n
\n
{ \"env\"<\/span>: { \"AUTOHARNESS_REFLECT_EVERY_N\"<\/span>: \"<\/span>3\"<\/span><\/span> } }<\/pre>\n<\/div>\n
Hooks read the environment on every event, so a change applies from the next session. The defaults
\nare deliberate placeholders pending empirical calibration (tracked under experiments\/<\/code>); size caps
\non captured windows and staged skill bodies are fixed constants, not env knobs.<\/p>\n
A learning pipeline runs beside the host and stays off its recall path \u2014 symbols are plain native
\nskills, recalled by the host’s own name-and-description mechanism as if a human had written them.<\/p>\n
\"autoharness<\/a><\/p>\n
Diagram source: docs\/assets\/pipeline.mmd<\/code><\/a> \u2014 re-render to pipeline.svg<\/code> after editing.<\/sub><\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Component<\/th>\nRole<\/th>\n<\/tr>\n<\/thead>\n
CAP<\/strong> \u00b7 capture<\/td>\nHook-driven dumb pipe: grabs each turn (user input, agent output, tool I\/O), redacts at egress, points back at the host log instead of copying it.<\/td>\n<\/tr>\n
REF<\/strong> \u00b7 reflect<\/td>\nAt an episode boundary, receives the current episode window in full detail (the last N turns, tool I\/O included) plus a compressed digest of the exchanges before it (text and tool names only), reads the existing skill index, and decides add \/ merge \/ patch \/ drop a support file \/ delete \u2014 emits an intent (body, delta, or path, plus reason and evidence). Proposes only; no write tools.<\/td>\n<\/tr>\n
promoter<\/strong> \u00b7 validate\u00b7store<\/td>\nThe only writer. Lints the intent in memory (safety, structure, ledger, completeness, self-authored-only) and on pass does an atomic rename into the live skill directory.<\/td>\n<\/tr>\n
MNG<\/strong> \u00b7 lifecycle<\/td>\nDaemon-free: recomputed lazily at session start, once per session. Ranks symbols by usage rate \u2014 uses over the requests that arrived since the symbol was created, so the measure is opportunity-relative and a closed laptop doesn’t age anyone out (the wall-clock replacement). A use is counted whenever the host consumes the skill: a Skill-tool invocation or a read of any file in the skill’s directory (measured to be the dominant path). New symbols sit in probation until they’ve had a fair sample of requests: recalled as usual, but neither counted against the cap nor evictable. At maturity, graduation review: zero use across the whole probation \u2192 archived, never enters the pool. For graduates, capacity contention is the only death \u2014 nothing is archived until a layer’s mature pool exceeds its cap, then the lowest rates go first. Archives, never deletes: an archived symbol is a directory moved out of recall, and moving it back revives it.<\/td>\n<\/tr>\n
LED<\/strong> \u00b7 ledger<\/td>\nPer-symbol append-only sidecar: why each symbol was born or changed, with evidence and a reflection watermark. Kept out of the skill body so recall stays clean.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
<\/markdown-accessiblity-table><\/p>\n
\n
Walkthrough: watching it learn<\/h2>\n
<\/svg><\/a><\/div>\n
Everything autoharness does lands on disk as plain files \u2014 a demo is just opening them in the
\nright order. For a fast-paced run, speed up the loop first (see Configuration<\/a>):<\/p>\n
\n
{ \"env\"<\/span>: { \"AUTOHARNESS_REFLECT_EVERY_N\"<\/span>: \"<\/span>1\"<\/span><\/span>,\n           \"AUTOHARNESS_MATURITY_PROJECT\"<\/span>: \"<\/span>5\"<\/span><\/span>,\n           \"AUTOHARNESS_CAPACITY_PROJECT\"<\/span>: \"<\/span>2\"<\/span><\/span> } }<\/pre>\n<\/div>\n
1 \u00b7 The pipeline running.<\/strong> Work a few normal turns on anything non-trivial (debug something,
\nfigure out a workflow). Every Nth turn a background reflection fires \u2014 nothing blocks your session.
\nIts bookkeeping is visible in the state dir:<\/p>\n
\n
ls .claude\/autoharness\/        # per project \u2014 ~\/.claude\/autoharness\/ for the global layer\n  requests                     # layer request counter (MNG's denominator)\n  session-<id>                 # per-session turn count toward the next reflection\n  offset-<id>                  # byte watermark: where the last captured window ended\n  intents\/                     # queued skill proposals awaiting the promoter\n<\/code><\/pre>\n<\/div>\n
2 \u00b7 A skill is born.<\/strong> After a reflection lands, a new folder appears under .claude\/skills\/<\/code>
\n(project) or ~\/.claude\/skills\/<\/code> (global \u2014 for techniques that aren’t repo-specific). Use ls -la<\/code>:
\nthe interesting files are hidden.<\/p>\n
\n
.claude\/skills\/<name>\/\n  SKILL.md                     # the skill itself \u2014 plain native format, nothing proprietary\n  .ledger.jsonl                # LED: why it was born \/ changed (append-only)\n  .sidecar.json                # lifecycle counters MNG reads\n  references\/evidence-*.md     # the transcript slice that justified each ledger entry\n  scripts\/ templates\/ ...      # optional support files the reflector attached\n<\/code><\/pre>\n<\/div>\n
3 \u00b7 LED \u2014 the paper trail.<\/strong> cat .ledger.jsonl<\/code> \u2014 one JSON line per lifecycle event:<\/p>\n
\n
{\"action\"<\/span>: \"<\/span>create\"<\/span><\/span>, \"reason\"<\/span>: \"<\/span>User asked about the correct command to update a plugin ...\"<\/span><\/span>, \"evidence\"<\/span>: \"<\/span>references\/evidence-21cd22cc.md\"<\/span><\/span>}\n{\"action\"<\/span>: \"<\/span>patch\"<\/span><\/span>,  \"reason\"<\/span>: \"<\/span>User discovered \/reload-plugins is required in-session ...\"<\/span><\/span>,  \"evidence\"<\/span>: \"<\/span>references\/evidence-1a4ec51d.md\"<\/span><\/span>}<\/pre>\n<\/div>\n
action<\/code> + reason<\/code> + evidence<\/code> \u2014 and the evidence file is a real, redacted slice of the session
\nthat taught it, materialized by the promoter (content-addressed, so the model never names files).
\nThis is the “evidence kept for later” from the table above.<\/p>\n
4 \u00b7 An update, not a duplicate.<\/strong> Hit the same scenario again with a correction (“that’s missing
\na step”) and let the next reflection run. The skill layer does not<\/strong> grow a near-duplicate:
\nthe existing skill’s SKILL.md<\/code> changes and its ledger appends a patch<\/code>\/update<\/code> line \u2014 the
\ntwo-line ledger above is a real example. git diff<\/code> on a project-layer skill shows the edit.<\/p>\n
5 \u00b7 Recall is the host’s, untouched.<\/strong> Landed skills load like hand-written ones \u2014 same
\nname-and-description recall, no autoharness code on that path. When one is used \u2014 invoked as a
\nskill or read from its directory \u2014 calls<\/code> in its .sidecar.json<\/code> ticks up: that adherence count
\nis the validation signal.<\/p>\n
6 \u00b7 Retirement is an archive, not a delete.<\/strong> Two paths out, both a folder move to
\n.claude\/skills\/.archive\/<name>\/<\/code> \u2014 ledger, evidence and all, out of recall. A skill never used
\nacross its whole probation is archived at graduation review; after graduation, once a layer’s
\nmature pool exceeds capacity the lowest-usage-rate skills go. Moving the folder back revives it,
\nhistory intact. With the shrunk knobs above this fires within one session; at defaults it takes
\nhundreds of turns.<\/p>\n
7 \u00b7 Yours are never touched.<\/strong> Every autoharness-authored skill carries the ledger marker;
\nanything without it \u2014 skills you wrote or installed \u2014 is invisible to the promoter and MNG.<\/p>\n
A self-learning skill layer can be validated against a held-out benchmark, or against its own use.
\nautoharness takes the second \u2014 cheaper, and it works on a live host doing open-ended work where no
\nbenchmark exists.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Grow unbounded<\/th>\nOffline-gated self-edit
(Self-Harness<\/a>)<\/th>\n		Timer + daemon
(hermes-agent<\/a>)<\/th>\n		autoharness<\/th>\n<\/tr>\n<\/thead>\n
Bounds the skill layer<\/td>\nNo<\/td>\nYes<\/td>\nYes<\/td>\nYes<\/td>\n<\/tr>\n
Validation signal<\/td>\nNone<\/td>\nHeld-out benchmark score<\/td>\nWall-clock inactivity<\/td>\nAdherence in use<\/td>\n<\/tr>\n
Needs a benchmark \/ oracle<\/td>\nNo<\/td>\nYes<\/td>\nNo<\/td>\nNo<\/td>\n<\/tr>\n
Needs a resident daemon<\/td>\nNo<\/td>\nNo<\/td>\nYes<\/td>\nNo<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
<\/markdown-accessiblity-table><\/p>\n
NousResearch\/hermes-agent<\/a> \u2014 studying its
\nauto-skill-creation and memory-consolidation design helped sharpen autoharness’s adherence-based,
\ndaemon-free take.<\/p>\n
Built by Tigerless Labs.<\/p>\n
MIT<\/a><\/p>\n<\/div>\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Self-Learning Skills for Claude Code autoharness is a self-learning skill layer for Claude Code. It learns skills from your real sessions, merges same-scenario ones instead of stacking near-duplicates, updates them in use, and prunes any that stop getting used \u2014 so the layer stays clean on its own, touching only the skills it wrote itself. […]<\/p>\n","protected":false},"author":16,"featured_media":22942,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[143],"tags":[],"class_list":["post-22941","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"_links":{"self":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/22941","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/comments?post=22941"}],"version-history":[{"count":0,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/22941\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media\/22942"}],"wp:attachment":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media?parent=22941"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/categories?post=22941"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/tags?post=22941"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}