The completion meter

An agent that says it is finished has made a claim. On 19 October I measured how often those claims held in my own setup.

A walk-away window is a fresh session that builds an approved plan with nobody watching. On 19 October all 5 of them had counted nothing, because only an approval in the window itself told Claude the meter was counting. Of the ticks the meter had kept, 76 of 263 rested on a single bare name such as REPORT.md: a file existing, not a check passing.

So the question changed. Not "did Claude say it is done" but "which command ran, exactly as written, and how did it exit".

Every item in an approved plan ends with the command that proves it. The plan cannot be approved without one.

The plan for this very post has 11 steps and 5 checks, and each carries a Proof: line read the way Markdown reads a code span. The plan-exit hook that guards approval refuses a missing proof, a bare name, a <placeholder> and a proof cut short inside its own backticks.

Two items name no command. S10 is mine: "your answer in the final question box, which names S10". S11 is a later event, the live build after my go. The meter files both outside Claude's part, so neither can be closed by Claude running something.

A proof that cannot fail proves nothing. Each one is chosen so the wrong world prints a different answer.

This is a rule the plan's author holds, not something the meter can check. Step 2 is a case in this build: its proof chains the kit's test suite to a privacy scan with &&, so either failing fails the line. Before trusting the scan's zero, the build planted a private address and an employer's name in a scratch repository and watched the same scan exit 1.

The same step fixed a check that could not fail. The kit's suite counted a skipped arm as a pass, so a machine missing a tool printed the same total as one that ran everything. It now prints a skip count beside the passes.

At approval the list is frozen by wording. Editing the plan later cannot move a proof onto a different item.

// stages.ts
export const itemKey = (list, full) =>
  `${list}:${full.replace(/\*\*/g, '').replace(/\s+/g, ' ').trim().toLowerCase()}`

Each item's key is its list and its whole text, so an inserted step or a reworded one leaves every other proof where it was. Before 19 October an inserted step silently dropped the proofs after it.

A walk-away window never saw the approval happen, so the plan is carried to it: the launcher's /goal line names the plan file, and the meter freezes the list from that. This window's band read "approved in the planning window" from its first turn.

A run counts only when it is the main session's own shell call, and its command is the proof's whole command as written.

export function runOf(u, sites, home) {
  if (!isEvidenceTool(u.tool)) return []
  const said = normCmd(String(input.command ?? ''), home)
  ...
  return sites.filter(s => normCmd(s.cmd, home) === said)
}

Equality, not containment. The matcher before 25 October accepted the proof anywhere in a command, so words before it, a longer word holding it, text typed into a browser and a helper's run could all count. normCmd forgives only what cannot change what runs: runs of spaces, a quoted plain path, and the home folder however it is written.

This build shows the rule biting. Step 3's first build ran with its output sent to a log file, a different command, and the meter ignored it. The second ran exactly as written and proved two items at once, S3 and V1, because the plan gives both the same proof.

Measured before the change: of 287 proofs run as their whole command in 30 days, 285 were typed exactly as written, and the other 2 differed only in path spelling, which normCmd reads one way.

The run's own exit decides. Nobody reads the output and judges it.

export function exitedClean(isError, result) {
  if (isError) return false
  const why = isRecord(result) ? result.returnCodeInterpretation : undefined
  return !(typeof why === 'string' && why.trim() !== '')
}

Step 1's proof prints nothing at all. grep -q answers only with its exit code, and that is the point: the meter reads the tool's error flag and Claude Code's own reading of the return code, never the text, which is the command's output and could say anything.

A proof started in the background, or one that outran its time limit, proves nothing until the engine reports how it ended.

Before 25 October one window kept a timed-out run as a pass, and that run's later failure never reopened it. Now an answer carrying a background task id marks the run pending. The task id is read from the answer's own field, never its text, and only the engine's own finish notice ends it. A line in a user message that looks like a notice ends nothing, because a pasted message or a peer session could carry one.

Each item keeps only its newest run. A pass proves it on the spot, and a later failure takes the proof back.

The rule's shape, not a record. This build's real runs are on the panel and in the last stage.

Before 25 October Claude claimed an item after a passing run, and the claim was what counted. The claim step is gone: the count sees the run and decides, so there is no moment where an item reads done because Claude said so. Building on a proved step and breaking it reopens it, which is exactly when it should reopen.

Some items only I can close. My words close them, and only words that name the item.

A quote counts only from a prompt typed at my terminal, sent from my phone, or my answer in a question box, and only when that prompt or box names the item's id. My answer about S2 decides nothing about S3. The meter used to read every prompt event, which also fires for task notices and messages from other sessions, and 110 of 140 quotes it had kept as mine were those.

My words never stand in for a command. On an item in Claude's part they count only once Claude has handed the item over as something nothing it can run will close.

The whole count lives on one line above the prompt, beside the HUD.

Seven stages, each done, under way, waiting or skipped. The headline counts Claude's part: its own items, plus commit, push and close, which is why this plan's 14 items make a part of 17. The bar fills whole cells rounded down, so it never reads full until every unit is done. Items planned as mine or as a later event's sit beside the count, never inside it.

Commit and push is read from git alone. The meter asks git whether each edited file is committed and pushed, and an edit after a commit sends the stage back.

Asked with no items, the meter prints the tally: what is open, who acts on it, and a last line that is the verdict.

This is this build's first tally, at 07:41 ET. Every answer the tool gives ends with that line, and the verdict asks exactly what Claude's part asks: every item proved, and commit and push done or not needed. Until 25 October the goal and the count judged different lists.

The walk-away window's goal is the tally. A separate model reads it after every turn and decides whether the build may stop.

The condition this window started with ran to 2,500 characters. Its done line names one thing a judge can read in the conversation: the tally's last line. The judge runs no commands, so the evidence has to be printed.

From the record of 30 October, a different build: its tally read ALL DONE, and the judge refused to stop six times running. One check had been handed to the next morning's run, the meter counted that as not blocking, and the judge read the plan and the tally and said the check was not proved. It was right. The meter's count and the judge's reading disagreed, and the build stopped only when the operator ended it.

Two weeks of fixes, each with its numbers, and this build's own runs.

• 15 October. The band and the panel are born: seven stages, and every plan must list its steps.
• 19 October. Proof lines and the frozen list. Measured first: 0 of 5 walk-away windows counted anything, and 195 of 533 claims were refused.
• 25 October. Twelve plan steps in a day: exact runs, background notices, no claim step, his words only his, one tally. The exact-run tests: 10 of 12 failed before the change and all passed after.
• 26 October. The walk-away's goal becomes the tally, and a real walk-away is judged end to end on the four failures I named.
• This build, to the cut. 11 proof runs: 11 passed and 0 failed.

Dates and before-and-after counts are from the meter's commit messages; the build's runs are from its session log, judged by the meter's own code.