Mechanism: Verify Loop

Part of Mechanisms of Vibe Coding · Core · Capstone: read after Goal, Constraints, Loop, Termination.

One-sentence definition

A verify loop is running an explicit check — test, lint, typecheck, or build — after each agent pass and feeding only the results back, so “done” means verified, not asserted.

The problem

The agent writes: “All tests should pass now.” You merge. CI fails — or you ship. The model optimizes for plausible completion; it does not run your suite unless you wire it to.

Without verification between passes, a loop is theatre: edits, victory declaration, truth discovered later. That is completion bias — trusting the last message as proof.

Symptoms:

• You trust “fixed” without re-running the failing command
• Agent describes passing tests but shows no output
• Each pass fixes syntax; regressions appear elsewhere
• CI is the first real verify

How Core fits together

The five Core mechanisms are one control stack:

1. Goal — symptom, boundary, verify line (what done means)
2. Constraints — what must not change while pursuing it
3. Loop — repeat until the goal holds
4. Termination condition — success signal + safety cap
5. Verify loop (this article) — raw evidence between every pass

Skip verify and the stack collapses into confident prose. Verify is how goals, loops, and termination stay honest.

How it works

Verification sits between work passes. Agent edits; you or a script runs the check; the next prompt has goal, constraints, and raw verify output — not your paraphrase of the error.

  Work pass (agent edits)
       │
       ▼
  Run verify command(s)
       │
       ├── pass ──► termination (success)
       │
       └── fail ──► paste raw output ──► next work pass

A good verify step is:

1. Named — exact command or test file
2. Fast enough to repeat — seconds to low minutes
3. Aligned with the goal — the symptom you care about, not an easy proxy
4. Raw — stderr/stdout in the chat; no agent-only summaries of failures it has not seen

Stack verifiers when needed (unit test + lint on touched files). Keep the list short so the loop stays runnable.

When to use it

• Inside every loop, every pass
• Before merge, commit, or PR — even after a one-shot agent session
• When the agent claims completion without command output
• As the success half of a termination condition

When not to use it

• No automated check yet — write a minimal repro test first; that becomes verify
• Full CI takes thirty minutes — local subset (pnpm test -- health.test.ts); full CI once at the end
• Non-trivial diff — run checks that catch regressions beyond touched lines

Failure modes

Trust the summary — Agent says pass; you skip the command. Fix: Run named verify every pass.

Wrong verifier — Lint clean; user bug remains. Fix: Tie to goal’s observable outcome.

Stale output — Old failure pasted on pass 3. Fix: Fresh verify after each edit.

Verify theater — Trivial check you did not ask for. Fix: Lock verify line in goal block.

Verify only at end — Five passes, one test run. Fix: Verify after each pass.

Minimal example

Context: Health endpoint loop — goal, constraints, and termination already defined.

Steps:

1. Session block:

   Verify (each pass): pnpm test -- health.test.ts
   Optional: pnpm run lint on src/health.ts src/bootstrap.ts
   On failure: paste full terminal output only; do not re-explain the goal.

1. Pass 1: Agent edits → you run test → copy full failure into chat.

1. Pass 2: Agent edits → verify again. If green, optional lint once before stop.

1. Do not accept “should work now” without exit code 0 on your machine.

Done when: Verify passes locally — not in the agent’s description.

Tool instances (optional deep-dive)

Portable idea above; tool-specific. Date: June 2026.

Cursor

• Agent proposes fixes; you run verify until hooks exist — clear trust boundary.
• Terminal: Run verify in integrated terminal; paste output in fenced blocks.
• Agent + commands: “Run pnpm test -- health.test.ts; stop if non-zero” beats open-ended “fix until done.”
• Hooks: Post-edit tests surface output automatically — see hooks.
• Rules: “Never claim tests pass without command output from this session.”

Other tools

Aider /run, Claude Code bash, CI on push — raw output drives the next pass.

Related mechanisms

• Loop — verify sits inside repetition
• Termination condition — verify pass often triggers stop
• Completion bias — what verify loop prevents

Try it yourself

Exercise: After the agent’s first fix attempt, do not reply until you run verify yourself. Paste only raw output if it fails.

Done when: You passed verify locally or pasted raw failure into pass 2 — without typing “are you sure?”

You have the Core stack. Next layers add safety, decomposition, and context — start with Scope fence when sessions sprawl.