Part of Mechanisms of Vibe Coding

Mechanism: Best-of-N

One-sentence definition

Best-of-N is running N independent agent attempts on the same goal — separate branches, chats, or models — then choosing the winner by verify, diff size, and checkpoint, not by which reply sounded best.

The problem

One agent path gets stuck. You keep nudging. A fresh approach might have worked in parallel, but you never ran it. Or you run two chats, pick the confident summary, and ship the one that failed tests.

Best-of-N is structured competition: same contract, different executions, evidence-based pick.

Symptoms:

• Long single-thread struggle while alternatives untried
• Picking the first green build without comparing diffs
• Identical prompts on identical branches — wasted duplicate
• N attempts entangled on one branch

How it works

  Same goal + constraints + verify
       │
       ├── Attempt 1 (branch/chat A)
       ├── Attempt 2 (branch/chat B)
       └── Attempt N …
       │
       ▼
  Score: tests, diff scope, readability, criteria
       │
       ▼
  Merge one; discard others

Keep N small (2–3). Each attempt needs branch-per-experiment isolation.

Scoring rubric (example):

• Verify pass (required)
• Files touched within constraints
• Smallest diff that passes
• Human checkpoint preference

When to use it

• Hard bugs after stuck detection
• Design forks with no clear winner upfront
• High-value one-shot tasks worth extra compute
• Escalation Level 4 with parallel models

When not to use it

• Trivial fix — N=1 is enough
• Attempts that only differ by random temperature with same stuck prompt — change strategy per attempt
• N so large you cannot review all diffs

Failure modes

Same prompt, same trap — Three identical failures. Fix: Vary slice, examples, or mode per attempt.

Winner by prose — Best explanation, failing tests. Fix: Verify pass is mandatory gate.

Unmerged loser pollution — Loser branch accidentally merged. Fix: One PR per attempt; close losers.

Cost spiral — N=10 on huge repos. Fix: Cap N; narrow scope fence.

Minimal example

Context: Uptime still 0 after single-thread fixes.

Attempts:

• A: experiment/uptime-process — use process.uptime()
• B: experiment/uptime-started-at — startedAt in bootstrap

Same goal block on both. Run tests on both. Merge smaller passing diff; delete other branch.

Done when: Winner chosen with written reason tied to verify + diff, not preference alone.

Tool instances (optional deep-dive)

Portable idea above; this section is tool-specific. Date: June 2026.

Cursor

• Cursor best-of-N runners / parallel agents on worktrees when available.
• Manual: two Composer tabs, two branches, same prompt template.
• Document scores in PR comment before merge.

Other tools

Multi-sample codegen with test harness; ensemble only if you can review all outputs.

Related mechanisms

• Branch-per-experiment — isolation per attempt
• Escalation — parallel attempts as escalation step
• Checkpoint — human tie-breaker

Try it yourself

Exercise: Run N=2 on a small task with different strategies (not duplicate prompts). Score both before picking.

Done when: You wrote one sentence: winner because [verify + diff reason].