Human-in-the-Loop PR Review When AI Writes Faster Than You Read
Route review by risk when codegen outruns reading
An agent can produce a plausible 100-file pull request faster than a human can decide if it is correct, safe, and worth merging. Codegen moved the bottleneck from typing to justified confidence. When merge gates become rubber stamps, you have not accelerated delivery - you have relocated risk into production.
Here's what this piece covers: why infinite-diff review fails, what shipped in import e55425c as a first-class strategy (~780 lines), the six-point playbook, routing by blast radius, and three lead habits before your next agent-generated PR.
Part of Field Notes: agentic delivery. Governance frame: engineering-led, agent-accelerated.
1. The leadership problem
Review queues swell. Seniors burn out on nits while semantic risk hides in generated tests and mechanical files. Failure modes are predictable: too many files, hidden behavior changes, missing contract, architecture risk buried in a broad diff.
Wrong: "Review harder."
Right: "Six-point strategy when AI writes faster than you read - who merges, what agents may touch, what stays human-only."
The anti-pattern is infinite diff review - the same process for five lines and five hundred files.
2. What the git history shows
docs/human-in-loop-pr-review-strategy.md shipped in the initial framework import (e55425c, 2026-06-18) - review strategy as a first-class artifact, not an appendix. Paired skills: pr-builder packages context and evidence; pr-review grades against the implementation contract and repo standards.
| Input every review needs | Without it |
|---|---|
| Implementation contract | Opinion tennis |
| Evidence + reviewer guidance | Humans become linters |
| Preparatory audit | Merge theater |
I have watched a near-miss where a broad agent PR looked "green" - tests present, CI happy - until someone asked which approved spec defined the review shape. There was none. The correct move was stop, write the contract, split the stack - not "approve with comments."
3. What I built: six-point strategy
From the strategy doc's executive summary - the reusable checklist:
- Documented implementation contract before implementation - merged spec, approved feature/architecture slice, or a tracker record detailed enough to review against. "Build it from this prompt" is not a contract until captured and approved.
- Every PR ships evidence + reviewer guidance - what to trust, what to inspect, what was verified.
- Agents grade against contract and standards (
pr-review) - AI review is part of the process, not approval authority. - Decomposition gate for broad agent output - verify the implementation followed the approved review shape; otherwise return to spec update before human review.
- Route review depth by category, blast radius, and risk - not by raw file count alone.
- Humans approve, reject, merge - and own product risk.
def review_path(change):
if not change.implementation_contract:
return "block_for_contract"
if change.broad and not change.decomposition_ok:
return "split_or_respec"
if change.risk in ("security", "data", "migration"):
return "critical_tier"
if change.category == "bug" and change.tracker_sufficient:
return "fast_path"
return "standard_tier"Routing in practice: Bugs and simple tasks take a fast path when the tracker record is a sufficient contract. Features need the specification reviewed, approved, and merged before implementation shape is set. Stacks should create coherent review units - not turn one unreviewable PR into fifty tiny ones.
4. Named pattern: speed-mismatch review
Speed-mismatch review - bounded autonomy, draft PRs, and routing by risk - not the same process for 5 lines and 500 files.
| Path | When | Human focus |
|---|---|---|
| Fast | Narrow bug / simple task with sufficient tracker record | Behavior + evidence credibility |
| Standard | Feature slice with approved spec | Spec alignment + important diff |
| Critical | Security, data, migration, high blast radius | Full risk ownership before merge |
What broke: without preparatory audit, humans become linters. Without a spec, review becomes opinion tennis. Draft PRs are the default packaging surface so "ready for review" means something.
5. What I'd do as a lead
Three habits - steal one this month:
- Block human review without an implementation contract + evidence. No contract, no queue time.
- Cap PR size by policy; the agent must split or justify a large exception.
- Track review latency separately from merge latency - different bottlenecks, different fixes.
Further reading
- Previous: Engineering-led, agent-accelerated
- Next: Spread triage: 393 → 27
- Framework:
docs/human-in-loop-pr-review-strategy.md· skillspr-builder,pr-review
