Field Notes

The Afternoon We Shipped Three Wrong Metrics

Bad numbers erode trust faster than no numbers

Iheatu Chris Wogu

Iheatu Chris Wogu

— 3 min read
Share

On June 1 we shipped three person-summary metrics that managers would have trusted in a staffing conversation. By 10:52 the same day, three commits corrected them: merges counted only PRs created in the window, open snapshots used the wrong catalog, and cycle time included still-open PRs. Bad numbers erode trust faster than no numbers. Here's the reconciliation identity we use now - and the three checks I'd run before putting any metrics in a staff meeting.

The morning CLI refactor and pipeline work mattered. The afternoon fixes mattered more.

Here's what this piece covers: the leadership cost of trusted fiction, the three wrong definitions, same-day corrections, the reconciliation identity, and three lead habits.

Part of Field Notes: agentic delivery.

1. The leadership problem

Managers staff from merged and open counts. If those columns are undefined - or defined wrong - you are not measuring engineering; you are laundering fiction through Excel.

Wrong: "We fixed some bugs in the afternoon."
Right: "Between 10:20 and 10:52 we corrected prs_merged, prs_open, and merge-cycle hours before anyone built a narrative on them."

The anti-pattern is metrics that lie - undefined columns, open PRs counted as cycle time, the wrong catalog for "open."

2. What the git history shows

Verified timestamps (EDT), same morning cluster as the productization:

Time Commit Change
10:20 dd6c672 Per-person min/max/avg hours open → merge
10:46 0f69edb Fix prs_open snapshot; exclude open PRs from merge cycle time
10:52 923250f Fix prs_merged - all merges in window, not only PRs created in May
sequenceDiagram participant AM as Morning ship participant Fix as Afternoon fixes participant Staff as Staff meeting risk AM->>Staff: Confident columns in Excel Fix->>AM: 10:20 cycle hours Fix->>AM: 10:46 open + cycle filter Fix->>AM: 10:52 merged window semantics Note over Staff: Narrative never hardens on fiction

3. The three bugs

prs_merged - counted merges only for PRs created in the window. Carry-over merges (opened earlier, merged in May) vanished. Staffing looked quieter than reality.

prs_open - snapshot at window end used the wrong catalog. Open queue overstated or understated depending on who had activity in-window.

Merge cycle hours - included still-open PRs. Averages inflated; "slow" looked systemic when the denominator was wrong.

Wrong definition before the fix (sketch):

# WRONG - merges only for PRs created in the window
def prs_merged_wrong(rows, start, end):
    return sum(
        1 for r in rows
        if start <= r.created < end and start <= r.merged < end
    )

# RIGHT - any merge whose merged_at falls in [start, end)
def prs_merged(rows, start, end):
    return sum(1 for r in rows if r.merged and start <= r.merged < end)

Phase 2 detail rows and Phase 3b catalogs are different sources for different columns - mixing them without a dictionary is how silent bugs ship. No automated tests that day; bugs found by manual reconciliation and later agent audit (article 11).

4. Named pattern: same-day metric correction

Same-day metric correction - ship, reconcile, fix before the narrative hardens.

The reconciliation identity

For window-opened PRs only, at window end:

authored ≈ merged + open + closed_unmerged

Verified example from the metrics review (monika-sbd, May 2026):

Repo authored merged + open + closed_um Match
global-services 2 0 + 1 + 1 = 2 Yes
global-user-services 3 3 + 0 + 0 = 3 Yes
polaris-turbo 10 9 + 0 + 1 = 10 Yes

Confidence: git timestamps verified; reconciliation rows verified against May 2026 production caches in docs/metrics-review-findings.md. Do not apply the identity to totals that include carry-over prs_merged / prs_open without scoping to window-opened PRs (README as contract).

5. What I'd do as a lead

  1. Demand a one-page metrics dictionary before any dashboard ships. Column name ≠ definition.
  2. Reconcile one person manually each month. Spot-check is not optional - it is how monika-sbd caught fiction.
  3. Never compare month-over-month until window semantics are frozen. Half-open windows and timezone belong in the contract first.

Further reading

Iheatu Chris Wogu

About the author

Iheatu Chris Wogu is an engineering manager focused on launch programs, honest engineering metrics, and engineering-led, agent-accelerated delivery - researching where work queues up when agents widen dev output, then building proof, gates, and measurement that scale.

Field notes live on First Context. Values treatise: engineering manager values.

LinkedIn: linkedin.com/in/iheatu-chris-wogu

Read more