Tooling · Code Review · CI

Checks that can actually fail

A green check is a claim, and most of this writing is about the times the claim was false. Code review that finds things, gates that hold, coverage that means something, and the specific ways a pipeline can report health it never measured.

Code Review CI Gates Mutation Testing False Greens Coverage Static Analysis Local-First Tooling Evidence

A check that cannot fail is not a check — it is a status light wired to the on switch.

The recurring discovery in this work is that tooling fails quietly and looks fine doing it. A scanner that matched nothing because the pattern was wrong. A gate scoped so narrowly it never saw the job it existed for. A test whose assertion was satisfied by its own success message. The writing here is mostly about finding those, and about the habit that finds them: break the thing on purpose and watch what goes red.

The frame

Four ways tooling lies

Read False Green →

Configured but inert

Wired up, present in the config, and never actually executed. Indistinguishable from working on every dashboard that exists.

Scoped too narrowly

The check is real and runs against a fraction of what it names. The pass is honest about what it read and silent about what it did not.

Asserting its own message

The guard matches the success string beside the thing it checks, so it stays green while the check under it is reduced to a presence test.

Could not look, reported nothing

An absent tool prints nothing, and nothing is exactly what a real negative looks like. Exit status is the difference.

Flagships

What a review is worth, and where it should run

Study · CI · Evidence

False Green

A pipeline reporting health it never checked, taken apart: what the checks claimed, what they actually read, and the arithmetic that showed the gap. Includes the bake-off record behind it as attached technical detail.

CIEvidenceWPR-MAX-015

Read the study

Study · Code Review · Local-First

Local-First Code Review

Review that runs on your own hardware, with the guard dog and the ledger: what it catches that a hosted reviewer does not, what it costs, and why the record of a review matters as much as the review.

Code reviewLocal-firstWPR-MAX-017

Read the study

Why it fits

Why this writing exists

Every finding is a real one

The examples are defects that shipped in these repositories, with the guard that now stops them and the mutation that proves the guard fails when it should.

Break it on purpose

Fault injection is the method throughout: a guard nobody has deliberately broken is unverified, and most of the interesting findings came from breaking an old one.

Cost is part of correctness

A gate that is right and takes an hour gets skipped. The writing treats runner minutes and review quota as first-class constraints, not footnotes.

Tooling for regulated work

Review and CI here have to produce an audit trail, not just a verdict, which changes what a good tool looks like.

Field notes

Developer tooling writing

See all →

See all developer tooling writing →

Engage

Pipeline green and you are not sure it should be?

A review of what your checks actually assert, where the gates are scoped, and what a deliberate break would reveal — plus what it is costing to run.

Start a conversation Read the background

This site: HTTPS only · no third-party trackers · no analytics cookies · security.txt