Ceiling Reported as Floor

Context

A tool reports findings about a codebase — static analysis, linters, dependency scanners, coverage, security scanners, AI reviewers. It knows what the code could do better. It does not know what is at stake here. So it reports the distance from the possible and presents it as the distance from the necessary.

Every unencrypted field, uncovered branch, missing rate limit and unpinned dependency arrives at the same severity, because the instrument has no way to know that this service is an internal timesheet app behind SSO with forty users.

The team mutes it. Then it misses the one that mattered.

Cost type

Behavioral, borne by users, arriving late. The proximate cost is cognitive — alert fatigue — but the actual damage is the real finding that was filtered out with the noise, and it lands on whoever the software fails.

What goes wrong

• A finding that is genuinely urgent in one venue and irrelevant in another is reported identically in both. The reader learns that severity carries no information, and stops reading severity.
• The team develops muting as a practice: .eslintignore entries nobody can explain, suppression attributes with no comment, a scanner whose report is forwarded and never opened.
• Because the suppressions carry no reasons, they cannot go stale. The app is exposed publicly two years later and every dismissal that depended on it being internal remains silently in force.
• Adoption fails and gets attributed to culture. “The team won’t engage with quality tooling.” The team is behaving rationally toward an instrument that cannot distinguish a catastrophe from a shrug.
• The next tool arrives promising better findings. It has the same missing input, and the cycle repeats — this being the reason so many analysis products are bought, deployed and abandoned in a two-year loop.

Prospective indicators

• A report can be generated without any input describing who uses the system, what it is exposed to, or what happens when it is wrong.
• Severity is a property of the rule rather than of the finding in this place.
• The rollout plan includes a triage phase to work through the initial backlog. That phase is where the tool dies.
• Nobody can say, in advance, what a correct dismissal of a given finding would look like.

Retrospective indicators

• Suppression comments with no reason attached, or with reasons that are not claims about the world (// false positive).
• A findings count that only ever goes up, and a dashboard nobody has opened in a quarter.
• Developers who can recite which tools to ignore.
• A post-incident review discovers the failure was reported, months earlier, in a list of four hundred other things.
• Two services with wildly different exposure receive the same report and the same treatment.

Prevention

• Supply the floor. Six sentences per repository: who uses this, what it can reach, what happens if it is wrong, what it is exposed to. Not derivable from source, stable for years, and it scopes every finding produced against that repo forever. A repo carrying nothing but that is already better instrumented than one with three scanners and no venue.
• Make correct dismissal a design requirement, not an afterthought. A finding is only worth emitting if a knowledgeable person can wave it off in one sentence — and that requires the finding to carry its own scope condition. “MD5 is retired where collisions are exploitable; this is a cache key” is dismissible. “MD5 is insecure” is a taboo with a checkbox.
• Record dismissals as data, with reopening conditions. “Internal-only, behind the IdP” is falsifiable and goes stale observably the day that changes. Suppression without a reason is how a correct dismissal becomes a permanent lie.
• Rank by floor gap, never by ceiling gap. Attainment below what is required is urgent. Attainment below what is possible but above what is required is a non-issue, and reporting it at all is a tax.
• Watch the third number. A tool with a few false positives that are all correctly dismissible survives contact with a team. A tool with perfect precision whose findings cannot be reasoned about does not.

When this isn’t a failure mode

• The floor is genuinely uniform. Some rules are near-absolute across every venue an organization has — no credentials in source, no unparameterized SQL against user input. Reporting those at fixed severity is correct, and the list is much shorter than any default ruleset.
• Regulated contexts where the ceiling is the floor. If an auditor requires the control regardless of exposure, distance-from-possible is the right measure and dismissal is not available.
• Exploratory use. Pointing a scanner at unfamiliar code to learn its shape is a legitimate one-off. The failure mode is about standing reporting into a team’s workflow.
• The team has the venue in their heads and the tool is only a prompt. Works at small scale with stable staff; fails silently the moment someone new arrives, because the dismissals were never written down.

Note

The missing input is the whole entry. This is not a user-experience problem and it is not solved by better ranking heuristics, because the information required is not in the artifact being analyzed and never was.

Two faces of the same venue do opposite work: what the venue makes possible is a ceiling on attainment; what the venue makes necessary is a floor. A finding is the gap between them. Instruments that know only one number produce garbage in a predictable direction — always toward more findings, never toward better ones.

This is also why the practice of “start with the default ruleset and tune it down” produces the outcome it does. Tuning down is the team supplying the floor, one suppression at a time, without writing any of it down.

Related: Verification Asymmetry is the mirror image — deferring to a confident report without checking. Here the report is correct and still worthless, because correctness at the wrong altitude is indistinguishable from noise.