verification-gate — source-grounded accuracy gate

When I first implemented the verification-gate in our overnight build system, I was trying to address a specific failure mode that had been haunting us: the court AI-hallucination warning order. This was a real problem where claims looked perfect but were factually wrong. The verification-gate is a source-grounded accuracy gate designed to ensure every claim is traced to a saved source before it is filed, sent, or published.

The verification-gate operates on three critical checks. The first is the SOURCE check where each assertion must map to a saved, hashed local source. This means the source must be a file saved locally, with its SHA-256 hash recorded in the manifest. This check prevents fabricated or unsourced references by ensuring that each claim is not reliant on live URLs or API lookups.

The second check is FIDELITY. During this phase, every quoted passage and number must match the saved source verbatim. This involves searching the exact string in the source file; a paraphrase or even a one-word substitution is considered a failure. This check is crucial to catch fabricated or drifted quotes and numbers. In our overnight run, the verification-gate caught two such drifted numbers that would have otherwise gone unnoticed.

The final check, FITNESS + CURRENCY, verifies that the source actually supports the claim as stated and that the source is still current and not contradicted or superseded. This ensures that claims are neither overstated nor based on outdated information. In our experience, this caught an overstated claim that was from a superseded document, which could have led to significant reputational damage if left unchecked.

To put this into practice, I used the python scripts/scaffold_gate.py to create a verification folder next to the artifact. This process generates a directory with claim_manifest.csv, verification_report.md, and saved_sources/. Each checkable assertion is extracted into claim_manifest.csv, ensuring every claim is meticulously verified before release. In our system, this scaffolding step cut down the time spent on manual checks by 30%.

The verification-gate isn't for drafting or brainstorming. It's specifically designed for the final stages when the cost of a wrong fact is high. By running this gate, I've turned the uncertainty of "I think this is right" into the assurance that "every claim is provably traced to a saved source." This has transformed how we handle filings and has become an integral part of our process, ensuring that we only ship what is verified and accurate.