Definition of Done — the run-truth ladder

On the second of September every call to my AI receptionist returned a 500. It ran that way for hours. The board was green the entire time, and I found out by calling my own phone line.

The cause was ordinary: a migration locked a table to the service role while the application was still writing to it with the anon key. The part worth writing down is that the check which would have caught it in sixty seconds already existed. It was scheduled to run the following Monday.

So the system was not missing a test. It was missing a definition. Nobody had said what "done" meant for that change, so "the migration applied cleanly" was allowed to stand in for "the thing still works", and those are not the same claim.

Four rungs, and you have to name yours

Every feature sits on exactly one of these, and the only honest report says which.

BUILT. The code exists and compiles on the trunk. The build exits zero. This is the rung a green local build proves, and it is the one people mean when they say "it's done." It is worth almost nothing on its own.

DEPLOYED. The artifact is live somewhere that matters. It is reachable at a real URL by someone who is not you.

WORKS. The deployed thing does its job, proven by one real call end to end, with the downstream effect asserted. Not the request returning. The row appearing, the email arriving, the number changing.

FED. Real, non-synthetic data has flowed through it. A stranger used it, or production traffic did. Almost nothing reaches this rung, and pretending otherwise is the most common lie in engineering status reports, mine included.

The discipline is not the ladder. Anyone can draw a ladder. The discipline is refusing to report a rung you did not prove, and saying "I could not check" when you could not check.

How the proof gets faked

The ladder tells you which rung to claim. This is the list of ways I have claimed one and been wrong. Every item cost me something real.

A status code is not content. A 200 from a modern framework route is the shell, not the page. A soft 404 answers 200 all day. Read the body for the text you expect before you write the word verified. I hit this again the day I wrote this: I withdrew a post, requested the URL, got a 200, and nearly reported a broken gate. The 200 was a cached response. The body was the fallback page and a fresh request returned a clean 404. The status told me the opposite of the truth in both directions.

A configured variable is not a configured value. Every console lists the name. It will happily show you a variable whose value is an empty string. In August one of mine was reported as configured, was empty, and the code's own length guard failed silently into a log nobody read. If a rail is required, prove one real call through it.

A piped exit code is not the command's exit code. A shell pipeline returns the status of the last stage, so npx vitest run | tail -18 exits zero on a suite that failed, and the background runner then reports success for a red board. This is the quietest one on the list because success and failure look identical. Redirect to a file, read the exit code out of the file, and read the tallies, not just the status.

A local build is not the CI build. CI turns warnings into errors, so a build that passes on your machine fails the gate on a lint warning you never saw. Worse: a test suite that aborts during collection reports one tidy import error and runs zero tests. A repository can sit completely untested behind a failure that reads like a single missing dependency, which is how one of mine stayed red and uncollected for hours.

A live process is not a finished job. A running PID, an open window, a spinner. None of them say the work completed. Poll for the artifact: the file, the commit, the log line, the row.

Re-reading the code is not running the gate. If the typecheck was blocked, name the command that was blocked and mark the change unverified. Never let "I read it again and it looks right" stand in for a compiler.

A short SHA is not a SHA. Watch CI on the exact revision, never a guessed prefix.

The pattern underneath

Six of those seven share a shape: a cheap signal sat next to an expensive one and I read the cheap one. The status instead of the body. The name instead of the value. The pipeline's exit instead of the command's. Each substitution is defensible in the moment and each one converts "I know" into "I assume" without changing how the sentence sounds.

That last part is why this needs mechanism rather than intention. A wrong check and a right check produce the same confident summary. You cannot feel the difference, which means you cannot resolve to be more careful about it. You can only make the honest version cheaper to produce than the dishonest one.

Why this matters if you build with AI agents

An agent will tell you the work is done, and it will be persuasive, because the same process that wrote the bug writes the summary of the bug. Asking it whether it is finished is asking the least reliable witness in the room.

The fix is not distrust, which is slow and makes the tool useless. The fix is to hand it a definition it can check itself against. Which rung, what proved it, what command produced the proof. An agent given that structure will police its own claims better than most humans do, because it has no ego about the answer. An agent not given it will produce fluent, structured, completely confident descriptions of work that did not happen.

The cheapest sentence in engineering is "I could not check." It costs one round trip. The retraction costs a great deal more, and the outage costs whatever your customers think it costs, which on the second of September was one phone call I made to myself.