Verification on film
2026-09-12 Draft
An agent tells you it works. It might. The question is what you would accept as proof from a stranger, and then asking the agent for exactly that.
Belief is not evidence
When a person hands you a feature, you trust them a little because you know them. An agent you do not know, and it does not know itself. It will report success in the same voice whether it ran the test or imagined running it. So I stopped asking whether it worked and started asking for the artifact. A file on disk, with a time on it, that I can open without the agent in the room.
That is the whole idea. Stories become videos that are corroborated by physical artifacts on disk. Everything else in the harness follows from that sentence.
Behaviours
The unit is a behaviour, not a feature and not a test. A behaviour is a thing a person can do with the product and what they see when they do it. Each one is a short document: what it promises, what it guarantees, the exact sentences a person reads, what it must not do, and then the steps. A step is a gesture and an expectation. Open the home page. Press the button with the field empty. The line under the field reads this sentence, in red, and the button does not move.
The behaviours live in the repository next to the code, as plain text, and an index is built from them. They are the contract. When I change my mind about how something should behave, I change the document, and everything downstream has to follow. Code is prose, and this is the prose that says what the code is for.
Stories
A story is a script that performs a behaviour's steps, in order, against the real site in a real browser. It does what the step says and checks what the step expects. It does not check anything the contract did not ask for, and it cannot skip a step, because the step's words are copied into the story verbatim and the harness refuses a story whose words drift from the index.
The stories are written by agents, from the contract, and they are the first place a bad contract shows up. If a step cannot be performed, the step was wrong. If a step passes but the film looks wrong, the expectation was missing something. Either way the fix is a sentence in a document, not a conversation.
Films
Every run of a story is filmed. The film is a real screen recording of a real browser doing the steps, with the pointer visible, at the speed a person would do it. A story that navigates by typing a URL where a person would have clicked a link gets rewritten, because the film has to show the cause of every change on screen. Beside the film is a take: which behaviour, which build, when, pass or fail, where it failed, and a pixel diff of the parts of the page that were supposed to hold still.
The films sit on a page I can open from my phone. One row per behaviour, the newest take on top, green or red. I do not read the story. I watch the film the way I would watch someone use the product, and I stop where anything looks wrong. When something moves that should not, or a sentence is not the one I wrote, that is a finding, and it goes on the row with the second in the film where I saw it.
The tester that writes no files
A film proves the story did what it said. It does not prove the product is good. For that there is a second agent, and its job is to be a hostile stranger. It opens the thing cold, with no access to the code and no idea what was intended. It pokes at it the way an annoyed person would. It reports what it can reproduce, with a screenshot or the exact text it saw, and it is not allowed to guess at a cause. It writes nothing but its report and its screenshots.
Two rules make this work. It reads no code, because an agent that has read the code will test what the code does instead of what a person expects. And it names no cause, because the moment it says why, the agent that built the thing starts arguing with the why instead of fixing the what. Its report goes on the row. The builder answers each line: fixed, or by design with a reason. I read both.
Evidence over belief
Put together, a row in verification carries a film, a take, a tester's report with the builder's answers, and the commit. I open the film first. If the film is wrong, nothing else matters. If the film is right, I read the tester. If the tester found nothing a person would call broken, I open the thing myself, once, cold, and pass it or send it back.
None of this requires me to believe the agent. That is the point. The agent's job is to produce the evidence a stranger would accept, and my job is to be that stranger. When the evidence is there, the work is done and I know it. When it is not, the work is not done, however confident the report sounded.
What it costs
Filming is slow. A two-minute story takes two minutes to film, and a full pass over the site takes a few. The pixel diffs flag motion I did not mean, and sometimes motion I did. The tester finds things that are by design and I have to say so, on the record, every time. All of it is slower than trusting the agent.
It is still the fastest way I have found to run a product with agents, because it removes the one step that does not scale, which is me reading everything. I do not read everything. I watch films, I read one hostile report per row, and I open the thing cold. Everything else is a document an agent can follow without me.