Proving Lab → Notes

Six things went wrong building software with an AI assistant in one day

Most writing about AI-assisted development falls into two camps: it wrote my whole app, or it can't be trusted with anything. Both are useless if you actually have to ship something. So here is a log of one working day — six failures, what each one would have cost, and which check caught it. Two were caught by luck.

1 August 2026 · six commits, one browser extension, one website

A personal report. This is a private account by an individual, not a review by an institution and not a certified test. Measurements are single runs on specific inputs on a stated date; assessments are the author's own opinion. Statements about third-party products come exclusively from publicly declared data with the retrieval date given — nothing was decompiled, and no claim is made about any provider's intentions. The author develops Full Page PDF Snap, which appears in some of these measurements. Corrections: GitHub issues. Details in About & disclosure.

The work was ordinary: port a browser extension from Manifest V2 to V3, change its notification behaviour on Android, write three articles with measured data, and move the site to a new domain. Nothing exotic. That is the point — the failures below are the boring kind that happen on a normal day, not the dramatic kind that make headlines.

Case 1 · wrong by default

The assumption that would have doubled the work

The extension already had a Chrome MV3 branch. Porting it had required rewriting the whole image pipeline, because a Chrome service worker has no DOM: no document.createElement("canvas"), no Image, no blob URLs. Everything moved to OffscreenCanvas and createImageBitmap.

The obvious inference for the Firefox port was: same again. That inference is wrong. Firefox implements MV3 with an event page, not a service worker — the DOM stays available. The entire rewrite was unnecessary:

Firefox MV3 background: {'page': 'background.html'}
Chrome  MV3 background: {'service_worker': 'background.js'}

What remained were four renames: browser_actionaction, tabs.executeScriptscripting.executeScript, the menu context, and a dropped persistent key. Half a day of expected work collapsed into twenty minutes.

Caught by: reading the platform documentation instead of reasoning from the adjacent case. The failure mode here is not hallucination — it is a confident, plausible generalisation from one browser to another.

Case 2 · nearly published

A factual claim that was not just unsupported — it was false

One article compared PDF extensions by what they declare in their manifests. A draft paragraph read, in essence: with a server-side converter, the contents of a logged-in banking view leave your machine. Plausible. Consistent with the permissions. Entirely wrong.

A check against the provider's own listing turned up this, in their words:

“Please note that for security reasons, it is not possible to create PDFs from pages that require a login, such as your webmail inbox, online banking, shopping cart contents […]”

A server-side converter fetches the URL itself, as an anonymous visitor. It never has your session. The claim was not a small exaggeration — it described something the product cannot do.

The corrected version turned out to be the better argument anyway: server-side converters cannot reach pages behind a login, and those are exactly the documents people want to archive. The honest version was stronger than the wrong one.

Caught by: a review step that requires every factual statement about a third party to be traced to a source before publication. Not by re-reading, not by thinking harder. By looking it up.

Case 3 · one commit from public

An automation quietly wrote 22 local paths into a public file

A background hook aggregates file activity into CHANGELOG.md. Useful in a private repository. This one is public — it is the support URL on a store listing.

Staged for commit, the changelog contained 22 absolute paths of the form /home/<user>/repos/…, plus an issue file with complete local shell commands including Windows profile paths. Two build scripts had the Windows username hardcoded; one of them had already been public for days.

Nothing here was catastrophic — usernames and directory layouts, not credentials. But it is the kind of leak that accumulates silently, because the automation refills the file after every run. Fixing it once is not enough; it needed a rule.

Caught by: a habit of grepping the staged set for local paths before every commit. Had that step been skipped — and it easily could have been, on a day with six commits — it would have shipped.

Case 4 · true when written

Documentation that contradicted the software it documented

The README promised: “the finished PDF opens in your device's default PDF app.” Accurate when written. Two versions later the behaviour had deliberately changed — nothing opens by itself any more; a notification appears and tapping it shows the file in the browser's own viewer.

Worse, the same stale sentence sat in the German store listing, alongside a keyboard shortcut that belonged to a different browser. Anyone who tried it would conclude the extension was broken.

This is the failure mode nobody credits to AI assistance, and it is the most common one: code changes, prose does not. An assistant will happily write a correct changelog entry for a behaviour change and leave three other files describing the old behaviour, because nobody asked about those files.

Caught by: accident. The documentation was only checked because a separate question required reading it. There was no process that would have found it.

Case 5 · the word meant two things

“Saved to the vault” — but which vault?

An API token had to be stored. It went into a locally encrypted bundle, and the result was reported as saved to the vault. There are two stores in this setup: a local encrypted file for infrastructure lookups, and a self-hosted password manager that is the actual vault. The written rule says the local one must never hold credentials.

The token was in the wrong one. It surfaced only because the operator asked a direct question: can you confirm the data is in the vault on the server? The honest answer was no.

The cost of not asking would have been real: the provider displays such a token exactly once. On a rebuilt machine, the local bundle gone, it would have been unrecoverable.

Caught by: the operator, not the process. The fix was to make the rule executable — a command that writes to the server, reads back, compares, and refuses to report success without a match.

Case 6 · green means compiled

A release that passed every check and was never run

The MV3 port passed the linter with zero errors and zero notices. Syntax checks passed on every file in both branches. No V2 identifiers remained. It shipped.

It had never been executed. Not once. Static analysis confirms that the code is well-formed and uses the right API names — it says nothing about whether a capture completes, whether the event page survives a long scroll, whether the notification behaves as intended on a phone.

The project's own notes record two earlier bugs that survived exactly this kind of clean check: an ImageBitmap lacking naturalWidth, and a screenshot API silently rate-limited in one browser. Both are behaviour, not syntax. Both were found by running the thing.

Caught by: nothing. It is live and still untested. Written here because a log that only lists solved problems is marketing.

What actually caught things

Sorting the six by what found them is more instructive than the failures themselves.

  1. Checking against reality — three of six. Reading the platform docs, reading the provider's own listing, grepping the staged files. In each case the wrong version was internally consistent and sounded right. No amount of additional thinking would have surfaced it; only contact with an external source did.
  2. A human asking a pointed question — one of six. Not “is everything fine?” but “confirm it is in this specific place.” Vague check-ins produce vague reassurance. Specific questions produce answers.
  3. Luck — one of six. The stale documentation was found while looking for something else. That is not a process.
  4. Nothing — one of six. The untested release. Every automated check was green, and green meant compiles and lints, not works.

The pattern worth taking away

None of these were hallucinations in the popular sense — no invented API, no fabricated library, no citation to a paper that does not exist. Those are easy: they fail loudly the first time you run the code.

Every failure here was plausible. A reasonable generalisation from Chrome to Firefox. A claim consistent with the permissions. Documentation that used to be true. A word that means two things. Checks that were genuinely green. Plausible failures do not announce themselves, which is why review-by-reading finds so few of them — reading confirms internal consistency, and internal consistency is exactly what they have.

The practical conclusion is unglamorous. The checks that earn their keep are the ones that touch something outside the work: the vendor's own documentation, a grep across staged files, a read-back from the server, an actual run on real input. Everything else is a second opinion from the same source.