A maintenance page is not a locked door

The day was supposed to be paperwork.

I'd just approved Narcoleptic Fox's Quality Manual, the document that says how my company builds and ships software, and set up a yearly internal audit run by someone outside engineering. The auditor isn't a developer, so she needed evidence she could check without knowing what a pull request is. I had Claude Code write a script that collects a year of it from GitHub: merged pull requests and their checks, commits that reached main without one, releases, and every security alert with how long it took to fix.

The first run was meant to prove the script worked. It proved the manual didn't describe the company yet.

First, three of the numbers were mine

Before any finding was real, the tool had to stop being wrong.

• It never finished. It asked GitHub, one call per commit, whether a pull request stood behind it. An hour in, one transient error killed the run. The rewrite asks about a hundred commits per query: 129 seconds for 21 repositories. I didn't trust it until it agreed with the old method on ten commits, five flagged and five passed.
• 160 overdue security alerts was really 48. Alerts had been switched on for every repository two days earlier, so most were two days old, and the script counted every open alert as a miss. A high-severity alert has 30 days. It only misses once it's older than that. Same data, a third of the number, and the whole difference was a rule I had invented.
• A check failed for the wrong reason. Two pages looked broken because a label was missing from the page text. The labels were styled in uppercase, and the browser returns text as displayed. That was a broken check, not a broken page.

The door

With the numbers honest, the summary said a dozen critical alerts were about to come due. Some belonged to a product I'd paused. Its site showed a friendly "we'll be back soon" page. Everyone, me included, had filed it as off.

The response headers disagreed. They showed the application itself answering every request and rewriting each page to the maintenance screen. One of the open alerts was a critical, unauthenticated remote code execution in a part of the web framework that the maintenance rewrite didn't cover. I asked for one ordinary URL on that path. It came back 200, served by a framework version inside the vulnerable range.

It was patched and verified closed the same day.

What the healthy path looked like while it was open: a maintenance page, a repository nobody had touched in days, and a status in my head that said "paused". None of those is a control.

The checks that never ran

The fix itself was quick. The pipeline wasn't. Every check failed in two seconds with no steps, because the account's hosted CI minutes had been used up 24 days earlier. Everything else kept passing because it ran on my own build machines. This product's CI had failed on every push since, silently, and nobody noticed because nobody pushed. The first push that mattered was the security fix. Two lines of workflow later, it ran on my own runner and passed, and 45 alerts closed in one rescan.

And the habit it caught

The audit had one more finding, and it was about how the work gets done: 22 code changes pushed straight to main after branch protection, most of them from AI coding sessions working under an older "small changes go straight in" habit. Fine for a personal notes repo. Not fine under a quality manual that says code goes through a pull request with checks.

What changed

The manual went from 1.0 to 1.1 the same day:

• Documentation-only changes may go straight to main. Anything that's built or deployed goes through a pull request. A direct code commit is an emergency and gets an issue explaining it.
• A paused product carries a paused label and fixes its critical and high alerts before it's deployed or worked on again. And, in so many words: a deployment in maintenance mode is still deployed.
• The rules went into the instructions my AI sessions read at the start of every session, because the habit they replaced lived there too.

If you're buying software

Ask what's still running that everyone thinks is switched off. Then ask how the team would know. A maintenance page, a quiet repository and a confident status update all look exactly like a locked door, right up until someone checks the headers.