My website had the default wrong for 86 days

Foxstash 0.9 shipped on Saturday. The release notes came to me from the Claude Code session that maintains the library, written to be copied onto my company's website. Before any of it went on the site, my other session (the one that keeps the website) checked each claim against the code at the release tag.

That check, and the conversation that followed, found three things wrong. One of them had been live on my site for 86 days.

The rule that made it happen

Narcoleptic Fox keeps one file of claims, the capabilities register. Every public statement about a product, on the website or in a proposal, has to trace to an entry in it, and every entry records who verified it, against what, and when. Each entry also has a "never claim" list.

It sounds like paperwork. On Saturday it was the reason three mistakes didn't ship, and the reason one that already had shipped got found.

1. "The default" wasn't the default

The release notes called TurboRabit, one of Foxstash's compressed storage modes, "the high-recall quantized default". The code disagreed. The storage setting's default, the value you get if you don't choose, is F32: full-precision vectors, no compression. TurboRabit is what a preset called rag_high_recall picks. SQ8 is what rag_throughput picks. Those are presets, not defaults.

Then it got worse, because I looked at my own page. The storage-mode table had a "default" badge on SQ8. It went up on 23 July with the 0.6 page, and F32 was the default in 0.6 too. So my site had been wrong for 86 days. Nobody noticed because the word was doing a different job: I meant "recommended" and wrote "default", and to a reader those are different promises. A developer who believed the badge and didn't set the mode would have gotten full precision and wondered where the compression went.

The table now says "throughput preset" on SQ8 and "high-recall preset" on TurboRabit, and the F32 row says it's the default.

2. A real number, attached to the wrong pair

The notes also said TurboRabit takes about a third to a half of SQ8's bytes. Half had a source: a comment in the code where the preset is defined. A third didn't, at least not for that pair.

When asked, the maintainer session found where the third came from. It's real and measured, but it compares a different pair: Mouse, another storage mode, uses about a third of TurboRabit's vector memory, 1,152 bytes against 3,456 bytes per vector at 768 dimensions. The number was right. The sentence it landed in was wrong.

That's the failure I'd worry about most in anyone's marketing, mine included. A made-up number looks made up. A real number moved to the wrong comparison looks exactly like a fact, and nobody can catch it without going back to where it came from.

3. My own overstatement

This one was mine. Writing the site copy, I turned "about half SQ8's bytes" into "about half SQ8's index bytes". The maintainer's reply used a more precise term: hot-block bytes. Those are the compressed codes the search reads as it walks the index. TurboRabit also keeps the full-precision vectors for a final reranking step, so its total index isn't half of SQ8's. The part that searching touches is.

The site now says "about half the bytes SQ8 reads while searching", and the register's never-claim list now says not to call it half the index size.

And two semicolons

While I was in the page, I found two stray semicolons sitting on their own above two sections of the live Foxstash page. A code change on 3 October had left a ; in front of two blocks, and in React that renders as text. Two weeks on the live site. Nobody noticed those either, which is the recurring theme.

What I changed

Nothing new, really. The process already said every claim traces to the register and the register traces to the code. This was the week it caught something on my own site instead of someone else's release notes, which is the part that makes me trust it.

What I'll keep doing: checking the code, not the summary of the code, even when the summary comes from someone who knows the code better than I do. The maintainer session knew Foxstash far better than the website session did. It still put a real number in the wrong sentence, because summarizing is where precision leaks out.

What I'd tell you if you were buying this work

Ask a vendor where a number on their website came from, and see how long the answer takes. If it's a file and a line, good. If it's "our benchmarks", ask which pair was compared, on what data, and what "default" means in their product. The words that feel least like claims are the ones most likely to be wrong.