<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[foxhole]]></title><description><![CDATA[Engineering notes from Narcoleptic Fox: Rust, Go and .NET builds, the bugs that fooled me, and the numbers that showed it.]]></description><link>https://blog.narcolepticfox.com</link><image><url>https://cdn.hashnode.com/uploads/logos/69dbec87aadf1107e21d2c6e/28778894-e0dc-45b2-ae23-df1ed7d498b4.png</url><title>foxhole</title><link>https://blog.narcolepticfox.com</link></image><generator>RSS for Node</generator><lastBuildDate>Sun, 18 Oct 2026 21:06:53 GMT</lastBuildDate><atom:link href="https://blog.narcolepticfox.com/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[The delete that didn't delete]]></title><description><![CDATA[Foxstash shipped three releases this week: 0.7 on Tuesday, 0.8 on Wednesday, 0.9 on Saturday. Foxstash
is my Rust retrieval library, the part of a search or AI system that finds the right passages,
ru]]></description><link>https://blog.narcolepticfox.com/the-delete-that-didnt-delete</link><guid isPermaLink="true">https://blog.narcolepticfox.com/the-delete-that-didnt-delete</guid><category><![CDATA[Rust]]></category><category><![CDATA[Databases]]></category><category><![CDATA[Security]]></category><category><![CDATA[Testing]]></category><category><![CDATA[open source]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Sun, 18 Oct 2026 17:57:07 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/gql/69dbec87aadf1107e21d2c6e/37739431-c247-48d8-92a8-ca40d97c93c9.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Foxstash shipped three releases this week: 0.7 on Tuesday, 0.8 on Wednesday, 0.9 on Saturday. Foxstash
is my Rust retrieval library, the part of a search or AI system that finds the right passages,
running inside your application with no server.</p>
<p>A changelog would tell you what's new. This is the other list: four things that were wrong, and
the specific moment each one got caught. Three of the four came to light because of the same
reviewer, and it wasn't me.</p>
<p>Some context first, because it matters to every story. I maintain Foxstash, and I'm also building
an internal meeting recorder that uses it to search transcripts. Claude Code sessions help me with
both, one per project, on separate machines. This week those sessions messaged each other directly,
worked each problem through between them, and brought it to my attention with the evidence
attached. So the library has a customer who reads every change, asks pointed questions, and has to
live with the answers. That turned out to be the most valuable reviewer of the week.</p>
<h2>1. The 80-second shortcut</h2>
<p>0.8 added encryption at rest: you supply a cipher and a key, and Foxstash encrypts the write-ahead
log, the checkpoints and the index snapshots. Foxstash never holds the key.</p>
<p>The index snapshot was the awkward one. It holds the vectors, so it has to be covered, and wiring
encryption into the index format is real work. The shortcut was obvious: when encryption is on,
don't save the snapshot at all, and rebuild the index from the encrypted checkpoint every time the
store opens. No plaintext vectors on disk. Much less code.</p>
<p>I asked for a measurement before anyone committed to it. Rebuilding at 768 dimensions took 23
seconds at 45,000 documents, 42 seconds at 90,000, and 72 seconds at 180,000. Loading the snapshot
took about one second. That's 77 to 125 times slower, and at the meeting recorder's scale of about
200,000 documents it puts an 80-second wait in front of a desktop app every time it opens. So the snapshot gets encrypted, not
dropped.</p>
<p>Then the design went to the meeting recorder's session with one question: will this cost you too
much? It came back with a design review instead. The first draft authenticated each encrypted file
but didn't bind it to anything, so an old snapshot, or one copied from a different store, would
have opened cleanly. That's a rollback attack, and nobody had thought about it. Now every file's
authenticated data names its role, the store it belongs to, and its generation. A wrong key, a
tampered file or a rolled-back file is a hard error. It never quietly falls back to a rebuild,
because a slow success that hides tampering is worse than a loud failure.</p>
<h2>2. The free lunch that wasn't</h2>
<p>EmbeddingGemma 2 lets you keep only the first 512, 256 or 128 of its 768 dimensions. Smaller
vectors, smaller index. The question was what that costs in search quality.</p>
<p>The first benchmark said nothing. Recall at 99.9%, at every size, in every storage mode. It got
written up as a free storage win and recommended.</p>
<p>The meeting recorder's session read the table and pointed out the flaw. Each truncated run had been
scored against exact search <em>at the same truncated size</em>. So the 128-dimension index was being asked
"do you find what a perfect 128-dimension search would find?" and the answer was yes. But the real
question is whether it finds what the full 768-dimension embedding would have found, and the
yardstick couldn't measure that, because it had shrunk along with the thing it was measuring.</p>
<p>Scored against the full embedding: 91.7% at 512 dimensions, 81.9% at 256, 67.1% at 128. A third of
the true neighbours gone at the smallest size. That's a real trade of quality for memory, not a free
lunch, and it's how the numbers appear on the Foxstash page today.</p>
<h2>3. The delete that didn't delete</h2>
<p>This is the one that gave the week its name.</p>
<p>The meeting recorder needs a retention feature: when a meeting is deleted, its text and its vectors
have to leave the disk. Its session asked a question with an assumption folded into it: "I've
written my test assuming delete, then compact, and the content is gone from every file. Is that
right?"</p>
<p>It wasn't. Delete in Foxstash is a soft delete: it writes a "removed" record and hides the document
from search. The original text is still in the log. If the document was checkpointed, it's in the
checkpoint. And it's in the index snapshot, which stores the passage text alongside the vector.</p>
<p>Compact is the cleanup. It rebuilds from the documents that are still live, writes a clean
checkpoint and snapshot, and prunes old ones. But it keeps the previous checkpoint as a fallback,
because the setting that controls that defaults to 2. So after one compact, the checkpoint from
before the delete is still on disk, with the deleted meeting inside it, in plain text.</p>
<p>The part worth sitting with is that the test was green. The test inserted a document, deleted it
and compacted, without ever making a checkpoint in between. So the document only ever lived in the
log, and one compact always clears the log. The test passed because it skipped the one step that
causes the failure. In real use, a meeting that's been around for a while has been checkpointed.</p>
<p>0.9 adds <code>purge</code>. It deletes, then compacts as many times as the retention setting requires, and
returns a report proving that nothing older than the purge survives. If anything does, that's an
error, never a partial success. It comes with a test that proves the old way <em>does</em> leave the data
behind, so the guarantee isn't vacuous: a test that can't fail proves nothing.</p>
<p>And one honest limit, written into the docs and the website: purge deletes files. It doesn't
overwrite the disk blocks underneath them, which is normal for any database. If deleted data must be
unrecoverable, pair purge with encryption at rest and rotate the key. I won't claim more than that.</p>
<h2>4. The position that wasn't an identity</h2>
<p>A collection that reranks its results was holding every full-precision vector twice: once in the
index and once in the document store. Holding it once cut the index's footprint from 74.3 MiB to
15.8 MiB at 20,000 documents of 768 dimensions. Results are identical.</p>
<p>The near-miss was in the obvious implementation: when the index needs a vector, ask the document
store for document number N, where N is the index's node number. It reads naturally. It's also
wrong, because the index reorders its nodes when it builds, for speed. Node 5 isn't the fifth
document. Nothing had ever depended on the two matching, which is exactly why it was easy to assume
they did.</p>
<p>It wouldn't have crashed. It would have reranked every result against some other document's vector
and returned slightly worse answers, the kind a quick test waves straight through. A design review
of the actual code caught it before any was written. The fix maps each node to its document from
the record the builder already keeps of where each node came from, and a test checks that searching
through the shared vectors gives exactly the same results and scores as the old way.</p>
<h2>Also this week</h2>
<p>0.9 is a breaking release. It removes TurboQuant, a storage mode that had been deprecated since 0.7
because TurboRabit beats it at every bit budget we measured. Removing a public option is the kind of
decision a maintainer shouldn't infer, so the work stopped and asked first, with the size of the
change in hand: 161 references across five crates. After the removal the core test count dropped
from 291 to 284, and the seven missing tests were accounted for one by one instead of being waved
off as "still green".</p>
<p>The Foxstash page on our site is updated for 0.9, and every claim on it traces to the code.</p>
<h2>If you're buying software</h2>
<p>Ask a vendor what "delete" means in their system, and ask to see the test. Most deletes hide data
rather than erase it, and that's often fine, as long as everyone knows which one they're getting.
Then ask who reviews their work besides them. The best review I got this week came from the one
user who'd have to live with the result. If a vendor can't name anyone like that, the first one is
probably you.</p>
<p><em>Built while the rest are sleeping. Narcoleptic Fox, Orlando FL.</em></p>
]]></content:encoded></item><item><title><![CDATA[Quiet has to be earned]]></title><description><![CDATA[FoxWatch is the PC health monitor I'm building at Narcoleptic Fox. The whole idea fits in one line: stay quiet unless there's something specific to do.
I last wrote about it in July, when it was a des]]></description><link>https://blog.narcolepticfox.com/quiet-has-to-be-earned</link><guid isPermaLink="true">https://blog.narcolepticfox.com/quiet-has-to-be-earned</guid><category><![CDATA[Rust]]></category><category><![CDATA[Windows]]></category><category><![CDATA[Linux]]></category><category><![CDATA[monitoring]]></category><category><![CDATA[Testing]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Sat, 10 Oct 2026 22:55:07 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/gql/69dbec87aadf1107e21d2c6e/9f11e4b3-0dcd-4f0f-8101-d21ad8f3bd41.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>FoxWatch is the PC health monitor I'm building at Narcoleptic Fox. The whole idea fits in one line: stay quiet unless there's something specific to do.</p>
<p>I last wrote about it in July, when it was a design document and a screen that deliberately opened to nothing. Since then it has become a working Windows desktop app and a command-line monitor for Linux servers. It isn't released yet. But it's far enough along that I can tell you what building it taught me, and it taught me one thing, over and over, from different directions: a quiet result has to earn its quiet.</p>
<h2>What it does today</h2>
<p>On Windows, FoxWatch reads the machine's own health signals: disk space, with a "full in N days" estimate instead of a percentage; drive health, read without administrator rights; crashes and unexpected shutdowns from the event log; hardware errors and graphics driver resets; devices that have stopped working; the network link; battery wear. It sorts every one into <strong>act now</strong>, <strong>review</strong> or <strong>ignore</strong> with plain, deterministic rules. No cloud, no telemetry, no machine learning. The data stays in a local database. A fox in the system tray changes colour with the state, and most days it just sleeps.</p>
<p>On Linux it runs headless, on a timer, and answers on the command line with an exit code a script can act on.</p>
<p>Here are five times a quiet result, or a loud one, turned out to be wrong.</p>
<h2>1. Fourteen unexpected shutdowns that weren't</h2>
<p>The first time I installed the app on a real machine and looked at what it said, the dashboard reported 14 unexpected shutdowns. That's alarming. It was also wrong.</p>
<p>Windows records one unclean shutdown twice, as two different events five to eleven seconds apart. So 4 power cuts became 8 rows. On top of that, 6 hardware-error records sat in the same table. 8 plus 6 is 14. And because every power cut counted double, a crash threshold of 3 was really tripping at one and a half real events.</p>
<p>Two of those six hardware errors were logged in the same second as a power cut. They were fallout from the power cut, not evidence of failing hardware.</p>
<p>The part that bothered me most was what the dashboard <em>didn't</em> say. The real early warning in that data was three hardware errors on one night, a minute apart: 01:31, 01:32, 01:33. They sat under a count threshold of 10, so FoxWatch said nothing. A count over a long window can't tell three errors in three minutes from three errors in three months. The rule now measures density: errors in any 10-minute span.</p>
<p>One more thing came out of it. Four different parts of the app had been counting shutdowns, and they gave four different numbers. They now share one function. Two implementations of one idea is where a bug hides; four is worse.</p>
<p>None of this showed up in code review. It showed up when I ran the real app against a real machine's history and read what it told me.</p>
<h2>2. The driver that turned into malware overnight</h2>
<p>A health monitor wants motherboard temperatures and fan speeds. On Windows, the usual way to read them for years was a driver called WinRing0, used by a lot of well-known hardware tools.</p>
<p>I checked it instead of trusting my memory of it. It has a published vulnerability (CVE-2020-14979) that lets any user on the machine read and write kernel memory. It hasn't changed since 2008. And in March 2025, Microsoft Defender started flagging it as a severe threat, which broke several popular fan and monitoring tools at once. It was fine for fifteen years, and then, overnight, it wasn't.</p>
<p>If FoxWatch had bundled it, the antivirus would have quarantined the health monitor. So it doesn't. Board sensors sit behind an interface that lets the driver underneath be swapped, and writing my own kernel driver was off the table.</p>
<p>The replacement I'm evaluating is PawnIO, a driver the user installs themselves. The question that decides the design is simple: can an ordinary user account open it? So I measured all three cases:</p>
<table>
<thead>
<tr>
<th>Case</th>
<th>Result</th>
</tr>
</thead>
<tbody><tr>
<td>Standard user, driver not installed</td>
<td>Not installed</td>
</tr>
<tr>
<td>Standard user, driver installed</td>
<td>Access denied</td>
</tr>
<tr>
<td>Administrator, driver installed</td>
<td>Opens, reports version 2.2.0</td>
</tr>
</tbody></table>
<p>That settles it: reading board sensors needs a small privileged service, not just a library call.</p>
<p>The interesting part is how that table almost went wrong. The administrator case is the control, the one that should simply work, and the first time I ran it, it failed. The bug was mine: I'd given the driver's version call an 8-byte buffer, and it accepts exactly 4. With that bug in place, the standard-user result still looked perfect, because the driver refused the standard user before my code ever reached the broken call. Without the control run, I'd have written down a correct conclusion, measured with a broken instrument, and never known.</p>
<p>Until that privileged service exists, FoxWatch on Windows shows board sensors as <strong>not watched</strong>. Not as fine.</p>
<h2>3. Every signal says why it's quiet</h2>
<p>That became the core design rule: no number without its verdict, and absence is never painted as health. A signal that isn't present on this machine, needs administrator rights, or has stopped reporting is labelled as exactly that. None of those is ever green.</p>
<p>It sounds obvious. Here's what the machines said when I measured instead of assuming:</p>
<ul>
<li><strong>On Linux, <code>journalctl</code> exits successfully even when it isn't allowed to read anything.</strong> "Insufficient permissions" still comes back as status 0. Trusting the exit status would have reported a machine whose crash log it was forbidden to read as a machine that had never crashed.</li>
<li><strong>A Linux system library reported 19 disks on a machine with 1.</strong> The rest were host mounts and package images. Count them as drives and the disk-space checks fill up with noise.</li>
<li><strong>Windows exposes 8 board sensors to an ordinary user: 4 temperature, 2 fan, 2 voltage.</strong> All 8 report "Status: OK". None of them report a reading.</li>
<li><strong>The one temperature zone an ordinary user <em>can</em> read said 3010</strong>, which is 301.0 kelvin, about 28°C. It said 3010 for 20 samples at idle. Then I ran all 32 processor cores at 100% for 90 seconds, and it still said 3010. A number that never moves isn't a measurement. That one test settled a design question in about forty minutes.</li>
</ul>
<p>Every one of those would have produced a calm, green, wrong dashboard.</p>
<h2>4. The leak that would have taken years</h2>
<p>FoxWatch keeps raw samples for a week, then rolls them up into hourly and daily summaries so the database doesn't grow forever. On a headless Linux install, the rollup had never run. Not once.</p>
<p>Eight hours of running produced 527 raw rows and zero hourly rows, about 160 KB. That projects to roughly 68 MB a year, with no end. Not an emergency. The kind of bug that bites in year three, when nobody remembers why the disk is filling. With the fix, the database stays flat at about 1.3 MB.</p>
<p>A health monitor slowly filling the disk it's supposed to be watching is exactly the kind of irony I'd rather find myself.</p>
<h2>5. The tests that agreed with me</h2>
<p>The last one is the most uncomfortable. FoxWatch had over a hundred test scenarios at the time, all passing, and none of them caught a message that would have told a user: "Tctl spent 100% of the last 24 hours."</p>
<p>Tctl is the register name of a temperature sensor on AMD processors. It means nothing to a person. The tests missed it because every fixture label was written by hand, and 13 of the 18 processor fixtures quietly defaulted to the class "other". The tests checked my assumptions about the data, not the data. A fixture is a memory of what I assumed.</p>
<p>The message now names the part in plain words: "The processor".</p>
<h2>Where it stands</h2>
<p>FoxWatch isn't released. Between here and a release: the privileged sensor service on Windows, board sensors on Linux, and a proper release pipeline and download. I'll write about it again when there's something you can install.</p>
<h2>Also on the blog this week</h2>
<p><strong><a href="https://foxhole.hashnode.dev/my-contrast-check-found-nothing">My contrast check found nothing, so I broke it on purpose</a>.</strong> An AI review of my company's site said some text looked muted. Before touching a colour, I measured. The first measurement reported every colour on the site as black: the converter couldn't parse the modern CSS colour format and silently fell back to black, then compared black with black everywhere and called it fact. The rebuilt check passed three pairs with known answers (21:1, 1:1 and 4.54:1), found zero real failures, and only then got trusted, after I planted two failures on purpose and watched it catch both. What was actually wrong was text size. No colour changed.</p>
<p>Same lesson, different week.</p>
<h2>If you're buying software</h2>
<p>Ask what a vendor's monitoring shows when it can't see something. If the answer is green, the dashboard is guessing. And when someone tells you a check passed, ask whether anyone has seen it fail. "No issues found" is only worth something from a check that has been caught finding one.</p>
<p><em>Built while the rest are sleeping. Narcoleptic Fox, Orlando FL.</em></p>
]]></content:encoded></item><item><title><![CDATA[My contrast check found nothing, so I broke it on purpose]]></title><description><![CDATA[An AI review of my company's website said some secondary text "appears small or muted" and suggested raising contrast. Fair enough. Muted grey on near-black is a classic accessibility failure, and the]]></description><link>https://blog.narcolepticfox.com/my-contrast-check-found-nothing</link><guid isPermaLink="true">https://blog.narcolepticfox.com/my-contrast-check-found-nothing</guid><category><![CDATA[Accessibility]]></category><category><![CDATA[Web Development]]></category><category><![CDATA[CSS]]></category><category><![CDATA[Testing]]></category><category><![CDATA[UX]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Wed, 07 Oct 2026 13:00:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/gql/69dbec87aadf1107e21d2c6e/8a954cde-3168-493e-a055-4cddcdda41f1.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>An AI review of my company's website said some secondary text "appears small or muted" and suggested raising contrast. Fair enough. Muted grey on near-black is a classic accessibility failure, and the fix sounds simple: brighten the grey.</p>
<p>But brightening a colour token changes every page at once, and the colours come from a design system with rules of its own. So before touching a token I wanted a number: which text, on which background, fails which standard.</p>
<p>Claude Code, my pair on this, wrote the measurement. The first one was useless.</p>
<h2>Every colour was black</h2>
<p>The first pass walked every text element on eight pages, worked out its colour and the colour behind it, and computed the WCAG contrast ratio. The standard wants at least 4.5:1 for body text and 3:1 for large text.</p>
<p>The results came back looking like a disaster. Ratios of 1.00, 1.01, 1.03. White buttons on white. Text invisible against its own background.</p>
<p>They weren't. The site defines its colours in <code>oklch()</code>, a modern CSS colour format. The measurement converted colours by painting them onto a canvas and reading the pixel back, and the canvas didn't accept <code>oklch()</code>. Every colour it couldn't parse silently stayed black. So it was comparing black with black, everywhere, and reporting it as fact.</p>
<p>What kept that from turning into a redesign was that the numbers were too absurd to believe, and the rule that came out of it: before trusting a converter, run it on pairs whose answers are already known. White on black must be 21:1. Black on black must be 1:1. <code>#767676</code> on white is about 4.54:1. The canvas converter would have failed the first one.</p>
<h2>Measuring again</h2>
<p>The rebuilt converter asks the browser itself to translate each colour into sRGB, then does the contrast arithmetic. It got the three known pairs exactly: 21, 1 and 4.54.</p>
<p>The real result: <strong>zero failures.</strong> Every text element on all eight pages meets WCAG AA. The "muted" grey is 7.63:1 on the page background and 6.91:1 on cards. That's well past 4.5:1, and on the background it even clears the stricter AAA level.</p>
<h2>A check that finds nothing has to prove it can find something</h2>
<p>Zero is the number I'd most want to hear and the one I trust least. A checker that is subtly broken (wrong selector, wrong background, an early return) also reports zero.</p>
<p>So we planted failures: a dark grey paragraph on the page background, and translucent white text on the teal band. The checker flagged both, at 1.75:1 and 2.70:1. Only then did the zero mean anything.</p>
<h2>What was actually wrong</h2>
<p>If contrast was fine, why did the text read as muted? Size. Counting text under 13 pixels on the home page found 19 blocks. Most were uppercase eyebrow labels, small by design and left alone. The rest were captions and chips inside the product illustrations, at 12 pixels. Those went to 13.</p>
<p>The other note in the review was "lengthy opening copy". Counted, most page openings ran 28 to 79 words. One product page ran 107 across four paragraphs. It's now 86 across three. The detail it dropped is still on the page, further down, where someone who wants it will look.</p>
<p>No colour token changed.</p>
<h2>If you're buying software</h2>
<p>When someone tells you a check passed, ask what it would have looked like if it had failed, and whether anyone has seen it fail. The broken version of this measurement didn't report "unknown". It reported, confidently, that every colour on the site was black. A tool that can't fail on purpose can't be trusted when it passes.</p>
]]></content:encoded></item><item><title><![CDATA[A maintenance page is not a locked door]]></title><description><![CDATA[The day was supposed to be paperwork.
I'd just approved Narcoleptic Fox's Quality Manual, the document that says how my company builds and ships software, and set up a yearly internal audit run by som]]></description><link>https://blog.narcolepticfox.com/a-maintenance-page-is-not-a-locked-door</link><guid isPermaLink="true">https://blog.narcolepticfox.com/a-maintenance-page-is-not-a-locked-door</guid><category><![CDATA[Security]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Quality Assurance]]></category><category><![CDATA[GitHub Actions]]></category><category><![CDATA[Software Engineering]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Tue, 06 Oct 2026 13:00:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/gql/69dbec87aadf1107e21d2c6e/fa2c29c1-c38e-4b67-af76-655a20c81378.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The day was supposed to be paperwork.</p>
<p>I'd just approved Narcoleptic Fox's Quality Manual, the document that says how my company builds and ships software, and set up a yearly internal audit run by someone outside engineering. The auditor isn't a developer, so she needed evidence she could check without knowing what a pull request is. I had Claude Code write a script that collects a year of it from GitHub: merged pull requests and their checks, commits that reached <code>main</code> without one, releases, and every security alert with how long it took to fix.</p>
<p>The first run was meant to prove the script worked. It proved the manual didn't describe the company yet.</p>
<h2>First, three of the numbers were mine</h2>
<p>Before any finding was real, the tool had to stop being wrong.</p>
<ul>
<li><strong>It never finished.</strong> It asked GitHub, one call per commit, whether a pull request stood behind it. An hour in, one transient error killed the run. The rewrite asks about a hundred commits per query: 129 seconds for 21 repositories. I didn't trust it until it agreed with the old method on ten commits, five flagged and five passed.</li>
<li><strong>160 overdue security alerts was really 48.</strong> Alerts had been switched on for every repository two days earlier, so most were two days old, and the script counted every open alert as a miss. A high-severity alert has 30 days. It only misses once it's older than that. Same data, a third of the number, and the whole difference was a rule I had invented.</li>
<li><strong>A check failed for the wrong reason.</strong> Two pages looked broken because a label was missing from the page text. The labels were styled in uppercase, and the browser returns text as displayed. That was a broken check, not a broken page.</li>
</ul>
<h2>The door</h2>
<p>With the numbers honest, the summary said a dozen critical alerts were about to come due. Some belonged to a product I'd paused. Its site showed a friendly "we'll be back soon" page. Everyone, me included, had filed it as off.</p>
<p>The response headers disagreed. They showed the application itself answering every request and rewriting each page to the maintenance screen. One of the open alerts was a critical, unauthenticated remote code execution in a part of the web framework that the maintenance rewrite didn't cover. I asked for one ordinary URL on that path. It came back <code>200</code>, served by a framework version inside the vulnerable range.</p>
<p>It was patched and verified closed the same day.</p>
<p>What the healthy path looked like while it was open: a maintenance page, a repository nobody had touched in days, and a status in my head that said "paused". None of those is a control.</p>
<h2>The checks that never ran</h2>
<p>The fix itself was quick. The pipeline wasn't. Every check failed in two seconds with no steps, because the account's hosted CI minutes had been used up 24 days earlier. Everything else kept passing because it ran on my own build machines. This product's CI had failed on every push since, silently, and nobody noticed because nobody pushed. The first push that mattered was the security fix. Two lines of workflow later, it ran on my own runner and passed, and 45 alerts closed in one rescan.</p>
<h2>And the habit it caught</h2>
<p>The audit had one more finding, and it was about how the work gets done: 22 code changes pushed straight to <code>main</code> after branch protection, most of them from AI coding sessions working under an older "small changes go straight in" habit. Fine for a personal notes repo. Not fine under a quality manual that says code goes through a pull request with checks.</p>
<h2>What changed</h2>
<p>The manual went from 1.0 to 1.1 the same day:</p>
<ul>
<li><strong>Documentation-only changes may go straight to <code>main</code>.</strong> Anything that's built or deployed goes through a pull request. A direct code commit is an emergency and gets an issue explaining it.</li>
<li><strong>A paused product carries a <code>paused</code> label and fixes its critical and high alerts before it's deployed or worked on again.</strong> And, in so many words: a deployment in maintenance mode is still deployed.</li>
<li>The rules went into the instructions my AI sessions read at the start of every session, because the habit they replaced lived there too.</li>
</ul>
<h2>If you're buying software</h2>
<p>Ask what's still running that everyone thinks is switched off. Then ask how the team would know. A maintenance page, a quiet repository and a confident status update all look exactly like a locked door, right up until someone checks the headers.</p>
]]></content:encoded></item><item><title><![CDATA[Classify the errors before you trust the score]]></title><description><![CDATA[OpenAI is shutting off gpt-5.4-nano on 1 April 2027. In Mousing, my document extraction platform, nano did two jobs: it classified incoming documents, and it was the verifier, the second model that re]]></description><link>https://blog.narcolepticfox.com/classify-the-errors-before-you-trust-the-score</link><guid isPermaLink="true">https://blog.narcolepticfox.com/classify-the-errors-before-you-trust-the-score</guid><category><![CDATA[AI]]></category><category><![CDATA[llm]]></category><category><![CDATA[Machine Learning]]></category><category><![CDATA[benchmarking]]></category><category><![CDATA[golang]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Mon, 05 Oct 2026 13:00:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/gql/69dbec87aadf1107e21d2c6e/1e4ed08e-b441-4abf-9b22-a59d3c331822.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>OpenAI is shutting off <code>gpt-5.4-nano</code> on 1 April 2027. In Mousing, my document extraction platform, nano did two jobs: it classified incoming documents, and it was the verifier, the second model that re-reads each page and checks every extracted value.</p>
<p>Swapping the model is one environment variable. Trusting the swap is not. Every number I had for the verifier (how well its confidence separates right from wrong, the review threshold, the token budget) was measured on nano. None of it transfers to a different model.</p>
<p>So I re-benchmarked before choosing, and the first answer was wrong.</p>
<h2>The setup</h2>
<p>Nine runs through the real pipeline: the production extraction template, the verifier and the plain-code magnitude check, all over the same 100 photographed receipts from the public CORD dataset, on the same day. Seven model pairings, plus two of them again with reasoning effort turned down.</p>
<p>Before reading anything into it, I measured the noise. Two runs with the same extractor scored 0.88 and 0.87 F1 and got 35 and 34 receipts fully correct. So a hundredth of F1 and a few receipts either way is just run-to-run variance.</p>
<h2>The first answer</h2>
<p>One pairing stood out. With <code>gpt-6.1-sol</code> as the extractor:</p>
<table>
<thead>
<tr>
<th>Extractor / verifier</th>
<th>F1</th>
<th>Receipts fully correct</th>
<th>Cost per page (est.)</th>
</tr>
</thead>
<tbody><tr>
<td><code>5.4-mini</code> / <code>5.4-nano</code> (production)</td>
<td>0.88</td>
<td>35</td>
<td>~$0.004</td>
</tr>
<tr>
<td><code>6-luna</code> / <code>6-luna</code>, low effort</td>
<td>0.89</td>
<td>38</td>
<td>~$0.0009</td>
</tr>
<tr>
<td><code>6.1-sol</code> / <code>6-luna</code></td>
<td><strong>0.91</strong></td>
<td><strong>52</strong></td>
<td>~$0.010</td>
</tr>
</tbody></table>
<p>Everything else landed between 33 and 38 fully correct receipts. Sol produced 35% fewer wrong fields than any other arm. Clearly outside the noise, at roughly ten times the cost of the cheapest pairing. I was ready to pick it.</p>
<h2>Then I read the wrong answers</h2>
<p>Every wrong field from all nine runs, classified:</p>
<ul>
<li><strong>60% were item names, and many were the answer key's fault.</strong> CORD's ground truth is built from OCR tokens. The "truth" for one item was <code>CHOCO CUS ARD PASTRY</code>. A model that wrote <code>CHOCO CUSTARD PASTRY</code> was marked wrong.</li>
<li><strong>26% were rows out of line.</strong> Items are scored by position. One receipt's labels listed a size modifier as its own item, and that single receipt caused about a fifth of all errors in every run.</li>
<li><strong>One field was ambiguous.</strong> Mousing's receipt template had a single <code>price</code> field. Mini and luna filled it with the unit price. Sol filled it with the line total. CORD's labels use line totals. Part of sol's lead was guessing the same way the answer key did.</li>
</ul>
<p>Re-scored with fuzzy matching on names, the gap closed: mini plus nano 0.90, luna 0.92, sol 0.92.</p>
<p>The fix for the ambiguity wasn't a model. I split <code>price</code> into <code>unit_price</code> and <code>amount</code>, both "exactly as printed, never calculated", and re-ran three arms:</p>
<table>
<thead>
<tr>
<th></th>
<th>mini + nano</th>
<th><strong>luna, both roles, low effort</strong></th>
<th>sol + luna</th>
</tr>
</thead>
<tbody><tr>
<td>Line total (<code>amount</code>) F1</td>
<td>0.76</td>
<td><strong>0.90</strong></td>
<td>0.89</td>
</tr>
<tr>
<td>Unit price F1</td>
<td>0.48</td>
<td><strong>0.89</strong></td>
<td>0.83</td>
</tr>
<tr>
<td>Item name F1</td>
<td>0.81</td>
<td>0.80</td>
<td><strong>0.85</strong></td>
</tr>
<tr>
<td>Cost per page (est.)</td>
<td>~$0.004</td>
<td><strong>~$0.0009</strong></td>
<td>~$0.010</td>
</tr>
</tbody></table>
<p>On the money fields, the ones a customer actually acts on, luna matched sol on line totals and beat it on unit prices, at about a tenth of the cost. Sol's remaining lead is spelling item names, which is exactly where the label noise lives. Mini, meanwhile, filed the line total under <code>unit_price</code> on 37 lines, which means the new schema must never ship without the model change.</p>
<h2>Two things I'd have missed</h2>
<p><strong>Reasoning effort was free.</strong> Turning luna's reasoning effort down kept the same accuracy, cut cost by about a third, and took the slow tail of extraction (the 95th percentile) from 64 seconds to 7. Nobody notices an average. Everybody notices a 64-second wait.</p>
<p><strong>Fewer false flags is a trade, not a win.</strong> Nano flagged 39% of correct values for human review, roughly five pointless reviews for every real catch. In practice that's "a person reviews everything." The newer verifiers cut false flags by about 95%, and let more real errors through without a flag. That's a product decision about review load against silent errors, and I'd rather make it on purpose than discover it.</p>
<h2>What ships</h2>
<p><code>gpt-6-luna</code> for extraction, verification and classification, at low effort. It's in production now.</p>
<p>The one number I'd keep from all of this isn't an F1 score. It's this: the first table pointed at a model that costs ten times more, and the reason was mostly in the benchmark. A score is a summary. The wrong answers are the evidence.</p>
<p><em>Numbers are one run per arm, on Indonesian café receipts, not the business documents Mousing is built for. Costs are estimates from token counts at OpenAI's standard prices on 2026-10-02.</em></p>
]]></content:encoded></item><item><title><![CDATA[I asked another tool to check my numbers. It found my bugs, not its own.]]></title><description><![CDATA[Tezcatl went public last week. It's a command-line tool that reads a C or C++ project's own build configuration and writes a code metrics baseline: lines of code, cyclomatic complexity, documentation ]]></description><link>https://blog.narcolepticfox.com/tezcatl-the-oracle-found-my-bugs</link><guid isPermaLink="true">https://blog.narcolepticfox.com/tezcatl-the-oracle-found-my-bugs</guid><category><![CDATA[C++]]></category><category><![CDATA[C]]></category><category><![CDATA[Code Quality]]></category><category><![CDATA[Testing]]></category><category><![CDATA[open source]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Sun, 04 Oct 2026 13:00:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/gql/69dbec87aadf1107e21d2c6e/2b807f43-59cc-4224-a750-e4af594f8111.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Tezcatl went public last week. It's a command-line tool that reads a C or C++ project's own build configuration and writes a code metrics baseline: lines of code, cyclomatic complexity, documentation coverage, test coverage, and the include graph with its cycles. MIT licence, source on <a href="https://github.com/Narcoleptic-Fox/tezcatl">GitHub</a>, docs at <a href="https://narcoleptic-fox.github.io/tezcatl">narcoleptic-fox.github.io/tezcatl</a>.</p>
<p>That's the announcement. The part worth writing about is how I decided whether to trust it.</p>
<h2>A metrics tool is a claim about someone else's code</h2>
<p>If Tezcatl says a function has a complexity of 14, somebody's going to use that number to decide where to spend review time. So the rule from day one was: every metric gets checked against an independent tool on real open-source code, and every difference gets explained or fixed. Not "close enough". Explained.</p>
<p>I expected that to be a chore. I figured I'd spend a day documenting the other tools' quirks.</p>
<p>That's not what happened.</p>
<h2>What lizard found</h2>
<p>First comparison: cyclomatic complexity against <a href="https://github.com/terryyin/lizard">lizard</a> on Catch2. 1,381 functions matched, and lizard found 315 that Tezcatl didn't.</p>
<p>The easy story was "header-only templates nothing includes". It was true for 121 of them. The other 194 were in files Tezcatl had parsed. Every member function of every class template was missing.</p>
<p>The cause was one flag. Catch2 builds as C++14 from an MSVC compilation database, and in clang-cl mode before C++20, template bodies aren't parsed until something instantiates them. That's how MSVC does it. Tezcatl's own test fixtures were C++20, where that behaviour is off, so no test of mine could ever have seen it. One flag brought 180 functions back.</p>
<p>Then the reverse: 108 functions only Tezcatl found. 63 of them were <code>= default</code>. My test for "is this a real function" was "does it have a body", and clang writes a body for a defaulted function once it's used. The fixture had one of those too. It passed, because nothing used it.</p>
<p>Then two functions where lizard said 2 and I said 1. A genuine <code>&amp;&amp;</code> inside a template that clang couldn't resolve yet, because the library overloads <code>operator&amp;&amp;</code>. It counts now.</p>
<p>After all that, the two tools agree on 99.0% of the functions both find, and every one of the remaining differences has a written reason. In half of them lizard is wrong. In the other half it's a convention I picked on purpose.</p>
<p>Three bugs, all mine, all invisible to a test suite that was green the whole time.</p>
<h2>The tests that couldn't fail</h2>
<p>That's the uncomfortable part. The suite wasn't just missing cases. Some of the tests couldn't fail at all.</p>
<p>So every rule got a sabotage: break the code on purpose and make sure a test goes red. 13 of the first 14 did. The fourteenth was a percentile calculation, tested on sample sizes where rounding up and rounding to nearest give the same answer. Wrong code passed. Six values tells them apart.</p>
<p>Another one stayed green because of a semicolon. CTest treats a semicolon in a pass pattern as a list separator and passes the test if <em>any</em> piece matches. One of those pieces was a fragment that matched almost any output.</p>
<p>Tezcatl's test suite now covers 95.1% of its own production lines, and there are 142 recorded deliberate faults, each caught by a test. That second number is the one I trust.</p>
<h2>The numbers, as they stand</h2>
<table>
<thead>
<tr>
<th>Check</th>
<th>Result</th>
</tr>
</thead>
<tbody><tr>
<td>Earthworm (open-source seismic processing), 929 C and C++ files</td>
<td>0 parse errors, full report in 3.5 s on 16 threads</td>
</tr>
<tr>
<td>Complexity vs lizard on Earthworm, outside bundled SQLite</td>
<td>99.0% identical (7,885 of 7,962 functions)</td>
</tr>
<tr>
<td>Coverage vs gcovr on libmseed, built with gcc 14</td>
<td>lines, branches and functions identical in all 42 files</td>
</tr>
<tr>
<td>Coverage vs gcovr on Tezcatl's own code</td>
<td>identical in all 85 files</td>
</tr>
</tbody></table>
<p>The remaining lizard differences on Earthworm trace to code that compiles away: disabled assertions and test macros. Tezcatl measures what the build actually compiles. That's a feature and a limit, and the docs say so.</p>
<h2>And one I had to take back</h2>
<p>While re-measuring for a capabilities write-up, I rebuilt in Release and Earthworm finished in 3.5 seconds. The number I'd been quoting everywhere was slower.</p>
<p>Every timing in the README, the docs and the release notes had come from a Debug build. The comparisons were fair, Debug against Debug, so the speed-ups held. But the seconds described a build nobody ships, and nothing said which build it was.</p>
<p>A number without its conditions isn't evidence. I've written that down for other people's code. Took me three days to catch it in mine.</p>
<h2>How it was built</h2>
<p>I built Tezcatl with Claude Code as my pair. That isn't a footnote; it's part of why the checking mattered. An AI pair writes plausible code quickly, and plausible is exactly what a comparison against lizard and a pile of deliberate faults are there to test. Every bug in this post got past both of us and a green test suite. None of them got past a second tool and a deliberate fault.</p>
<h2>What it doesn't do</h2>
<p>It doesn't check MISRA or CERT rules. It doesn't run your tests; it imports the coverage gcov, llvm-cov or lcov already produced. It isn't certified or audited, and it's brand new: first release was 2026-09-26. If you need any of those, it's not the tool yet.</p>
<p>If you've got a C or C++ codebase and you want a baseline before someone starts changing it, it's free. Tell me where it's wrong.</p>
]]></content:encoded></item><item><title><![CDATA[Effects, Actions, and Why Workflow Runtimes Need Side-Effect Boundaries]]></title><description><![CDATA[First published in my LinkedIn newsletter. Corsac is still a work in progress; this describes it as it was then.
The annoying part of a workflow runtime isn't running code.
It's knowing what code is s]]></description><link>https://blog.narcolepticfox.com/effects-actions-and-why-workflow-runtimes-need-side-effect-boundaries</link><guid isPermaLink="true">https://blog.narcolepticfox.com/effects-actions-and-why-workflow-runtimes-need-side-effect-boundaries</guid><category><![CDATA[workflow]]></category><category><![CDATA[distributed systems]]></category><category><![CDATA[Rust]]></category><category><![CDATA[golang]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Sat, 03 Oct 2026 17:21:24 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/7cd92b41-2f78-4278-b52b-930d8d2cbc34.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>First published in my LinkedIn newsletter. Corsac is still a work in progress; this describes it as it was then.</em></p>
<p>The annoying part of a workflow runtime isn't running code.</p>
<p>It's knowing what code is safe to run again.</p>
<p>That one question shapes almost everything: retries, replay, recovery, failure handling, run history, operator trust. If a process dies halfway through a workflow, the runtime has to decide what probably happened and what it's allowed to repeat.</p>
<p>Get that wrong and you charge a card twice. Or you don't charge it at all. Or you send the same Slack message six times during a deploy and pretend it was a glitch.</p>
<p>That's why Gust and Corsac now distinguish <code>effect</code> from <code>action</code>.</p>
<p>The distinction is small in syntax, large in runtime meaning.</p>
<h2>A workflow is mostly side effects</h2>
<p>Consider an order notification workflow:</p>
<pre><code class="language-gust">effect parse_order_json(body: String) -&gt; OrderPayload
effect format_slack_message(order: OrderPayload) -&gt; String
action post_slack(channel: String, text: String, credential_id: String) -&gt; String
effect compute_retry_eligible(step: String, attempt: i64) -&gt; bool
</code></pre>
<p>All four declarations are side-effectful from the language's point of view because the workflow calls out to host code.</p>
<p>But operationally, they aren't the same.</p>
<p>Parsing a JSON body isn't the same as posting to Slack. Formatting a message isn't the same as charging a card. Computing retry eligibility isn't the same as sending an email.</p>
<p>If a runtime repeats parsing or formatting, that's usually fine. If it repeats a Slack post, a card charge, or a webhook call, the user sees duplicated work — and the user is the one who notices first.</p>
<p>A workflow runtime needs that distinction in the contract, not hidden in handler names.</p>
<h2>The rough model</h2>
<p>The current model:</p>
<ul>
<li><code>effect</code>: replay-safe, idempotent, or non-committing work in the workflow path</li>
<li><code>action</code>: non-idempotent or externally visible committed work</li>
</ul>
<p>That's intentionally practical language.</p>
<p>An <code>effect</code> can still talk to the outside world. It might read from a cache, call a validation service, or compute something through host code. The point is that the workflow runtime is allowed to treat it as safe enough to repeat in the durability model being used.</p>
<p>An <code>action</code> is different. It's the operation where the workflow commits something visible outside the runtime. Sending the message. Charging the card. Creating the ticket. Posting the webhook.</p>
<p>That operation needs a boundary. Otherwise the runtime is guessing in the dark.</p>
<h2>The failure case</h2>
<p>The easiest way to see the problem: imagine a crash at the worst possible time.</p>
<p>Suppose a handler does this:</p>
<pre><code class="language-gust">on notify(ctx: MessageFormattedCtx) {
    let ts = perform post_slack("#orders", ctx.slack_text, "cred-slack-prod");
    let retry = perform compute_retry_eligible("notify", 1);
    goto NotificationSent(ctx.order.order_id, ts);
}
</code></pre>
<p>Now imagine this timeline:</p>
<table>
<thead>
<tr>
<th>Step</th>
<th>What happened</th>
<th>What the runtime knows</th>
</tr>
</thead>
<tbody><tr>
<td>1</td>
<td>The workflow entered <code>MessageFormatted</code>.</td>
<td>This state was recorded.</td>
</tr>
<tr>
<td>2</td>
<td><code>post_slack</code> sent the message.</td>
<td>Maybe nothing was recorded yet.</td>
</tr>
<tr>
<td>3</td>
<td>The process died before <code>goto NotificationSent</code>.</td>
<td>The last durable boundary still looks like <code>MessageFormatted</code>.</td>
</tr>
<tr>
<td>4</td>
<td>The runtime retries from the last boundary.</td>
<td>It might call <code>post_slack</code> again.</td>
</tr>
</tbody></table>
<p>That's the duplicate-action bug.</p>
<p>Not theoretical. Just bad timing.</p>
<p>The issue isn't Slack specifically. It could be a payment charge, a ticket creation, a webhook, an email, or any operation where the outside world observes the result. Once that operation succeeds, the runtime needs a way to avoid pretending it didn't happen.</p>
<h2>Why Corsac cares</h2>
<p>Corsac Phase 1 uses node-boundary durability.</p>
<p>Which means it records progress around meaningful workflow boundaries instead of implementing full event-sourced replay of every internal step and effect result. That's the right Phase 1 scope because it keeps the runtime small enough to build and inspect.</p>
<p>But node-boundary durability creates a constraint: handlers have to be honest about committed side effects.</p>
<p>If a handler performs an action, then performs more effects, then transitions, the runtime has the problem above. If the process dies after the action but before the later work completes, a naive retry could run the action again.</p>
<p>Which is why Corsac's accepted Phase 1 rule is conservative:</p>
<ul>
<li>a handler path may perform many effects</li>
<li>a handler path may perform at most one action</li>
<li>that action should be the last side-effectful step before transition</li>
</ul>
<p>This isn't the only possible workflow durability model. It's the one that matches the current Corsac runtime. Picking a model and being honest about its constraints is more useful than promising a model the runtime can't actually back up.</p>
<h2>What the rule looks like</h2>
<p>This is the safe shape:</p>
<pre><code class="language-gust">on notify(ctx: MessageFormattedCtx) {
    let ts = perform post_slack("#orders", ctx.slack_text, "cred-slack-prod");
    goto NotificationSent(ctx.order.order_id, ts);
}
</code></pre>
<p>The action is the final side-effectful call before the transition.</p>
<p>The suspicious shape is the one from the failure timeline. The problem isn't that <code>compute_retry_eligible</code> is bad. The problem is that it happens after the committed action. If the runtime needs to reason about replay from a boundary, that path is harder to make safe.</p>
<p>In a stronger future runtime with full event logging, checkpointed action results, and replay semantics, the rule might evolve. In Phase 1, conservative is the correct default.</p>
<h2>What Gust contributes</h2>
<p>The important design move was pushing this distinction into Gust itself.</p>
<p>Gust now has an <code>action</code> keyword alongside <code>effect</code>, and the compiler preserves that distinction through the AST, formatting, code generation metadata, and MCP parse output.</p>
<p>The validator also warns when action usage violates the handler-safety rules:</p>
<ul>
<li>more than one action on a path</li>
<li>an action followed by later side-effectful work before transition</li>
</ul>
<p>Which means the language can help runtime authors before Corsac even enters the picture.</p>
<p>It also means tooling can see the distinction. A parser, MCP tool, editor integration, or workflow runtime can inspect a machine and know which declared operations are effects and which are actions.</p>
<h2>What Corsac contributes</h2>
<p>Corsac also performs conservative workflow linting before <code>check</code>, <code>build</code>, and <code>run</code>.</p>
<p>That linter scans <code>.gu</code> source for declared effects/actions and performed calls. It rejects workflows that violate the Phase 1 safety model.</p>
<p>This isn't presented as a perfect proof. The current Corsac linter is source-level, not a full Gust parser. It can be confused by comments, strings, quoted braces, or future Gust syntax.</p>
<p>That limitation is acceptable for Phase 1 because it's explicit. The stronger long-term answer is to consume Gust compiler metadata or use a parser-backed validation layer.</p>
<p>The important thing is that Corsac doesn't silently ignore the boundary. It enforces the rule its current runtime model depends on. Which also makes the limitation visible — if a workflow violates the side-effect placement rule, it fails early during <code>check</code>, <code>build</code>, or <code>run</code> instead of becoming an ambiguous recovery problem at 3am.</p>
<h2>Why not make every side effect an action?</h2>
<p>Because then the runtime loses useful information.</p>
<p>If every call is treated as non-idempotent committed work, the workflow becomes harder to retry, harder to replay, and harder to optimize. The runtime can't distinguish between formatting a message and sending it.</p>
<p>That pushes too much caution into the system. It also trains operators to ignore the warnings, because the warnings are everywhere.</p>
<p>The goal isn't to make workflows afraid of side effects. The goal is to make side-effect categories explicit enough that the runtime can make better decisions.</p>
<h2>Why not make actions automatically safe?</h2>
<p>Because a keyword can't make an external operation idempotent.</p>
<p>Calling something an <code>action</code> doesn't prevent Slack, Stripe, or another API from receiving duplicate requests if the runtime is careless. The keyword is a signal. The runtime still has to decide how to record action results, how to pass idempotency keys, how to retry, and how to surface uncertainty to an operator.</p>
<p>In Corsac Phase 1, the answer is intentionally modest: keep action placement constrained and record run history clearly.</p>
<p>Later phases can go further:</p>
<ul>
<li>checkpoint action results</li>
<li>add idempotency-key conventions</li>
<li>persist effect/action event logs</li>
<li>replay from recorded results</li>
<li>expose action boundaries in run inspection</li>
<li>derive stronger metadata from Gust directly</li>
</ul>
<p>Those are runtime features. The language distinction is the foundation.</p>
<h2>What changes later</h2>
<p>The Phase 1 rule is conservative because the Phase 1 runtime is conservative.</p>
<p>A more advanced runtime could relax parts of it if it records enough information:</p>
<ul>
<li>action request ID</li>
<li>action arguments</li>
<li>action result</li>
<li>idempotency key</li>
<li>state before the action</li>
<li>state after the action</li>
<li>whether replay should return the recorded result or fail for operator review</li>
</ul>
<p>At that point, the runtime is no longer relying only on action placement. It has an event log or checkpoint model strong enough to remember what happened inside the boundary.</p>
<p>Corsac isn't there yet.</p>
<p>That's fine. The point of Phase 1 is to make the honest boundary explicit before building a larger durability system. Promising the larger system before the smaller one works is how workflow projects accumulate the kind of vague durability guarantees that fall apart on the third production incident.</p>
<h2>The bigger point</h2>
<p>The <code>effect</code> versus <code>action</code> split is one of the places where Gust and Corsac meet.</p>
<p>Gust needs to express side-effect intent in the workflow contract.</p>
<p>Corsac needs that intent to operate compiled workflows safely enough to be useful.</p>
<p>That's the pattern I want for both projects: the language shouldn't pretend runtime problems don't exist, and the runtime shouldn't reinvent information the language already knows.</p>
<p>A workflow runtime doesn't need every advanced durability feature on day one.</p>
<p>It does need honest boundaries.</p>
<p><code>effect</code> and <code>action</code> are the first version of that boundary.</p>
]]></content:encoded></item><item><title><![CDATA[Corsac Phase 2: Closing the Runtime Gap]]></title><description><![CDATA[First published in my LinkedIn newsletter. Corsac is still a work in progress; this describes it as it was then.
At the end of Corsac Phase 1, the project had crossed an important line.
It was no long]]></description><link>https://blog.narcolepticfox.com/corsac-phase-2-closing-the-runtime-gap</link><guid isPermaLink="true">https://blog.narcolepticfox.com/corsac-phase-2-closing-the-runtime-gap</guid><category><![CDATA[golang]]></category><category><![CDATA[aws lambda]]></category><category><![CDATA[workflow]]></category><category><![CDATA[Devops]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Sat, 03 Oct 2026 17:21:17 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/43667cf5-f2f7-48da-a721-01aad5af900c.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>First published in my LinkedIn newsletter. Corsac is still a work in progress; this describes it as it was then.</em></p>
<p>At the end of Corsac Phase 1, the project had crossed an important line.</p>
<p>It was no longer just an architecture document around Gust. It had a real CLI, a local runtime, run records, repair tooling, runtime configuration, a Lambda packaging path, and tests around the parts most likely to break.</p>
<p>But it still had a gap I didn't want to hand-wave away.</p>
<p>The Lambda path existed. The deploy and invoke commands existed. The fake-AWS tests passed. Docker could produce the Lambda package. The docs were written.</p>
<p>What I didn't have yet was the thing that actually matters for an AWS-facing runtime: proof that the canonical workflow could go through real Lambda and behave the same way.</p>
<p>Phase 2 was about closing that kind of gap.</p>
<p>Not by making Corsac bigger. Not by adding a hosted control plane. Not by starting a visual designer. Not by adding more product surface just because the roadmap had room.</p>
<p>Phase 2 made the narrow thing harder to fool.</p>
<p>The current wedge is still small:</p>
<ul>
<li>CLI-first, Lambda-friendly for AWS-heavy teams</li>
<li>compiled Gust workflows with explicit <code>effect</code> versus <code>action</code> semantics</li>
<li>node-boundary durability (records progress around meaningful workflow boundaries instead of full event-sourced replay — see the post on effects and actions)</li>
<li>local run inspection</li>
<li>no hosted control plane, no visual designer, no shared remote run store</li>
</ul>
<p>The question changed from "can Corsac run a compiled Gust workflow locally?" to "can this path hold up when it leaves the laptop, and can an operator understand what happened afterward?"</p>
<p>That's a more useful question.</p>
<h2>The AWS smoke test finally ran</h2>
<p>The biggest open Phase 1 confidence gap was real AWS validation.</p>
<p>Corsac had already implemented the Lambda path:</p>
<pre><code class="language-sh">corsac package lambda docs/examples/workflows/order-notification.gu --use-docker
corsac deploy lambda docs/examples/workflows/order-notification.gu \
  --function-name &lt;name&gt; \
  --role &lt;role-arn&gt; \
  --region &lt;region&gt; \
  --lambda-env-from CORSAC_SLACK_TOKEN=CORSAC_SLACK_TOKEN \
  --yes
corsac invoke lambda &lt;function-name&gt; \
  --region &lt;region&gt; \
  --payload @docs/examples/payloads/success.json
</code></pre>
<p>That was implementation. It wasn't external validation.</p>
<p>So Phase 2 put the canonical order notification workflow through real Lambda in <code>us-east-1</code>.</p>
<p>The success fixture completed with final state <code>NotificationSent</code>.</p>
<p>The failure fixture returned the expected handled workflow shape: <code>CorsacWorkflowError</code> with <code>failure_class: handler_error</code>.</p>
<p>CloudWatch logs were verified for both requests. The success path showed the simulated Slack handler path and confirmed the credential was present without writing the secret into the notes or docs.</p>
<p>That closed the Phase 1 external validation gap.</p>
<p>But it's important to say exactly what closed and what didn't.</p>
<p>Corsac has now run the canonical workflow through real Lambda. That's real evidence. It's also not the same as automated AWS validation in CI. AWS smoke is still a manual release or design-partner gate, because account setup, IAM permissions, Docker, costs, and cleanup boundaries should be explicit every time.</p>
<p>Manual validation isn't CI.</p>
<p>It's still useful, as long as it's named honestly.</p>
<p>After validation, the temporary smoke resources were deleted: the Lambda function, the CloudWatch log group, and the dedicated execution role. The point was to prove the path, not leave stray infrastructure around because it was convenient.</p>
<h2>The smoke test caught something real</h2>
<p>The best thing about the AWS smoke test is that it didn't just produce a green checkmark.</p>
<p>It found a real bug.</p>
<p>The first deploy and invoke worked. So I let the runtime sit warm for a few minutes and tried again with the failure payload — the kind of "second invoke after the cold start is over" pattern any real workflow would hit in production.</p>
<p>Instead of returning the expected <code>CorsacWorkflowError</code>, the function came back as <code>Runtime.ExitError</code>.</p>
<p>That's the failure mode that turns into a 2am page somewhere. The function appears broken. The workflow logic is fine. The operator stares at CloudWatch trying to figure out which layer to blame, and the answer is in the runtime bootstrap, not the workflow.</p>
<p>The custom Lambda bootstrap had the wrong timeout shape for AWS's actual Runtime API behavior. The HTTP client timeout was reasonable for normal requests but wrong for the <code>invocation/next</code> endpoint. That endpoint is a long-poll. When the runtime is waiting for the next event, it's supposed to sit there. Timing out at the client layer turns a warm idle period into a runtime failure.</p>
<p>The fix was straightforward once the behavior was visible: remove the client-level timeout for the Runtime API long-poll and let Lambda's invocation lifecycle drive the wait.</p>
<p>There was a second AWS-specific cleanup in the same area. Existing-function redeploys needed to stop passing <code>--architectures</code> to <code>update-function-configuration</code>; that belongs on <code>update-function-code</code>.</p>
<p>Neither bug changes the product thesis. Both matter because they're the kind of edge that appears only when the integration is real.</p>
<p>The point of the smoke test wasn't the green checkmark. It was forcing the runtime through AWS's actual behavior.</p>
<h2>Inspection became less awkward</h2>
<p>A workflow runtime gets much less mysterious when every run leaves a record you can ask questions of.</p>
<p>Phase 1 already had local run records under <code>.corsac/runs/</code>. It had <code>history</code>, <code>show-run</code>, and <code>repair-runs</code>. That was enough to prove the local runtime wasn't a black box.</p>
<p>Phase 2 made the inspection surface more operator-shaped.</p>
<p>The first improvement was small but useful: <code>corsac show-run</code> now defaults to the most recent run when no execution ID is supplied. The common local question after a failed run isn't "let me copy that execution ID and paste it somewhere." It's "what just happened?"</p>
<p>The second improvement was machine-readable output. <code>history</code> and <code>show-run</code> now support stable JSON output, so inspection can feed scripts without fragile table parsing.</p>
<p>The third improvement was scope.</p>
<p>Once <code>corsac invoke lambda</code> started recording invocation results, local runs and Lambda records needed to be easy to separate. Phase 2 added explicit scopes: <code>local</code>, <code>lambda</code>, and <code>all</code>.</p>
<p>That means the operator can ask different questions:</p>
<pre><code class="language-sh">corsac history --scope local
corsac history --scope lambda
corsac show-run --scope lambda
</code></pre>
<p>Lambda invocation recording now defaults on. If you invoke a Lambda through Corsac, the result is recorded into <code>.corsac/runs</code> with <code>scope: lambda</code>. If you don't want that, there's a <code>--no-record</code> opt-out.</p>
<p>That default matters. A runtime should leave breadcrumbs unless the operator explicitly asks it not to.</p>
<p>The first version could run workflows. The Phase 2 version is easier to ask what happened.</p>
<h2>Artifacts need receipts</h2>
<p>The other inspection gap wasn't about runs. It was about artifacts.</p>
<p>Before Phase 2, Corsac could build and package workflows. But explaining exactly what produced a given artifact required too much context outside the artifact directory.</p>
<p>That's a bad habit for deployment tooling.</p>
<p>If a zip is going to be deployed, it should be able to explain what produced it.</p>
<p>Phase 2 added metadata to the build and package outputs.</p>
<p>Local build artifacts now get an <code>artifact.json</code> that records the useful facts:</p>
<ul>
<li>schema and artifact kind</li>
<li>workflow digest</li>
<li>target</li>
<li>build mode</li>
<li>selected config environment</li>
<li>Gust version</li>
<li>runtime wrapper hash and revision</li>
<li>cache key inputs</li>
<li>cache decision</li>
</ul>
<p>Lambda packages now get <code>lambda-package.json</code> with package-specific facts:</p>
<ul>
<li>package schema and kind</li>
<li>Lambda runtime</li>
<li>architecture</li>
<li>function and environment fields</li>
<li>source artifact cache fields</li>
</ul>
<p>The CLI also prints metadata paths from build, package, and deploy output, so the operator doesn't have to know the artifact directory layout by memory.</p>
<p>This isn't glamorous work. It isn't a feature that demos well.</p>
<p>It's the kind of thing that makes a runtime easier to trust later, when a generated binary isn't just something you ran once from your own terminal.</p>
<p>Artifacts need receipts.</p>
<h2>Gust 0.2.1 closed the language gate</h2>
<p>The post on effects and actions is about the <code>effect</code> versus <code>action</code> boundary: why workflow runtimes need to distinguish replay-safe work from externally visible committed work.</p>
<p>Phase 2 turned that from a design rule into a release gate.</p>
<p>Corsac had requested annotation comments in Gust for the side-effect contract:</p>
<pre><code class="language-gust">/// gust:effect -- replay-safe / idempotent
effect parse_order_json(body: String) -&gt; OrderPayload

/// gust:action -- not replay-safe / externally visible
action post_slack(channel: String, text: String, credential_id: String) -&gt; String
</code></pre>
<p>Gust 0.2.1 landed with the path Corsac needed.</p>
<p>Corsac then validated the canonical workflow against the real Gust 0.2.1 binary, refreshed the pinned generated Rust fixture, and reran the live compatibility gate.</p>
<p>That gate matters because default CI intentionally uses a checked-in generated fixture. That keeps the normal test suite stable while Gust and Corsac evolve together. But release validation still needs to prove that the current real compiler can build the canonical workflow.</p>
<p>That's what the live Gust gate does.</p>
<p>The fixture refresh also caught a small tooling detail worth documenting: <code>gust build -o</code> now targets an output directory, not a direct output file. The docs and fixture header were updated so the next compiler upgrade doesn't repeat that mistake.</p>
<p>The generated fixture changed in expected ways. It preserved the side-effect annotations in the generated trait comments, included codegen updates from Gust 0.2.1, and became the new pinned fixture for the acceptance path.</p>
<p>After that, Corsac's Phase 2 closeout was no longer waiting on upstream Gust work.</p>
<p>Gust 0.2.1 closed the loop between the language contract Corsac wanted and the runtime validation Corsac needed.</p>
<p>There's one boundary to keep explicit: this doesn't mean Corsac now has full parser-backed semantic proof. Corsac's Phase 2 linter is still a hardened source-level bridge. It masks comments and string literals before matching declarations, handlers, braces, and <code>perform</code> calls, and tests cover comments, strings, quoted braces, block-comment braces, and valid current syntax.</p>
<p>That's stronger than where Phase 1 started.</p>
<p>It's still not the final form. The long-term authority for side-effect semantics should live in Gust metadata or diagnostics. Corsac should consume that contract instead of growing a parallel parser forever.</p>
<h2>What still isn't done</h2>
<p>Phase 2 closed a lot of real gaps. It didn't make Corsac a finished workflow platform.</p>
<p>The remaining limits are still visible:</p>
<ul>
<li>AWS smoke validation is manual, not default CI.</li>
<li>Execution history is local unless a Lambda invocation is recorded through the CLI; there is no shared remote run store.</li>
<li>There is no hosted control plane.</li>
<li>There is no visual workflow designer.</li>
<li>There is no multi-environment promotion workflow.</li>
<li>Parser-backed side-effect proof is still future Gust metadata or diagnostics work.</li>
<li>Full event-sourced replay is deferred.</li>
</ul>
<p>Some of those are gaps. Some are intentional boundaries.</p>
<p>That distinction still matters.</p>
<p>No hosted control plane isn't an accident. It's a choice to keep the current wedge focused until the compiled runtime has enough pressure from real workflows to justify a bigger surface.</p>
<p>The same is true for AWS automation. A live AWS smoke test proved the path. Automating that in CI has different tradeoffs: account boundaries, IAM, cleanup, cost, and secrets. That's a real decision, not something to sneak in because a checklist wants another green square.</p>
<h2>The current verdict</h2>
<p>Corsac Phase 2 did what it needed to do.</p>
<p>It closed the real AWS smoke gap. It found and fixed a warm-runtime Lambda bug. It made run inspection more useful. It made Lambda invocation records visible by default. It gave artifacts metadata. It validated the canonical workflow against Gust 0.2.1. It refreshed the fixture. It cleaned up the temporary AWS resources when the smoke test was done.</p>
<p>That's a useful checkpoint.</p>
<p>The project is still intentionally small. But it's a more credible small thing now.</p>
<p>The next question is no longer "can Corsac run a compiled Gust workflow?"</p>
<p>It can.</p>
<p>The next question is whether the right move is to keep hardening the CLI/Lambda wedge or start a separate hosted/control-plane phase.</p>
<p>I don't want to answer that just because a roadmap wants a next box.</p>
<p>The next phase should be shaped by the friction that shows up while using Corsac for real workflows.</p>
]]></content:encoded></item><item><title><![CDATA[Two beliefs that parked my Go rewrite. Both were wrong.]]></title><description><![CDATA[First published in my LinkedIn newsletter. Mousing has since moved to Go in production.
Last year I started a Go/Lambda rewrite of my SaaS API. Got it about 60% feature-complete. Then parked it.
The t]]></description><link>https://blog.narcolepticfox.com/two-beliefs-that-parked-my-go-rewrite-both-were-wrong</link><guid isPermaLink="true">https://blog.narcolepticfox.com/two-beliefs-that-parked-my-go-rewrite-both-were-wrong</guid><category><![CDATA[golang]]></category><category><![CDATA[aws lambda]]></category><category><![CDATA[WebAssembly]]></category><category><![CDATA[dotnet]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Sat, 03 Oct 2026 17:21:10 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/00caac1b-65f4-4339-bc37-ce1f2327a38c.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>First published in my LinkedIn newsletter. Mousing has since moved to Go in production.</em></p>
<p>Last year I started a Go/Lambda rewrite of my SaaS API. Got it about 60% feature-complete. Then parked it.</p>
<p>The two reasons I parked it felt solid at the time:</p>
<ol>
<li>Go's document-rendering libraries are too weak for what I need.</li>
<li>Rewriting all the database adapters by hand will take forever.</li>
</ol>
<p>When I came back to the spike last week, both turned out to be wrong. Not "kind of wrong" — wrong in ways I should have caught earlier if I'd spent another afternoon on each.</p>
<p>This post is about what I was wrong about, what the actual answers were, and the meta-lesson I want to remember next time I'm tempted to park a spike on a single library-shaped objection.</p>
<h2>Belief 1: "Go can't render documents."</h2>
<p>Mousing's core pipeline takes a PDF or DOCX, rasterizes each page to an image, and sends those images to OpenAI's vision API for extraction. On .NET, I use <code>Docnet.Core</code> — a wrapper around the pdfium library Chromium uses to render PDFs.</p>
<p>When I looked at Go's options the first time, I saw <code>go-fitz</code> (a MuPDF binding) and immediately had two problems:</p>
<ol>
<li>It uses cgo, which means I can't build a static Go binary. Lambda deploys get pushed from "upload a zip" to "build and push a container image" — a step up in operational complexity I didn't want.</li>
<li>MuPDF is AGPL-licensed. For commercial SaaS, AGPL is a serious legal problem — anyone interacting with the service over a network can demand source code.</li>
</ol>
<p>That was enough to park the spike. I didn't keep looking.</p>
<p>What I missed: <code>github.com/klippa-app/go-pdfium</code> exists, has a WebAssembly backend that runs <strong>the same pdfium engine .NET uses</strong>, via pure-Go wazero — no cgo. Static binary. BSD-licensed. Bundled <code>pdfium.wasm</code> ships with the package.</p>
<p>The actual implementation:</p>
<ul>
<li>Rasterize at ~150 DPI (matches .NET's default; 300 DPI quadruples your vision-token cost for printed text and you don't get meaningfully better OCR)</li>
<li>The wasm instance is single-threaded — initialize once, serialize access via a mutex-protected singleton or a small pool</li>
</ul>
<p>That's it. The whole "Go can't render documents" objection dissolved in about an hour of looking again.</p>
<p>The pattern worth keeping: <strong>when porting a .NET imaging or media pipeline to Go, before concluding "Go can't," look for a wazero-backed binding of the same underlying native library.</strong> wazero is mature enough now that this is increasingly the answer to "how do I get this C/C++ library into a static Go binary."</p>
<h2>Belief 2: "Hand-writing all the database adapters will take forever."</h2>
<p>Each feature in the .NET stack hits the database directly via Entity Framework. The Go spike's handlers had been written assuming each feature would declare a <em>local</em> <code>DB interface</code> with its own row and parameter types — but the actual SQL execution layer underneath that interface was never wired up.</p>
<p>I had a mental model that filling in all 15 features' adapter layers would be tedious sequential work — generate the queries, write the mapping, plumb it through, repeat 15 times.</p>
<p>The actual pattern that worked:</p>
<p><strong>One</strong>: Generate the SQL execution code once with <code>sqlc</code>, configured so nullable columns become standard Go pointer types (<code>*string</code>, <code>*uuid.UUID</code>) instead of sqlc's default wrapper types. That one config tweak removed most of the type-conversion friction that would otherwise be everywhere.</p>
<p><strong>Two</strong>: One shared helper package with the small conversion utilities every adapter needs — context handling, "not found" checks, nullable conversions. Every feature's adapter reuses it instead of re-deriving these from scratch.</p>
<p><strong>Three</strong>: Per feature, a single file in that feature's own package. That file holds a small store type wrapping the generated queries, plus the mapping between generated row types and the local types the feature exposes.</p>
<p>Because each feature owns its own adapter file in its own package, every feature is <em>parallelizable</em>. I fanned out 15 features to subagents working in parallel. Zero merge conflicts. The only files I owned as the lead were the shared ones: the router, the composition root, the helper package.</p>
<p>That took a day. Not a week.</p>
<p>The pattern worth keeping: <strong>make your per-feature adapter layer live in the feature's own package, not in a shared infrastructure layer.</strong> Shared infra layers serialize the work — everyone's editing the same files. Per-feature adapter packages parallelize it.</p>
<h2>The other things that bit me</h2>
<p>Two smaller gotchas worth flagging for anyone doing similar work:</p>
<p><strong>The chi-double-mount panic.</strong> The Go router I'm using, <code>chi</code>, panics on startup if you accidentally register the same path prefix in two places via its <code>Route</code> and <code>Mount</code> patterns. Caught me with a <code>/billing</code> mount where the Stripe webhook (no auth) needed to be split out from the authenticated billing routes. The route registration looked clean to the eye but crashed on every cold start. The fix is to register unauthenticated paths explicitly on the root router and put authenticated ones inside the group — never mount the same prefix twice. Easy to miss until it crashes live.</p>
<p><strong>Batch tracking doesn't survive Lambda.</strong> The .NET stack tracks in-flight batch extractions in an in-memory dictionary. Fine on a long-lived server. On Lambda, every invocation is potentially a fresh container — that dictionary doesn't persist. The right fix wasn't to fake an in-memory cache on Lambda. It was to add a <code>BatchId</code> column to the database and look up batch state there. This isn't a parity violation. It's the correct serverless adaptation.</p>
<p><strong>One binary, two run modes.</strong> A small thing that paid off: a single Go binary that becomes a Lambda handler when <code>AWS_LAMBDA_FUNCTION_NAME</code> is set in the environment, and a normal HTTP server otherwise. Same code path, different entry point. Local development runs the same binary as production deploys, which removes a whole category of "works locally, broken in Lambda" debugging.</p>
<h2>The meta-lesson</h2>
<p>I parked this spike on two objections that, in hindsight, were thin.</p>
<p>The "Go can't render documents" objection was a 30-second search away from being resolved. I didn't do that search because the objection felt solid enough at the time.</p>
<p>The "hand-writing adapters will take forever" objection assumed sequential work. It was actually trivially parallel work that I was modeling sequentially.</p>
<p>Both objections shared a structure: they were beliefs I'd formed during a frustrating first pass at the spike, and they hardened into reasons-not-to-revisit. The next time something I think I want to build feels stuck, the question I want to ask is "<strong>is this stuck because the work is genuinely hard, or because I formed an opinion last time that I haven't tested since?</strong>"</p>
<p>Most of the time, the answer is the second one.</p>
<p>The Mousing Go rewrite still isn't deployed. The migration decision is still about hosting and cost, not technical feasibility. But the spike isn't parked anymore — and the two reasons I parked it in the first place don't apply.</p>
<p>That's the value of unparking. Sometimes the work is easier than your past self decided.</p>
]]></content:encoded></item><item><title><![CDATA[Corsac Phase 1: From Workflow Sketches to a Real Runtime]]></title><description><![CDATA[First published in my LinkedIn newsletter. Corsac is still a work in progress; this describes it as it was then.
Gust is the language side of this project. It describes a workflow as typed states, tra]]></description><link>https://blog.narcolepticfox.com/corsac-phase-1-from-workflow-sketches-to-a-real-runtime</link><guid isPermaLink="true">https://blog.narcolepticfox.com/corsac-phase-1-from-workflow-sketches-to-a-real-runtime</guid><category><![CDATA[golang]]></category><category><![CDATA[aws lambda]]></category><category><![CDATA[workflow]]></category><category><![CDATA[cli]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Tue, 29 Sep 2026 02:14:22 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/403ce793-5bb3-4bbb-a2fa-be7850a70681.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>First published in my LinkedIn newsletter. Corsac is still a work in progress; this describes it as it was then.</em></p>
<p>Gust is the language side of this project. It describes a workflow as typed states, transitions, handlers, effects, and actions.</p>
<p>Corsac is the operating layer around that contract. It's the thing that makes a compiled workflow usable by an operator: validate it, build it, run it, inspect what happened, repair stale runs, and eventually package it for a deployment target.</p>
<p>Phase 1 had one job: cross the line from architecture document to actual runtime.</p>
<p>Not a hosted product. Not a visual designer. Not a full workflow platform. The goal was narrower and more useful — prove that compiled Gust workflows can be operated from a local CLI with enough structure around configuration, durability, side effects, and packaging that the next phase has something real to build on.</p>
<p>The short version: Phase 1 is complete for local CLI/runtime use. The main external gap left is a real AWS Lambda smoke test against an actual AWS account. The deploy and invoke commands exist and are locally tested with fake AWS CLI helpers, but I'm not going to claim real cloud validation until I've run the canonical workflow through AWS itself. That's the gap I keep walking past, and I'd rather name it than dress it up.</p>
<h2>The wedge</h2>
<p>The accepted Phase 1 wedge was deliberately constrained:</p>
<ul>
<li>CLI-first</li>
<li>Lambda-friendly for AWS-heavy teams</li>
<li>compiled Gust workflows</li>
<li>node-boundary durability</li>
<li>explicit <code>effect</code> versus <code>action</code> semantics</li>
<li>no visual designer</li>
<li>no hosted control plane</li>
</ul>
<p>That constraint mattered. Workflow tools sprawl fast.</p>
<p>It's tempting to jump straight to dashboards, collaboration, hosted run storage, trigger management, and a marketplace of handlers. Those might become useful later. None of them answer the first question: can a typed workflow be compiled and operated reliably from a developer machine?</p>
<p>Phase 1 focused on that question.</p>
<h2>What landed</h2>
<p>The CLI now has the core local workflow lifecycle:</p>
<pre><code class="language-sh">corsac check docs/examples/workflows/order-notification.gu
corsac build docs/examples/workflows/order-notification.gu
corsac run docs/examples/workflows/order-notification.gu \
  --payload @docs/examples/payloads/success.json
corsac history
corsac show-run &lt;execution-id&gt;
corsac repair-runs
</code></pre>
<p>That path is backed by a real runtime boundary instead of a one-off demo command. There's an app layer, executor layer, local run store, runtime configuration loader, workflow linter, Lambda packaging code, and a test suite around the pieces most likely to regress.</p>
<p>The canonical example is an order notification workflow.</p>
<p>Intentionally ordinary: receive a webhook-shaped payload, parse an order, format a message, call a Slack-like action. That kind of boring example is useful because it forces the runtime to deal with the things production automation actually cares about — payload shape, handler binding, credentials, side effects, failure records, repeatable execution.</p>
<p>It also resists demo inflation. If the runtime can't make a webhook-to-notification workflow inspectable and repeatable, it has no business growing a larger product surface yet.</p>
<h2>Configuration became real</h2>
<p>Early versions of Corsac were wired around local defaults. That was fine while proving the executor, but it wasn't enough for a product-shaped runtime. Phase 1 added <code>corsac.toml</code> as the project configuration file.</p>
<p>The config model is intentionally small:</p>
<ul>
<li>named environments</li>
<li>handler bindings</li>
<li>env-var-backed credentials</li>
<li>strict validation</li>
<li>sanitized config staging for packaged deployments</li>
</ul>
<p>One important cleanup: moving away from plaintext deploy-time secrets. The Lambda deploy path uses <code>--lambda-env-from KEY=ENV_VAR</code> instead of <code>--lambda-env KEY=value</code>, so secrets come from the caller's environment and pass through the AWS CLI without baking secret values into command history or docs examples.</p>
<p>The config migration also moved from an earlier YAML shape to TOML. Not glamorous. The kind of choice that matters anyway for a CLI tool: the checked-in config is easier to read, and strict unknown-field rejection keeps typos from silently changing runtime behavior.</p>
<h2>Side effects needed a runtime rule</h2>
<p>The most important semantic distinction in Corsac Phase 1 is <code>effect</code> versus <code>action</code>.</p>
<p>The language distinction comes from Gust. Corsac needs it for a runtime reason.</p>
<p>The rough model in Phase 1:</p>
<ul>
<li>An <code>effect</code> can do external work that's safe to treat as non-committing or repeatable in the workflow path.</li>
<li>An <code>action</code> is the committed side-effectful step that should happen at most once in a handler path and should appear at the boundary where the workflow transitions onward.</li>
</ul>
<p>This isn't just vocabulary. It's part of the durability model.</p>
<p>Corsac records execution progress at node boundaries. If a run is interrupted, the runtime can reason about the last completed node and repair stale records. But that model only works if the workflow is honest about where committed side effects happen.</p>
<p>Phase 1 therefore added conservative linting around <code>effect</code> and <code>action</code> usage. Today that linting is source-level enforcement, not parser-backed proof from Gust metadata. It can be confused by comments, strings, quoted braces, or future Gust syntax. That's acceptable for Phase 1 because the limitation is explicit and documented. Hidden limitations are the dangerous kind.</p>
<p>The deeper design point deserves its own post (and gets one — see post 5). For Phase 1, the important result is that the runtime now enforces the side-effect rule it depends on instead of treating it as a comment.</p>
<h2>Run records and repair</h2>
<p>A workflow runtime without inspection is just a black box with a nicer command name.</p>
<p>Corsac now stores local execution records under <code>.corsac/runs/</code>. The records include status, failure classification, node progress, and event data sufficient for local inspection and repair. The CLI can list history, show an individual run, and repair runs that were left stale by interruption.</p>
<p>Some less visible hardening landed here:</p>
<ul>
<li>atomic temp-and-rename record writes</li>
<li>synchronized event repair records</li>
<li>per-run repair locks</li>
<li>child process cleanup on protocol errors</li>
<li>stderr draining before returning executor failures</li>
</ul>
<p>None of that's flashy. It's the difference between "the demo worked once" and "I can keep using this while I build the rest of the system."</p>
<h2>Packaging for Lambda</h2>
<p>The Phase 1 product wedge points toward Lambda-heavy AWS teams, so local execution alone wasn't enough. Corsac now has commands for Lambda packaging, deployment, and invocation:</p>
<pre><code class="language-sh">corsac package lambda docs/examples/workflows/order-notification.gu --use-docker
corsac deploy lambda docs/examples/workflows/order-notification.gu \
  --function-name &lt;name&gt; \
  --role &lt;role-arn&gt; \
  --region &lt;region&gt; \
  --lambda-env-from CORSAC_SLACK_TOKEN=CORSAC_SLACK_TOKEN \
  --yes
corsac invoke lambda &lt;function-name&gt; \
  --region &lt;region&gt; \
  --payload @docs/examples/payloads/success.json
</code></pre>
<p>The Windows-friendly path uses Docker to produce a Linux/x86_64 Lambda package. On Linux, the native toolchain path can be used directly. The package zip is written through a temporary path and renamed into place to avoid leaving corrupt artifacts after interrupted builds.</p>
<p>The deploy path also got stricter during release-readiness work. It requires AWS caller identity discovery and explicit <code>--yes</code> confirmation before mutating Lambda resources. It waits between Lambda update operations and uses AWS CLI <code>file://</code> payloads for environment updates.</p>
<p>The caveat is important: this path is implemented and covered with fake AWS CLI helpers, but the real AWS smoke test is still pending.</p>
<p>I'm treating that as external validation, not as a reason to keep expanding Phase 1 indefinitely. The temptation is real — there's always one more polish item — but the smoke test is the actual unblock.</p>
<h2>CI and the Gust fixture choice</h2>
<p>The canonical workflow is now a release-gate acceptance fixture. CI exercises the local <code>check -&gt; build -&gt; run</code> path, including success and failure payloads.</p>
<p>One deliberate choice: the acceptance test uses a pinned generated Rust fixture for the Gust output instead of invoking a freshly installed <code>gust</code> binary in the default CI path.</p>
<p>That tradeoff keeps CI stable while Gust and Corsac are evolving together. The fixture has update instructions, and the backlog includes a live Gust compatibility gate for release validation. For now, the fast test proves Corsac against the pinned compiler output. The live compiler test can become a separate release or manual gate.</p>
<p>This is a good example of the kind of honesty I want in this project. The fixture-backed test is valuable. It isn't the same as proving compatibility with every future Gust change. Saying so is part of the work.</p>
<h2>What I'm not claiming yet</h2>
<p>Phase 1 has a real runtime now, but there are limits I want to keep visible:</p>
<ul>
<li>The AWS smoke test hasn't run against a real account yet.</li>
<li>The <code>effect</code> and <code>action</code> linter is source-level, not parser-backed.</li>
<li>Execution history is local filesystem only.</li>
<li>There's no multi-environment promotion workflow.</li>
<li>Windows Lambda packaging requires Docker.</li>
<li>Full event-sourced replay is deferred.</li>
<li>There's no hosted control plane or visual designer.</li>
</ul>
<p>Some of these are gaps. Some are intentional boundaries. The distinction matters.</p>
<p>For example, local filesystem history isn't a mistake. It's the right Phase 1 choice because it keeps the runtime inspectable without forcing a database or service dependency too early. If Corsac becomes useful for teams, shared history will become a real product requirement. Today it isn't one yet.</p>
<h2>What Phase 2 is likely about</h2>
<p>The Phase 2 backlog is clearer because Phase 1 forced the rough edges into the open.</p>
<p>The likely next items:</p>
<ul>
<li>run the real AWS smoke test</li>
<li>add a live Gust compatibility gate</li>
<li>make <code>corsac show-run</code> default to the latest run when no ID is provided</li>
<li>improve machine-readable history and inspection output</li>
<li>move <code>effect</code> and <code>action</code> validation closer to Gust metadata or a parser</li>
<li>improve generated artifact metadata</li>
<li>make payload fixtures first-class in docs</li>
<li>explore shared execution history</li>
</ul>
<p>The deeper question for Phase 2 is where Corsac should stop being a local runtime and start becoming team infrastructure. I don't want to answer that too early. The next phase should be driven by the pressure that shows up while using the CLI for real workflows — not by what would look impressive in a roadmap.</p>
<h2>The current verdict</h2>
<p>Phase 1 did what it needed to do. Corsac is no longer just an architecture sketch around Gust. It's a working local runtime with a CLI, configuration model, run records, repair path, release docs, tests, and a Lambda packaging/deploy story that's ready for real AWS validation.</p>
<p>That's a useful checkpoint.</p>
<p>The next step isn't to pretend the system is bigger than it is.</p>
<p>The next step is to use it, close the AWS smoke gap, and let the Phase 2 backlog be shaped by actual friction.</p>
]]></content:encoded></item><item><title><![CDATA[Four bugs I would have shipped without a parity harness]]></title><description><![CDATA[First published in my LinkedIn newsletter. Mousing has since moved to Go in production.
Quick note: Gust and Corsac will continue, the next couple of posts cover Mousing, my SaaS, which is what I'm in]]></description><link>https://blog.narcolepticfox.com/four-bugs-i-would-have-shipped-without-a-parity-harness</link><guid isPermaLink="true">https://blog.narcolepticfox.com/four-bugs-i-would-have-shipped-without-a-parity-harness</guid><category><![CDATA[golang]]></category><category><![CDATA[Testing]]></category><category><![CDATA[dotnet]]></category><category><![CDATA[SaaS]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Mon, 28 Sep 2026 01:09:06 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/204e243a-296f-4915-a691-541203bf8c94.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>First published in my LinkedIn newsletter. Mousing has since moved to Go in production.</em></p>
<p><em>Quick note: Gust and Corsac will continue, the next couple of posts cover Mousing, my SaaS, which is what I'm in the code for right now. The series threads pick back up as I'm in those codebases again.</em></p>
<p>Every time you port a serious codebase to a new language, the question is the same:</p>
<p><em>Did I actually match the original behavior?</em></p>
<p>Unit tests don't answer it. They prove your new code is internally consistent. They don't prove it behaves like the old code under real-world traffic.</p>
<p>For Mousing, my document-extraction SaaS, originally built on .NET, I'd been carrying around an unanswered version of that question for a while. There was a parked Go/Lambda rewrite branch from earlier in the year. It was mostly feature-complete and had been sitting there because "Go's libraries are too weak to render documents" felt true. (It wasn't, but that's a different post.)</p>
<p>When I came back to it last week and brought it to functional parity, I needed to know whether "functional parity" actually meant "behaviorally identical."</p>
<p>The cheapest tool that answered that question wasn't a test suite. It was a <strong>differential parity harness</strong>, a docker-compose stack that stands up the Go API and the .NET API side by side, runs the same requests against both, and diffs the responses.</p>
<p>Built in one afternoon. Found four real bugs in the next afternoon. Three of them weren't going to show up in any unit test I'd have written.</p>
<p>This post is about what made the harness cheap, what it caught, and the part that surprised me: a parity diff isn't just a "make them match" tool. It's a product-decision tool.</p>
<h2>What made it cheap to build</h2>
<p>The whole point was building it fast, slow tools never get used.</p>
<p>Three design choices made it work:</p>
<p><strong>Compare at the HTTP layer, never at the database.</strong> The .NET stack owns its schema one way; the Go stack uses raw SQL another way. They've deliberately diverged in a few places. Comparing at the database means fighting that divergence forever. Comparing at the HTTP layer treats both APIs as black boxes, what goes in, what comes out, do they match? Everything else is implementation detail you're not allowed to care about.</p>
<p><strong>One API key authenticates both stacks.</strong> Both APIs hash incoming keys the same way (HMAC-SHA256, lowercase hex). Same secret, same hash, same <code>Authorization: Bearer msk_...</code> header works against either stack. Seed the identical row into both databases, done. No shared auth service, no JWT plumbing, no test-environment fiddling. This was the single biggest enabler.</p>
<p><strong>A diff tool that masks the volatile stuff.</strong> IDs, timestamps, request tokens, these change every run and would create false positives forever. The diff masks them out, normalizes line endings (CRLF in C# string literals vs LF in Go embeds will bite you), and lets you allowlist intentional differences while still failing when something regresses.</p>
<p>The whole thing runs as <code>bash parity/run.sh</code>. About a dozen tests pass deterministically; eight fail by design and are documented as such. The expensive tier, real upload, real OpenAI call, real extraction, real comparison, runs only when you opt in.</p>
<h2>The bugs it caught</h2>
<p>These are the ones I want to be specific about, because the failure modes weren't subtle:</p>
<p><strong>1. The OpenAI version bug.</strong> The Go OpenAI client was sending <code>max_tokens</code> in the request. Fine on older models. gpt-5.x rejects it, returns a 400 with "use <code>max_completion_tokens</code> instead." Every extraction in the Go stack was silently failing.</p>
<p>Unit tests pass. Mock OpenAI passes. The bug only exists when the live model gets the live request. If I'd migrated production traffic to Go without running the live tier of this harness, every customer's extraction would have broken.</p>
<p>This one was the single most valuable find. There's no way to test for it short of actually making the call.</p>
<p><strong>2. Pagination envelope drift.</strong> The Go list responses returned the right data but were missing the <code>hasNextPage / hasPreviousPage / totalPages</code> fields in the envelope. Frontend would have rendered broken pagination UI. Fixed in one place, the shared response wrapper, without touching any endpoint code.</p>
<p><strong>3.</strong> <code>/user/me</code> <strong>ignored API-key authentication.</strong> When you call <code>/user/me</code> with an API key (not a JWT), the Go stack only looked for a JWT subject claim, found nothing, and resolved to a null user. The .NET stack correctly resolves to the user who created the API key. Fixed by routing API-key auth through the same user-resolution path the .NET stack already used.</p>
<p><strong>4. .NET needs Redis. Go doesn't.</strong> The .NET extraction enqueue uses a real-time messaging backplane that talks to Redis. The harness initially had no Redis service. .NET returned 500 on every extraction request. Adding Redis to docker-compose fixed it. Go uses no Redis for the same operation, a quiet architectural difference the diff surfaced.</p>
<p>Four bugs. Three are runtime issues that compile and test green. Without a live, side-by-side comparison, every one of them would have shipped.</p>
<h2>The product-decision surprise</h2>
<p>Here's the part I didn't expect.</p>
<p>When the harness fails on a diff, the question isn't always "make them match." Sometimes the question is "<strong>which one is right?</strong>"</p>
<p>Specifically: the <code>/tenant/settings</code> endpoint was exposing <code>defaultAIProvider</code>, <code>hasOpenAIKey</code>, <code>hasAnthropicKey</code>. The .NET stack returned them; the Go stack didn't. Parity says match, drop them on the .NET side or add them to the Go side.</p>
<p>I looked at it and realized the AI provider is supposed to be an internal implementation detail. The frontend shouldn't be making decisions based on which provider is in use. Exposing it was a leftover from earlier exploration where I thought users would care about that.</p>
<p>The diff didn't just surface a bug. It surfaced a product question I'd been carrying without noticing.</p>
<p>The right answer was to drop the fields on both sides. Make the API more correct, not just consistent.</p>
<p>A parity harness ends up being a forcing function. Every diff is a chance to ask "is this the way it <em>should</em> be?" not just "is this the way it <em>is</em>?"</p>
<h2>Where this leaves Mousing</h2>
<p>The Go rewrite isn't deployed. The harness proves the two stacks now behave identically for the customer-facing API surface. That's a milestone, not a decision.</p>
<p>The real question, whether to ship Go to production or stay on .NET, is about hosting cost and operational complexity, not parity. The harness took that question off the table cleanly. I know exactly where each implementation diverges, what's intentional, and what isn't. The migration call gets made on real grounds instead of on "did I actually match the original?" anxiety.</p>
<p>If you're building a SaaS and considering a rewrite, the technique generalizes:</p>
<ul>
<li><p>Compare at the HTTP layer, not the database.</p>
</li>
<li><p>Hash auth the same way on both sides so one credential works for both.</p>
</li>
<li><p>Mask the volatile fields. Allow intentional diffs. Fail on regressions.</p>
</li>
<li><p>Build the harness <em>before</em> you commit to the migration, not after.</p>
</li>
</ul>
<p>The bugs you find won't be the ones you expected.</p>
<p>That's the point.</p>
<hr />
<p><em>Next post: the two beliefs that parked this Go spike for half a year in the first place, and why both turned out to be wrong.</em></p>
]]></content:encoded></item><item><title><![CDATA[Why FoxWatch opens to nothing]]></title><description><![CDATA[I just wrapped Phase 0 of FoxWatch, a free PC health monitor I'm building, and the milestone is kind of funny to type out: the app now successfully does nothing.
You launch it and there's no window. J]]></description><link>https://blog.narcolepticfox.com/why-foxwatch-opens-to-nothing</link><guid isPermaLink="true">https://blog.narcolepticfox.com/why-foxwatch-opens-to-nothing</guid><category><![CDATA[foxwatch]]></category><category><![CDATA[Rust]]></category><category><![CDATA[Tauri]]></category><category><![CDATA[Build In Public]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Wed, 29 Jul 2026 11:26:07 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/2eb9c712-3313-47bf-a7fb-ce5749a0a9f8.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I just wrapped Phase 0 of FoxWatch, a free PC health monitor I'm building, and the milestone is kind of funny to type out: the app now successfully does <em>nothing</em>.</p>
<p>You launch it and there's no window. Just a little fox in your system tray. Click it and you get a dashboard that basically says "everything's fine." That's it. That's the whole Phase 0 deliverable, and honestly I think it's the most important call in the entire project.</p>
<p>Let me explain why I'm weirdly proud of an app that opens to a blank slate.</p>
<h2>The whole idea</h2>
<p>Every PC monitor out there shows you the same stuff: CPU graph, memory graph, disk gauge, a wall of temperatures. And here's the thing. Collecting all that is <em>free</em>. The OS just hands it to you. There are a dozen libraries that wrap it in an afternoon.</p>
<p>So if the data's free, the data can't be the product. Right?</p>
<p>The actual product is the part that decides <em>what's worth bugging you about</em>, and 99% of the time the honest answer is "nothing, your machine is fine." That's the bet FoxWatch is making. Not "here's another dashboard to check." More like a quiet watcher that earns the right to interrupt you exactly once, when your SSD is actually dying, and stays invisible every other day.</p>
<p>Once I framed it that way, a lot of decisions got easy. FoxWatch isn't something you open. It's something you forget about until it taps you on the shoulder.</p>
<p>And honestly that's the part I need, personally. My brain (ADHD, plus a few years of a military always-be-scanning habit) does not need one more thing to half-watch in the background. I want something that's got my six. Something quietly holding the watch so I don't have to carry it around in my head. Less to track, less load. For me that <em>is</em> the feature, more than any graph could be.</p>
<h2>So yeah, the first feature is being invisible</h2>
<p>Which is why Phase 0 is deliberately an app that opens to nothing:</p>
<ul>
<li><strong>Lives in the tray, not the taskbar.</strong> The tray icon color <em>is</em> the UI. Green means ignore, amber means "eh, worth a look, no rush," red means deal with this now. You read your machine's health with a glance and never open anything.</li>
<li><strong>Closing it hides it, doesn't quit it.</strong> Hit the X and it tucks back into the tray and keeps running. An always-on monitor you can accidentally close by clicking X isn't really always-on.</li>
<li><strong>Starts with Windows, straight to the tray.</strong> No window flashing on login, it's just quietly there.</li>
<li><strong>The dashboard default is "you're good."</strong> Not a grid of gauges, just "everything's healthy, nothing needs you," plus a little sleep log: how long the fox has been napping. (Mine's on a 2-day nap as I write this.)</li>
</ul>
<p>None of that was technically hard. The hard part was not filling the empty space. Every builder instinct I have says <em>put something on the screen</em>. But the rule I set is: if it doesn't map to something you'd actually do, it doesn't get shown. The empty screen is the feature.</p>
<h2>What's actually under it</h2>
<p>Phase 0's a <a href="https://tauri.app">Tauri v2</a> shell: Rust core, Svelte frontend running in the native WebView. I went Tauri specifically because I didn't want to ship a whole browser like Electron would; an always-on watcher needs to be tiny. Tray, hide-on-close, autostart, all wired and working.</p>
<p>The part I'm quietly happy about: the triage engine is <em>already in there</em>. I'd written it as a spike before committing to the architecture, and it's a pure function (metrics + history in, decisions out, no I/O, no reading the clock) so I can test the hell out of it. It's sitting in the core right now with literally nothing to do. Asleep, basically. Which, for a thing built by Narcoleptic Fox, feels about right... it looks like it's dozing, but it's wired to snap awake the second you need it. Phase 1 gives it real data. Phase 2 is where it actually wakes up.</p>
<p>Also, real talk: I nearly nuked the whole design folder this week. Ran a scaffolder in a folder that wasn't empty and it happily wrote over my stuff. Got it back, moved the project somewhere clean, and left myself a big angry warning in my own notes. Building in public means posting the dumb ones too.</p>
<h2>The thing I had to get comfortable with</h2>
<p>An app whose goal is "the user never opens it" is a strange thing to build. There's no engagement number to chase. If it's doing its job you forget it exists for weeks, and then one random Tuesday it goes <em>"hey, your drive's failing, back up today,"</em> — and that one correct interruption is worth more than a thousand dashboard opens.</p>
<p>So Phase 0 opening to nothing isn't missing features. It's the whole personality showing up in the first ten seconds: quiet, private, out of your way, only speaks when it matters.</p>
<h2>Next up</h2>
<p>Phase 1 is where it starts breathing. A <code>sysinfo</code> collector feeding real samples into local SQLite, and the dashboard finally showing your actual machine instead of a placeholder. Still quiet, but now quiet about something real.</p>
<p>More soon.</p>
]]></content:encoded></item><item><title><![CDATA[States, Transitions, Effects: What a `.gu` File Actually Buys]]></title><description><![CDATA[The previous post made the case for Gust as a workflow contract.
This one is more concrete.
Abstract language posts are cheap. The more useful question: what does a .gu file buy that ordinary code doe]]></description><link>https://blog.narcolepticfox.com/states-transitions-effects-what-a-gu-file-actually-buys</link><guid isPermaLink="true">https://blog.narcolepticfox.com/states-transitions-effects-what-a-gu-file-actually-buys</guid><category><![CDATA[workflow]]></category><category><![CDATA[gust]]></category><category><![CDATA[state-machines]]></category><category><![CDATA[codegen]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Wed, 20 May 2026 18:12:43 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/3945d886-c70e-4682-ace1-26f023b5f983.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The previous post made the case for Gust as a workflow contract.</p>
<p>This one is more concrete.</p>
<p>Abstract language posts are cheap. The more useful question: what does a <code>.gu</code> file buy that ordinary code doesn't?</p>
<p>Right now the answer is:</p>
<ul>
<li>one place to read the workflow contract</li>
<li>explicit state data</li>
<li>enumerated transitions</li>
<li>generated host-language boundary code</li>
<li>declared side effects</li>
<li>a cleaner runtime integration point</li>
</ul>
<p>That's the practical value.</p>
<p>Gust isn't useful because it has syntax. It's useful only if the syntax preserves information the rest of the system needs.</p>
<p>Workflow code has been relying on convention for decades. The <code>.gu</code> file is what stops being conventional.</p>
<h2>A workflow file is a contract</h2>
<p>Here's the declaration portion of the order notification workflow Corsac uses as its canonical example (handler bodies come later):</p>
<pre><code class="language-gust">type OrderPayload {
    order_id: String,
    customer: String,
    total_cents: i64,
    currency: String,
    items_count: i64,
}

machine OrderNotificationWorkflow {
    state Idle
    state WebhookReceived(body: String, source_ip: String)
    state OrderParsed(order: OrderPayload, original_body: String)
    state MessageFormatted(order: OrderPayload, slack_text: String, original_body: String)
    state NotificationSent(order_id: String, slack_ts: String)
    state Failed(step: String, reason: String, original_body: String)
    state DeadLettered(original_body: String, attempts: i64)

    transition receive: Idle -&gt; WebhookReceived
    transition parse: WebhookReceived -&gt; OrderParsed | Failed
    transition format: OrderParsed -&gt; MessageFormatted | Failed
    transition notify: MessageFormatted -&gt; NotificationSent | Failed
    transition retry: Failed -&gt; WebhookReceived | DeadLettered
    transition reset: NotificationSent -&gt; Idle
    
    effect parse_order_json(body: String) -&gt; OrderPayload
    effect format_slack_message(order: OrderPayload) -&gt; String
    action post_slack(channel: String, text: String, credential_id: String) -&gt; String
    effect log_failure(step: String, reason: String) -&gt; ()
    effect compute_retry_eligible(step: String, attempt: i64) -&gt; bool
}
</code></pre>
<p>Even before reading any handler bodies, this file says a lot.</p>
<p>It says the workflow has an <code>Idle</code> state. It says a webhook carries <code>body</code> and <code>source_ip</code>. It says a parsed order carries the parsed order and the original body. It says a failed workflow can retry back to <code>WebhookReceived</code> or move to <code>DeadLettered</code>.</p>
<p>That information often exists in regular code. It's just rarely this visible.</p>
<h2>What each state actually carries</h2>
<p>One thing I like about state machines is that they force a basic question: what data exists at this point?</p>
<p>In a normal implementation, the answer can be muddy. A function might receive a large context object. A database row might have nullable columns for every step. A job payload might accumulate fields over time. By month six, the answer to "what data exists at this point" is "let me grep for it."</p>
<p>In a Gust machine, each state declares its own data:</p>
<pre><code class="language-gust">state WebhookReceived(body: String, source_ip: String)
state OrderParsed(order: OrderPayload, original_body: String)
state MessageFormatted(order: OrderPayload, slack_text: String, original_body: String)
</code></pre>
<p>Data threading is now visible.</p>
<p>The <code>original_body</code> field isn't glamorous, but it matters. It exists because retry needs to reconstruct <code>WebhookReceived</code>.</p>
<p>That's the kind of detail that disappears when the workflow is just a pile of handlers.</p>
<p>With explicit state data, the workflow contract tells the runtime what can be serialized, inspected, resumed, and displayed.</p>
<h2>Transitions, not status strings</h2>
<p>The transition list answers another important question: where can the workflow go next?</p>
<pre><code class="language-gust">transition parse: WebhookReceived -&gt; OrderParsed | Failed
transition format: OrderParsed -&gt; MessageFormatted | Failed
transition notify: MessageFormatted -&gt; NotificationSent | Failed
transition retry: Failed -&gt; WebhookReceived | DeadLettered
</code></pre>
<p>This is the difference between status as a string and state as a contract.</p>
<p>If status is just a string, a caller can write any value it wants. The runtime might not know the value is invalid until much later. If transitions are declared, the compiler and generated code can reject movement that doesn't fit the machine.</p>
<p>That doesn't solve every runtime problem. It removes a class of accidental states — the ones that get created by a careless update and discovered by a customer.</p>
<h2>What handlers make first-class</h2>
<p>Handlers attach logic to transitions:</p>
<pre><code class="language-gust">on format(ctx: OrderParsedCtx) {
    let text = perform format_slack_message(ctx.order);
    goto MessageFormatted(ctx.order, text, ctx.original_body);
}

on notify(ctx: MessageFormattedCtx) {
    let ts = perform post_slack("#orders", ctx.slack_text, "cred-slack-prod");
    goto NotificationSent(ctx.order.order_id, ts);
}
</code></pre>
<p>This is where the workflow moves from declaration to behavior.</p>
<p>The handler says what data it reads from the current state, what side effects it performs, and where it transitions next.</p>
<p>The important part isn't that this is shorter than Rust or Go. Sometimes it is, sometimes it isn't. The important part is that the workflow-specific pieces are first-class:</p>
<ul>
<li><code>perform</code> names a declared effect or action</li>
<li><code>goto</code> names a declared target state</li>
<li><code>ctx</code> is tied to the source state of the transition</li>
</ul>
<p>Those are workflow concepts. They should be visible in the workflow source.</p>
<h2>Effects define the host boundary</h2>
<p>Gust doesn't implement the actual Slack call. It declares that the machine needs one:</p>
<pre><code class="language-gust">action post_slack(channel: String, text: String, credential_id: String) -&gt; String
</code></pre>
<p>The generated host-language code then exposes a boundary the runtime must implement.</p>
<p>In Rust, this becomes an effect trait shape. In Go, it becomes an interface shape. The exact generated code varies by target, but the idea is the same: the workflow contract names the side-effectful call and the host application provides the implementation.</p>
<p>This is one of the main reasons I prefer code generation here.</p>
<p>The generated code is boring boundary code: state shapes, transition methods, invalid-transition errors, serialization, effect interfaces, target-language type mappings. The kind of code I want to exist, but don't want to hand-maintain across every workflow.</p>
<p>The <code>.gu</code> file stays focused on the workflow contract. The generated code does the repetitive host-language work.</p>
<p>Two snippets from other Gust machines show the shape. (The order notification workflow's full generated code runs to several hundred lines — same patterns, too long to inline.) Here's a Rust effect trait that preserves the side-effect categories as code comments:</p>
<pre><code class="language-rust">pub trait WorkflowEngineEffects {
    /// gust:effect -- replay-safe / idempotent
    fn execute_step(&amp;self, step_name: &amp;str) -&gt; String;
    /// gust:effect -- replay-safe / idempotent
    fn needs_approval(&amp;self, step_name: &amp;str) -&gt; bool;
    /// gust:effect -- replay-safe / idempotent
    fn next_step_name(&amp;self, current_step: &amp;str) -&gt; String;
    /// gust:effect -- replay-safe / idempotent
    fn produce_failure(&amp;self, reason: &amp;str) -&gt; EngineFailure;
    /// gust:action -- not replay-safe / externally visible
    fn notify_rejection(&amp;self, step_name: &amp;str, reason: &amp;str) -&gt; String;
}
</code></pre>
<p>And a Go target that becomes the state and handler boundary you'd otherwise maintain by hand:</p>
<pre><code class="language-go">type OrderProcessorEffects interface {
    // gust:effect -- replay-safe / idempotent
    CalculateTotal(order Order) Money
    // gust:effect -- replay-safe / idempotent
    ProcessPayment(total Money) Receipt
    // gust:effect -- replay-safe / idempotent
    CreateShipment(order Order) string
}

func (m *OrderProcessor) Charge(effects OrderProcessorEffects) error {
    if m.State != OrderProcessorStateValidated {
        return &amp;OrderProcessorError{Transition: "charge", From: m.State.String()}
    }

    receipt := effects.ProcessPayment(m.ValidatedData.Total)
    m.State = OrderProcessorStateCharged
    // state data update omitted
    return nil
}
</code></pre>
<p>That's the payoff: the source contract turns into the host-language shapes a runtime or application can actually call.</p>
<p>The <code>action</code> keyword compiles to the same host-language shape as <code>effect</code> today, but it means something different to a runtime. The <code>gust:action</code> marker says this call is externally visible — not safe to repeat on replay. Gust enforces a small but meaningful safety rule around it: a handler shouldn't perform more than one action, and the action should be the last side-effectful step before the transition. I'll go deeper on that boundary in a later post. For this one, the point is that the <code>.gu</code> file gives a runtime a signal that handwritten code wouldn't carry.</p>
<h2>The compiler becomes a workflow reviewer</h2>
<p>The compiler can catch simple mistakes that are easy to miss in hand-written workflow code:</p>
<ul>
<li>a transition points at an unknown state</li>
<li>a handler goes to a state that isn't a declared target</li>
<li>a <code>perform</code> call has the wrong number of arguments</li>
<li>a <code>goto</code> passes the wrong number of fields</li>
<li>a handler has code paths that don't end in a transition</li>
<li>an action is followed by more side-effectful work</li>
</ul>
<p>That doesn't replace tests. It doesn't replace runtime validation. It gives the edit loop a better reviewer than convention alone.</p>
<h2>What the <code>.gu</code> file buys</h2>
<p>A <code>.gu</code> file is useful if it becomes the shared source of truth for multiple consumers:</p>
<ul>
<li>the compiler</li>
<li>generated Rust or Go code</li>
<li>runtime dispatch</li>
<li>validation tooling</li>
<li>diagrams</li>
<li>schema generation</li>
<li>eventually a visual designer</li>
</ul>
<p>That's the bigger point.</p>
<p>I don't want a workflow definition, a UI graph, a worker implementation, a runtime config file, and a docs page each inventing their own version of the workflow. I want the typed contract to sit underneath those surfaces.</p>
<p>That's what Gust is trying to provide.</p>
<p>Not magic. Not a general platform. A compact workflow contract that produces the boring code and metadata the rest of the system needs.</p>
<p>The next question is whether that contract can actually be operated. That's where Corsac begins. </p>
]]></content:encoded></item><item><title><![CDATA[Why I'm Building a Typed Workflow Language Instead of Writing Glue Code]]></title><description><![CDATA[Post 1 was the diagnosis: workflow logic keeps wanting to be more structured than ordinary glue code, but less heavy than a full workflow platform. The contract is hiding in the implementation, and th]]></description><link>https://blog.narcolepticfox.com/why-i-m-building-a-typed-workflow-language-instead-of-writing-glue-code</link><guid isPermaLink="true">https://blog.narcolepticfox.com/why-i-m-building-a-typed-workflow-language-instead-of-writing-glue-code</guid><category><![CDATA[Rust]]></category><category><![CDATA[workflow]]></category><category><![CDATA[Go Language]]></category><category><![CDATA[gust]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Wed, 13 May 2026 13:00:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/e76ad6f5-ddc3-4340-b57f-51c0bdc5a49c.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Post 1 was the diagnosis: workflow logic keeps wanting to be more structured than ordinary glue code, but less heavy than a full workflow platform. The contract is hiding in the implementation, and that's where the production incidents come from.</p>
<p>This one is about what should replace it.</p>
<p>I didn't want to start by building a control plane. I wanted the map first.</p>
<p>A way to describe a workflow as a contract:</p>
<ul>
<li>these are the states</li>
<li>these are the transitions</li>
<li>this is the data each state carries</li>
<li>these are the side effects</li>
<li>these are the committed actions</li>
<li>this is the code the host application must provide</li>
</ul>
<p>Then I wanted that contract to compile into boring, ordinary code.</p>
<p>That's the shape of Gust: a typed state-machine language that currently compiles to Rust and Go.</p>
<h2>From code path to contract</h2>
<p>A small workflow can start as a straightforward code path:</p>
<pre><code class="language-go">func HandleOrder(ctx context.Context, payload Payload) error {
	order, err := parseOrder(payload.Body)
	if err != nil {
		return markFailed("parse", err)
	}

	message := formatSlackMessage(order)

	if err := postSlack(ctx, message); err != nil {
		return markFailed("notify", err)
	}

	return markComplete(order.ID)
}
</code></pre>
<p>There's nothing obviously wrong with this function. For a small feature, it might be exactly the right code.</p>
<p>But as the workflow grows, the contract gets harder to see.</p>
<p>Where are the states? Which errors can retry? What data exists after parse but before notify? Is <code>postSlack</code> safe to repeat? If the process dies after posting but before marking complete, what should happen? Can a caller skip directly from received to notified? Does the UI know the same states the worker knows?</p>
<p>You can answer those questions with discipline, tests, comments, and docs. Those answers live outside the workflow itself. Six months later they live in someone's head. A year later they don't live anywhere.</p>
<p>Gust exists because I want the workflow contract to carry those answers directly.</p>
<h2>Why not just JSON or YAML?</h2>
<p>One possible answer is a declarative graph format. Put the states and edges in JSON or YAML, then let a runtime interpret the graph.</p>
<p>That can work. It's also the direction many workflow tools naturally take.</p>
<p>The tradeoff is that a graph file usually becomes another runtime input rather than a typed contract. The interesting behavior moves somewhere else: custom handler code, embedded expressions, loosely typed payloads, stringly named effects, runtime-only validation. You end up debugging a graph by guessing what the runtime did with it, which is the exact problem we were trying to escape.</p>
<p>Gust is trying to keep the contract closer to code without making the workflow disappear into ordinary application functions. The workflow stays explicit enough to validate and generate from, but it compiles into host-language code that Rust and Go projects can own.</p>
<p>That's the middle ground I want. More structure than glue code. Less platform gravity than a full interpreted workflow engine.</p>
<h2>What a Gust machine says explicitly</h2>
<p>Here's what that looks like in source. A Gust workflow is a state machine:</p>
<pre><code class="language-gust">type OrderPayload {
    order_id: String,
    customer: String,
    total_cents: i64,
    currency: String,
    items_count: i64,
}

machine OrderNotificationWorkflow {
    state Idle
    state WebhookReceived(body: String, source_ip: String)
    state OrderParsed(order: OrderPayload, original_body: String)
    state MessageFormatted(order: OrderPayload, slack_text: String, original_body: String)
    state NotificationSent(order_id: String, slack_ts: String)
    state Failed(step: String, reason: String, original_body: String)

    transition receive: Idle -&gt; WebhookReceived
    transition parse: WebhookReceived -&gt; OrderParsed | Failed
    transition format: OrderParsed -&gt; MessageFormatted | Failed
    transition notify: MessageFormatted -&gt; NotificationSent | Failed

    effect parse_order_json(body: String) -&gt; OrderPayload
    effect format_slack_message(order: OrderPayload) -&gt; String
    action post_slack(channel: String, text: String, credential_id: String) -&gt; String
}
</code></pre>
<p>That file isn't an implementation detail. It's the contract.</p>
<p>It says which states exist. It says which transitions are allowed. It says what data is available in each state. It says which side-effectful calls the host runtime must implement. It says which operation is an externally visible action.</p>
<p>Handler bodies fill in the transition logic, but the shape above is already useful. The compiler can validate pieces of that contract before the runtime ever sees it.</p>
<h2>Why compile instead of interpret?</h2>
<p>I didn't want the first version of Gust to be a giant runtime.</p>
<p>That matters. Runtimes attract scope. Once the runtime owns everything, every design question becomes a platform question. Every feature request becomes a hosted product feature request. That's a different project, with a different burn rate, and I'm not ready to commit to that project yet.</p>
<p>The current approach is deliberately conservative: Gust compiles <code>.gu</code> source into host-language code. Rust output gets enums, machine structs, transition methods, effect traits, and serialization support. Go output gets state constants, state data structs, transition methods, interfaces, and JSON tags.</p>
<p>That keeps the generated code close to the systems I already want to run. It also makes Gust useful as a companion language rather than a replacement for Rust or Go. The host application still owns real I/O, credentials, deployment, observability, and business integrations. Gust owns the workflow contract and the generated state-machine layer.</p>
<p>That split matters. It keeps the language smaller and makes adoption less all-or-nothing. Pulling Gust out later — if it doesn't earn its keep — should be a deletion, not a migration.</p>
<h2>Effects are part of the interface</h2>
<p>The most important part of the language might not be the state syntax. It's the effect boundary.</p>
<p>In Gust, side effects are declared:</p>
<pre><code class="language-gust">effect parse_order_json(body: String) -&gt; OrderPayload
effect format_slack_message(order: OrderPayload) -&gt; String
action post_slack(channel: String, text: String, credential_id: String) -&gt; String
</code></pre>
<p>The generated code requires the host runtime to provide implementations for those calls.</p>
<p>Which means the workflow doesn't secretly talk to Slack, the database, or an HTTP API. The contract names the call. The runtime binds it.</p>
<p>The newer <code>action</code> distinction exists for workflow runtimes like Corsac. A runtime can treat normal <code>effect</code> calls as replay-safe or non-committing, while treating <code>action</code> calls as externally visible work that must not be repeated casually.</p>
<p>That's the kind of contract glue code usually leaves implicit. Implicit is fine until the day a payment gets charged twice and someone has to explain it on a call.</p>
<h2>What the compiler can catch</h2>
<p>Gust is at v0.2 — still young — but the compiler already applies useful validation pressure across three areas:</p>
<ul>
<li><strong>Structure:</strong> duplicate state and transition names, unknown states in transitions, unreachable states.</li>
<li><strong>Flow:</strong> missing handlers, invalid <code>goto</code> targets, mismatched <code>goto</code> argument counts, branch termination warnings.</li>
<li><strong>Safety:</strong> effect argument arity, selected expression type mismatches, action handler-safety warnings.</li>
</ul>
<p>None of that makes workflow bugs impossible. But it moves basic contract mistakes out of the runtime path and into the edit/build loop.</p>
<p>That's the point.</p>
<p>I don't want to discover during a production run that a transition targets a state nobody declared, or that a handler can fall through without a state transition, or that a committed action is followed by more side-effectful work in the same path. Those bugs have always been catchable. They just usually weren't caught.</p>
<h2>What Gust is and isn't</h2>
<p>Gust isn't a hosted workflow product. It isn't a queue, scheduler, visual designer, deployment system, or run database. Those are runtime and product concerns. Corsac is where I'm exploring some of that operational surface.</p>
<p>It also isn't trying to replace Rust or Go. The host language still matters. The generated code should feel like code you could have written, but didn't want to maintain by hand. A small language has to justify every feature it adds, because every feature is one more thing to teach, document, and not regret.</p>
<p>The long-term question is how far Gust should go. It can stay a focused state-machine DSL and still be valuable. The current roadmap leaves room for a broader direction — free functions, stronger type checking, contracts, effect polymorphism, more owned semantics instead of passing work to target-language compilers — but I don't want to rush that. The useful thing right now is narrower: prove that typed workflow contracts can generate real code, support real tooling, and give a runtime like Corsac enough structure to operate compiled workflows.</p>
<p>That's why I'm building a typed workflow language instead of writing more glue code. The glue code still exists. It just moves to the boundary where it belongs: effect implementations, runtime adapters, deployment scripts, product-specific integrations.</p>
<p>The next post walks through what a <code>.gu</code> file actually buys, line by line.</p>
]]></content:encoded></item><item><title><![CDATA[The Workflow Problem That Made Me Stop Trusting Glue Code]]></title><description><![CDATA[A few years ago I watched a webhook handler charge a customer's card twice in the same minute. The success path committed. The retry path committed. They'd been written six months apart by different e]]></description><link>https://blog.narcolepticfox.com/the-workflow-problem-that-made-me-stop-trusting-glue-code</link><guid isPermaLink="true">https://blog.narcolepticfox.com/the-workflow-problem-that-made-me-stop-trusting-glue-code</guid><category><![CDATA[Rust]]></category><category><![CDATA[Go Language]]></category><category><![CDATA[architecture]]></category><category><![CDATA[workflow]]></category><dc:creator><![CDATA[Brock Claussen]]></dc:creator><pubDate>Thu, 07 May 2026 00:14:07 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69dbec87aadf1107e21d2c6e/464b1d5a-6535-4f3b-9cdc-ecf62c0287d7.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A few years ago I watched a webhook handler charge a customer's card twice in the same minute. The success path committed. The retry path committed. They'd been written six months apart by different engineers, neither aware the other existed. The fix took an afternoon. The conversation about why the system <em>allowed</em> it took two weeks.</p>
<p>The fix was an idempotency key. The two weeks were about everything else.</p>
<p>I've been writing this kind of code for 20 years, and that incident wasn't the first or the worst. Across consulting gigs, Fortune 500 integrations, and products I've shipped, the same shape keeps showing up. A webhook comes in. A payload gets parsed. We enrich it with a customer record, call an API, post a notification, write a row, enqueue a follow-up job, and handle the error path if something goes wrong.</p>
<p>At first, it's just application code. A function here, a queue consumer there, a few retries, a few logs, maybe a database status column. It doesn't feel like a workflow system. It feels like the practical thing you write because the product needs to move.</p>
<p>That's the trap.</p>
<p>Useful workflows don't stay small. They grow branches. They get approval steps. They need retries. They need cancellation. They need to avoid charging a card twice or posting the same Slack message three times. They need to resume after a deploy, a crash, a timeout, or a handler bug. They need enough history that someone can answer a basic operator question: what happened to this run?</p>
<p>At that point, the workflow is no longer just glue code. It's a state machine. The only question is whether that state machine is explicit, or scattered through conditionals, database fields, queue messages, and side effects. In my experience it's almost always the second one.</p>
<h2>The shape of the bug</h2>
<p>The bugs that bother me most in workflow code aren't clever algorithm bugs.</p>
<p>They're state and boundary bugs:</p>
<ul>
<li>A handler assumes a step already happened, but the persisted state doesn't prove it.</li>
<li>A retry repeats a side effect that should have been committed once.</li>
<li>A failure is stored as a string, so retry policy depends on text that was never meant to be a contract.</li>
<li>A function looks like a pure transform but quietly calls a database, an API, or a notification service.</li>
<li>A workflow can enter a state no caller expected because transitions are just convention.</li>
<li>The UI, worker, API, and deploy target each have a slightly different idea of what the workflow is.</li>
</ul>
<p>I've been in the meeting where I asked "what state can this be in?" and got a 20-minute conversation involving three engineers and a database query.</p>
<p>None of those problems are exotic. That's what makes them annoying. They're the normal failure mode of workflow automation when the workflow model is implicit. The code can look clean locally. Each function reads fine in review. The system-level contract is still informal — and the informal part is exactly where the production incident is going to come from.</p>
<h2>The hidden state machine</h2>
<p>In the Marines, an operations order is a contract. It names the phases of the operation, the conditions to move between them, the abort criteria, what each element is doing in each phase, and what counts as mission complete. It isn't a vibe. It's a document everyone references when things start going sideways. Because things will start going sideways.</p>
<p>A workflow is the same kind of contract. It just doesn't usually look like one.</p>
<p>The double-charge incident I opened with had an implicit state machine. There were states — "received," "processing," "charged," "failed," "retried" — but none of them were named anywhere. They lived in a status column, a queue message, and a handful of conditionals. When the retry path was added, nobody wrote down the rule that "charged" was terminal. Nobody had to. The code worked.</p>
<p>Until it didn't.</p>
<p>If those states had been explicit, the questions would have answered themselves before shipping: What transitions are legal? Which state is terminal? Which step commits an externally visible action? Which failure states can retry?</p>
<p>When states are implicit, you still answer those questions. You just answer them at 2 AM, after the fact, distributed across code paths, database columns, log messages, and the heads of whoever wrote the code three sprints ago.</p>
<p>That's where glue code becomes expensive. Not because functions are bad. Not because JSON is bad. Because the workflow contract is hiding in the implementation instead of being something the implementation follows.</p>
<h2>The side-effect problem</h2>
<p>Workflows are mostly about side effects.</p>
<p>Parsing JSON isn't the hard part. The hard part is deciding what the runtime is allowed to repeat.</p>
<p>There's a real difference between these operations:</p>
<ul>
<li>validate a payload</li>
<li>compute a retry decision</li>
<li>format a message</li>
<li>read a cached record</li>
<li>charge a card</li>
<li>send an email</li>
<li>post to Slack</li>
<li>write to an external system</li>
</ul>
<p>In ordinary glue code, the line between which of those can replay safely and which can't is expressed by naming, comments, or reviewer discipline. Discipline doesn't save you when things go sideways. The runtime needs to know which calls are replay-safe and which calls are externally visible commitments — and it needs to know that without inferring it from a function name.</p>
<p>A workflow runtime needs a side-effect boundary it can reason about. If the contract doesn't say where committed work happens, the runtime is guessing from implementation details, and the customer is the one who finds out.</p>
<h2>The bet</h2>
<p>There are good workflow engines already. I'm not skeptical of them because durable-execution systems are useless. I'm skeptical because most of them put the runtime at the center of gravity. You describe a graph or write code against a framework, and the platform interprets or orchestrates the execution. That's a fine answer for a lot of teams. It isn't the answer I want as an engineer.</p>
<p>I want the workflow contract to be the source of truth, not the runtime.</p>
<p>A workflow should be readable before it's runnable. You should be able to open one file and see the states the workflow can be in, the transitions it allows, the data carried by each state, the side effects it can perform, the committed actions it might take, and the failure shape it exposes to the runtime. Then the generated code and operational tooling should preserve that contract instead of re-inventing it somewhere else.</p>
<p>That's why I stopped trusting workflow glue code as the primary abstraction. Not because glue code is always bad. Because workflows eventually need stronger boundaries than glue code naturally provides — and by the time you notice, you're already paying for it.</p>
<p>The next post is about what I'm building to fix it.</p>
]]></content:encoded></item></channel></rss>