Architecture · 11.04
Snapshots & Crash Recovery
How recoverInterruptedRuns rehydrates mid-flight work at boot, and why snapshots are content-hashed for safe, cheap resume.
A restart is a scheduled event, not a disaster
A long-running workflow periodically snapshots its state so that a process restart — a deploy, a crash, a planned upgrade — rehydrates interrupted work instead of losing it outright. At boot, recoverInterruptedRuns() scans specifically for runs that were mid-flight when the process last stopped, re-arms any parked wait nodes and sessions so they can resume waiting exactly where they left off, and picks back up from the most recent snapshot rather than restarting the whole run from its first node.
Content-hashed, so an unchanged graph skips redundant work
Every snapshot carries a content hash of the workflow graph it was taken against — see Graph Hashing. Resuming against a graph whose hash hasn't changed since the snapshot skips re-validation entirely, because nothing about the structure could have invalidated what was already checked. Resuming against a graph that has changed correctly triggers fresh validation instead of blindly trusting stale evidence — the hash is what lets the system tell those two situations apart automatically.
Continue
The realtime publish bridge for workspace, run, and workflow rooms, and why subscribe-before-read is the rule.
Agent, Subject, Connection, Orchestration, Experiment, Interface — the backing systems behind each, and why there's no seventh.