Persistence Sync

Key Ideas

• Sync Is Product Core: Refine’s two primary benefits are governance and fleet management, and syncing is the heart of fleet management. Persistence sync is a first-class capability, not plumbing.
• One Capability, Policy Differences: refine state and target-app code are all files synced over git. State and code differ only in policy — acceptance gates, resolution context, escalation route — never in mechanism.
• Conflicts Are Invitations: a conflict is an invitation to resolve it as best as possible. It only escalates when the system truly needs help understanding which path to take.
• Deterministic Speed, Agent Judgment: no conflict means deterministic code handles it in milliseconds. A conflict means an agent tries. Deterministic code prepares, gates, retains, and publishes, but never decides which side wins.
• Nothing Is Silently Destroyed: every losing side of every publication is retained as a ref before overwrite, and publication is compare-and-swap.

Purpose

Fleet management only works if every node can trust that its durable state converges with every other node’s. Persistence sync provides that convergence for everything Refine keeps in git: the synchronized state branch, candidate branches, and target-app fetches. It exists as one capability because state sync, candidate refresh, candidate integration, and operator recovery are the same problem — merging divergent lines of files — and splitting them multiplies locking disciplines, error shapes, and failure policies without adding judgment anywhere.

The Application rule is that convergence should almost never require a person. Divergence without overlap is resolved deterministically and instantly. Divergence with overlap is resolved by an agent that understands both intents or by the retained background recovery policy. Only genuine ambiguity — including competing Goal transfers or incomplete ownership operands — surfaces to an operator, and it surfaces as a domain-terms question about goals and intents, never as a fence or a raw Git failure.

Expected Role

Every merge-shaped operation walks the same deterministic ladder, each rung strictly cheaper than the next. Equal trees finish immediately. Ancestry classification fast-forwards ancestor-related heads so they never enter a merge at all; state heads that share no history — two nodes bootstrapping the state branch independently — join through an ordinary two-parent commit rather than wedging, in the same single merge call as every other divergence. That merge is Git’s own three-way tree merge over object ids: it commits clean results without checking out a single file, and it decides by identity rather than by text — a record changed on one side takes that side, a record changed on both is contested, and no line-level merge is ever allowed to splice two versions of a contested record together before judgment reaches it. Refine carries no merge implementation of its own to go wrong or to diverge from Git’s, which is the whole point; holding Git to the blob-level decision instead of its default content merge is what makes the rule true, because a content merge will silently splice two edits to one record when they fall far enough apart. That is a version requirement rather than a fallback: a node needs a Git new enough to be held that way, it is told so plainly at daemon start, at node bootstrap, and on every sync attempt, and a node that is not is reported as that node’s own condition while the rest of the fleet keeps converging. A structural JSON driver merges state members it can prove disjoint — it is a merge driver, not a judge, and anything contested falls through. Remaining conflicts go to agent resolution. Only when resolution is exhausted does the operation escalate as NeedsDecision, carrying a domain-terms question an operator can answer in one read.

Agent resolution follows a fixed contract. The agent works in an isolated, Refine-owned workspace, never the human checkout, and receives base, ours, and theirs plus domain context: for state, the goal records and ownership doctrine as guidance; for code, the goal prompts and round intents so it understands both intents, not just both diffs. Deterministic acceptance gates validate the output — state must parse, satisfy schema, hold record invariants, preserve a proven Goal owner, and never delete that Goal; code must confine its edits to the conflicted files and re-enters exact-candidate Quality. A rejected output re-prompts the agent; it never fences. Ambiguous ownership bypasses agent judgment and immediately records the supported-surface decision it needs. Escalation is the agent’s own move first for other ambiguity: an agent that cannot choose declares so instead of editing, and its question travels verbatim. Two spent attempts escalate too, naming the contested records. The agent never runs inside the repository lock: one short hold claims the divergence, pins inputs, and materializes the workspace, and a second short hold re-verifies the pinned inputs and publishes. A resolution’s entire state lives in refs under refs/refine/resolve/<id>, so any crash at any point is answered by rerunning — everything re-derives, publication is idempotent because it is compare-and-swap, and a surviving result is revalidated against the freshly pinned ownership operands before publication. The workspace of a resolution nobody is running is reclaimed by the next one. The claim is what keeps two operations from resolving one divergence at once: the second defers rather than editing the same workspace. There are no side files and no parallel journals; the resolution note is recorded as workflow evidence.

State sync uses bounded conflict resolution with parse, schema, record-invariant, and ownership gates. It may invoke configured state-resolution agents and retains its separate state_sync_agent_resolution and state_sync_auto_recovery settings. Failed workflow candidate refreshes emit Error with retained candidate and conflict evidence; they do not invoke recovery agents, retry work, or generate Rounds automatically. Skills and operators can use shared workflow controls to request recovery explicitly. State recovery preserves proven transferred owners and fails closed when ownership evidence is ambiguous.

A node that holds a question is not asked again while the same contention stands, and neither is it asked to keep paying for one. The unit both rules key on is the CONTENTION — the contested records together with the remote head that has to be reconciled with them — never the divergence, because a node that keeps working snapshots live state every pass and mints a new divergence while nothing the question asked about has moved. So a standing escalation is carried forward verbatim onto every later report of the same contention, and a contended record may buy only a bounded number of resolution engagements against one remote head before this node holds: agent calls cost real money, and a resolution that answered nothing will answer nothing again on the same evidence. Both rules describe the question and the spend, not the record’s bytes: a carried-forward question states what an agent could not decide when it was authored, while the report’s per-path summaries beside it are always recomputed from the current sides. The hold is never a fence. The pass still fails closed with the full conflict report, --authority still settles it in one command, safe ordinary conflicts still recover automatically under the separate opt-out, a record contended for the first time brings its own budget with it and is decided at once, and any movement of the remote side re-engages the resolver immediately — the budget refs are targeted at the head they were bought against, so a superseded head’s budget is swept the next time that record is contested.

The branch has to hold a fleet together while that fleet is being upgraded, because nodes upgrade one at a time and in any order. A node still on the previous build rebases its state branch onto the remote and pushes linear commits; an upgraded node classifies ancestry and publishes merges. Each is an ordinary remote head to the other, so a mixed fleet converges in either push order, a linear history never makes the ladder invent a deletion or drop the records only the other node has, and a node that upgrades mid-rollout inherits the branch, worktree, and live store it already had while its retired baseline file and anchor refs are retired on its first pass. The API contract version is the one place the difference is visible, and it is reported as that node’s pending-upgrade status rather than as a fleet failure. Nothing about the branch format asks for an atomic fleet upgrade, and no procedure may.

The surface is one command family. sync runs the ladder for everything the node manages. sync --preview is a read-only divergence summary that writes nothing on error. sync --authority live|remote [--path] is recovery — and recovery is simply sync with a decision attached: the chosen side becomes a merge commit, the losing ref is retained, and rerunning is a no-op. Fleet sync is pure orchestration of the same code path across nodes; there is no fleet-specific sync mechanism. The daemon API mirrors the family under /api/sync; the retired project sync and project state-recovery commands and their routes are deleted, not aliased.

Changes to this capability are gated by the multi-node sync simulation harness: simulated nodes against a local remote under adversarial timing, crash-and-rerun interleavings, and scripted resolvers, checking convergence, no lost work, stable identity, and crash-only reruns after every interleaving. The harness also runs not-yet-upgraded nodes — pre-upgrade behavior driven mechanically against the same remote rather than by keeping retired code alive — so mixed-version convergence, both push orders, a legacy publish racing a pending merge, and the retirement of a node’s legacy leftovers are pinned the same way.

After Skills configuration is installed, a sync pass retires leftover standalone Governance, Guidance, and Quality configuration before snapshotting live state. Retirement preserves exact source text in synchronized, content-addressed migration archives and publishes ordinary file deletions through the same sync protocol. Older-node reintroductions are retired on the next local snapshot without overwriting Skills or historical evidence. Stores that have not installed valid Skills configuration retain their migration inputs.

Future Direction

The staged rollout is complete: the module carve plus the simulation harness, the state-sync pipeline swap that deleted the baseline and arbitration machinery, the surface collapse, the agent resolution wiring — inline in state sync and resolve-in-place in candidate refresh — and base refresh at the Implement→Quality boundary, which resolves collisions while the implementing agent’s context is fresh. What remains is ahead of the conflict rather than inside it: resolution in place should extend to the integration merge itself, and sync should predict likely collisions before they land and order fleet work with awareness of dependencies, so divergence that needs judgment is created less often.