Temporal history: what was connected when, and why

mushroomdb records every write — including when derived edges were created or retracted by rules — and lets you query that history at any granularity: per node, per edge pair, or as a point-in-time read-only snapshot of the whole graph. This is the temporal story: history APIs with rule attribution, as-of time travel, compare-and-set writes, and WAL archives.

History APIs

Three read surfaces expose the recorded timeline. All three scan the on-disk WAL and include a total_commits horizon field so callers can always determine which portion of history is visible.

node_history — per-node change log

Returns every WAL-visible change for a node since the last WAL-truncating snapshot.

MCP: node_history(key) → {key, history, total_commits, horizon}

HTTP: GET /node/{key}/history

{
  "key": "alice",
  "history": [
    { "commit": 0, "change": { "type": "NodeInserted", "label": "Person" } },
    { "commit": 1, "change": { "type": "PropSet", "field": "age", "value": 30 } },
    { "commit": 2, "change": { "type": "EdgeAdded", "edge_type": "KNOWS", "other": "bob", "outgoing": true } }
  ],
  "total_commits": 3,
  "horizon": 0
}

Change types: NodeInserted, PropSet, PropRemoved, EdgeAdded, EdgeRemoved, NodeDeleted.

edge_history — add/retract lifecycle with rule attribution

Returns the full add/retract history for all edges between two nodes. Includes derived edges — those created by rules — with the rule name attributed via rule. That is the point of the record: not just what was connected, but why (which rule fired) and when (at which commit).

How rule attribution works: When the rule engine fires or retracts a derived edge, a DerivedEdgeAdded (WAL discriminant 18) or DerivedEdgeRetracted (discriminant 19) HISTORY-MARKER record is appended in the same commit frame as the triggering mutation. These markers are pure history — they are state no-ops on replay (derived edges are re-derived from rules deterministically). History scans read them as ground truth of when the edge was created or retracted and by which rule.

MCP: edge_history(a, b) → {a, b, events, total_commits, horizon}

HTTP: GET /history/edge?a=&b=

{
  "a": "alice",
  "b": "bob",
  "events": [
    { "edge_type": "KNOWS",   "commit": 2, "event": "Added",     "rule": null },
    { "edge_type": "SIMILAR", "commit": 3, "event": "Added",     "rule": "sim_emb" },
    { "edge_type": "SIMILAR", "commit": 7, "event": "Retracted", "rule": "sim_emb" }
  ],
  "total_commits": 10,
  "horizon": 4
}

event is "Added" or "Retracted". rule is the rule name for derived edges, null for manually written edges.

was_linked — point-in-time edge check

Answers whether an edge of a given type was active between two nodes at a specific WAL commit.

Index spaces: at_commit is a 0-based frame index (the same space as edge_history event commits and total_commits). last_changed() returns a 1-based commit sequence from the CAS machinery — a different counter. Do not pass last_changed(key) directly as at_commit; to probe “at the moment of the last change”, use the commit values reported by edge_history/node_history events instead.

Or pass a date and let the engine do it. As of v0.6.11 at_commit also takes an RFC 3339 string — "2026-06-19", "2026-06-19T12:00:00Z" — and resolves it to the last commit at or before that instant, in the 0-based space, so the two counters cannot be confused. resolve_date() exposes the same resolution on its own, and commit_time_ms(frame) reads a frame’s recorded time back. Times live in a commit_times.bin sidecar that is part of neither the WAL nor the snapshot, so a store written before v0.6.11 has none and says no_recorded_time rather than guessing a commit.

Which frame a date lands on. A commit whose rules fire writes two frames: its own record, and a history marker carrying the rule attribution the edge readouts report. A date resolves to the last frame the commit wrote, so it sees that commit’s derived edges as well as its record — asking for a date is asking for the graph as of that commit, complete.

Sidecars written before v0.6.12 are discarded on open. Through v0.6.11 the map recorded a commit counter rather than a frame index. The two agree only on a store whose rules never fire, and diverge by one frame per rule-firing commit thereafter, so those entries resolve dates to an ever-earlier graph. The drift depends on which commits fired rules, which the file does not record, so it cannot be repaired: the map is dropped and the store answers no_recorded_time until its next commit starts a usable one. History itself is untouched — every commit index keeps addressing exactly what it did.

MCP: was_linked(a, b, edge_type, at_commit) → {linked} or error when outside horizon; at_commit takes a commit index or an RFC 3339 date

HTTP: GET /history/was_linked?a=&b=&edge_type=&at_commit=

{ "a": "alice", "b": "bob", "edge_type": "SIMILAR", "at_commit": 4, "linked": true }

Returns 400 (not 500) when at_commit is outside the retained horizon. The body carries the range it will accept, and says so when older events are gone:

{ "error": "commit 999 is out of range; valid range is 4..10 — events before commit 4 are not retained" }

Horizon contract

All three history endpoints report the window they can answer over. Valid commit indices are horizon..total_commits: total_commits is the exclusive upper bound, and horizon is the oldest commit still retained — 0 when nothing has been pruned. node_history and edge_history carry both fields in their response; was_linked, edges_at and open_at/query_at name both in the CommitOutOfRange they return for a commit outside it. When the WAL is empty (immediately after a WAL-truncating snapshot and before any new writes), total_commits is 0 and the history list is empty. Commits before the last truncating snapshot are not visible.

These two fields are the honesty contract: a client can always tell what portion of history it is looking at, and a short answer is never silent. The same pair is reported by mushroomdb stats (history_floor), /stats, the MCP stats tool, mushroomdb doctor’s store line, and the asof header.

Role-token masking

Role tokens may call all three history endpoints. Masking follows the same-as-absent rule:

• node_history: if the target key is outside the role’s visibility mask, the response is 404 — identical to querying an absent key (no existence oracle).
• edge_history: BOTH a AND b must be visible. If either is hidden, 404 for that key.
• was_linked: same two-key visibility requirement as edge_history.

Write endpoints (POST/PUT/DELETE) remain 403 for role tokens.

Compare-and-set writes

mushroomdb supports optimistic concurrency via compare-and-set (CAS) preconditions on write batches. A CAS batch atomically checks that all preconditions hold before applying any operation; if any precondition fails, the entire batch is rejected and no WAL frame is written.

last_changed and commit_seq

db.last_changed(key) returns Some(commit_seq) — the sequence number of the commit that last touched the node — or None if the node does not exist or has been deleted. The commit_seq is a monotonically increasing counter (1-based; every WAL frame advances it).

Touch definition: a node’s last_changed is updated when:

• InsertNode — the newly-inserted node
• SetProp / RemoveProp — the property-bearing node
• InsertEdge / DeleteEdge — both src and dst endpoints (an edge change touches both sides)
• DeleteNode — node is tombstoned; last_changed then returns None

History markers (DerivedEdgeAdded / DerivedEdgeRetracted) are state no-ops and do not update last_changed.

Precondition types

#![allow(unused)]
fn main() {
use mushroomdb::Precondition;

// Fails if the node was modified since `expected` (or does not exist).
Precondition::NodeUnchangedSince { key: "alice".into(), expected: seq }

// Fails if the node exists (for insert-only semantics).
Precondition::NodeAbsent { key: "alice".into() }
}

write_batch_cas (Rust API)

write_batch_cas takes Vec<Precondition> and Vec<BatchOp> — not a closure. See rust.md § Several writes as one for a compilable example using BatchOp directly.

All preconditions are checked atomically under the write lock before any operation is applied. If any precondition fails, the entire batch is rejected: no WAL bytes written, no in-memory state changes.

submit_batch_cas (SharedDb / synchronous API)

#![allow(unused)]
fn main() {
shared_db.submit_batch_cas(preconditions, ops)?;
}

The same atomicity guarantee holds: preconditions are evaluated under the same write guard as apply, so there is no TOCTOU window.

CasConflict error

#![allow(unused)]
fn main() {
GraphError::CasConflict { key: String, expected: u64, actual: u64 }
}

actual is the current last_changed value at the time of the check. NodeAbsent failures use expected=u64::MAX, actual=last_changed.unwrap_or(0).

Persistence

last_changed values are stored in V8 snapshot section 11 (LAST_CHANGE, HashMap<node_id, commit_seq>) so they survive snapshots. CAS preconditions remain trustworthy across restarts.

As-of time travel

GraphDb::open_at(dir, commit) replays WAL frames 0 through commit (inclusive) into a fresh in-memory graph, then marks the instance read-only. Every mutation method on the returned instance returns GraphError::ReadOnly.

As-of with a role or a mask

GraphDb::query_at_scoped(commit, cypher, params, scope) runs a read-only query at commit under a restriction. AsOfScope says which one:

Over HTTP this is as_of on POST /query: it composes with a role token and with a client mask. Over MCP it is the query tool’s as_of argument, which composes with role or with mask — that tool refuses the two together, with or without as_of. Writes are refused at any commit.

The one semantic to get right: the graph is historical, the role definition is current. roles.json is a sidecar — it is never written as a WAL record, so it has no past version to read. What time-travels is which nodes the role’s keys and labels resolve to: a role that may see the Public label sees exactly the Public nodes that existed at commit. If you rename a role’s labels today, an as-of read from last week answers with today’s label list against last week’s graph.

Deletion is not retroactive. Deleting a node does not remove it from a role’s past: a role that may see Public reads a now-deleted Public node, and its edges, at any retained commit where it was live, and a role with keys: ["k"] reads k after k is gone. DELETE changes the present, not the WAL. To revoke history, prune the archives (below) or narrow the role — narrowing takes effect at every commit at once, because the role definition is always the current one.

Keys are not identities. A role’s keys resolve to whichever node held that key at the commit asked for. Renaming a node frees its key for reuse, so keys: ["alice"] read at an old commit sees the node that was alice then, not the one called alice now. The same caveat reaches GET /node/{key}/history under a role token: its edge-endpoint filter resolves the other endpoint’s key against today’s ids, not the ids as of the historical event, so a recycled key is judged by who holds it now, not who held it then.

stub_hidden does not compose with as_of; see masks.md.

WAL retention and snapshot interaction

Every write appends a frame to the WAL. Commits are numbered 0-based: commit 0 is the state after the first WAL frame.

GraphDb::snapshot() writes the current state to snapshot.bin and truncates the WAL to empty (Rust API default: SnapshotOptions { keep_wal: false, archive_wal: false }). The CLI command mushroomdb snapshot archives the WAL by default instead of discarding it — pass --truncate to get the discard behaviour. After a truncating snapshot:

• GraphDb::open() loads from the snapshot plus any post-snapshot WAL frames.
• The commits before the snapshot are discarded, and they keep their numbers. The store’s floor moves to where they ended, and everything written after the snapshot is numbered above it. The first write is not always at the floor: a store with full-text fields or indexes re-records each of those declarations first, one commit apiece. GraphDb::open_at() of a discarded number returns CommitOutOfRange, naming the floor; it never answers with a later commit that happens to sit at the same position in the new WAL.
• stats reports the floor (history_floor), and the CLI prints it as history: reaches back to commit N.

A store truncated by 0.7.1 or an earlier release is the exception. Those releases numbered the new WAL from 0 again and left the floor where it was, so such a store still reports its history as complete, and a commit number taken before the truncation now names a different commit. Nothing on disk records how many commits were discarded, so this cannot be repaired on open. Its next truncating snapshot stops numbers being reused from then on; a number held from before an older truncation can still be answered with the wrong commit.

snapshot_with(SnapshotOptions { keep_wal: true, archive_wal: false }) writes the snapshot but leaves the WAL intact, so open_at can reach all WAL commits including those before the snapshot.

That holds while the WAL reaches the store’s first commit. Once a truncating or archiving snapshot has been taken, the WAL starts above it, on that snapshot as its base — and a keep_wal snapshot replaces the base with the state at the head. The commits in the live WAL can then no longer be rebuilt, and open_at refuses them with CommitOutOfRange rather than replay them over nothing. History scans (node_history, edge_history, was_linked) are not affected: they read the frames, not a rebuilt graph.

Choose based on operational priorities:

Torn WAL tail: if the WAL has a partial frame (e.g., after a crash mid-write), open_at silently treats it as fewer commits — only complete, valid frames are counted. No error is returned; the valid prefix is replayed.

Rust API

#![allow(unused)]
fn main() {
use mushroomdb::{GraphDb, GraphError};

// Count total WAL commits without opening the db.
let total = mushroomdb::wal_commit_count_at(&dir)?;

// Open a read-only view at commit 3.
let mut db = GraphDb::open_at(&dir, 3)?;

// Queries and explain work normally.
let rs = db.query("MATCH (n:Person) RETURN n", &Default::default())?;

// Mutations return ReadOnly.
let err = db.insert_node("Person", "charlie", vec![]).unwrap_err();
assert!(matches!(err, GraphError::ReadOnly));

// Commit out of range.
let err = GraphDb::open_at(&dir, 999).unwrap_err();
assert!(matches!(err, GraphError::CommitOutOfRange { .. }));
}

CLI

mushroomdb asof ./db --commit 5 --query "MATCH (n:Person)-[r:FIT]->(p:Project) RETURN n, p, r.score"
# as-of commit 5 of 42
# columns: n, p, score
#   n=alice  p=proj-01  score=0.87

# One namespace, as it was at that commit.
mushroomdb asof ./db --commit 5 --namespace tenant-a --query "MATCH (n) RETURN n"

--namespace restricts the read to one namespace. A namespace cannot change — it is set at insert and immutable — so that is simply the nodes which existed then and are in it; a name no node uses answers with nothing, never with everything. The same composition is available through AsOfScope::Namespace, POST /query with "namespace", and the MCP query tool, and it intersects a role there rather than replacing it. See masks.md.

Replay cost

open_at replays rules incrementally, exactly as live writes do. On a database with many commits this can be slow — rule evaluation re-runs for every node touched by every commit in the replay window. The replay is CPU-bound and does not involve disk I/O beyond the initial WAL read.

WAL archives and retention

Archives let you retain long-horizon history across truncating snapshots by keeping old WAL files as numbered sidecars instead of deleting them.

Enabling archives

#![allow(unused)]
fn main() {
use mushroomdb::{SnapshotOptions, GraphDb};

db.snapshot_with(SnapshotOptions {
    keep_wal: false,   // truncating snapshot (default)
    archive_wal: true, // rename the live WAL to wal.<N>.archive before truncating
})?;
}

When archive_wal: true, the live wal.bin is renamed to wal.<end_frame_index>.archive before the snapshot is written. Subsequent writes go to a new wal.bin. The archive files live alongside snapshot.bin in the database directory.

Sidecar files

The genesis marker is written only when the first archive snapshot is also the store’s very first snapshot. A keep_wal=false snapshot always writes snapshot.bin before truncating the WAL, so any later archiving session in a new session sees snapshot.bin present and conservatively refuses the marker. Legacy stores (any prior snapshot.bin) are treated identically.

Archive reachability — stated plainly

Archives extend two different guarantees and they differ:

History scans always reach archives. node_history, edge_history, and was_linked scan WAL records directly (record-scanning is prefix-independent). Archives are included in all history scans regardless of whether the genesis chain is intact. If you need deep per-node or per-edge history, archives are always useful.

As-of time travel (open_at) requires an unpruned genesis chain. open_at replays the WAL from scratch to reconstruct graph state at a point in time. It can only replay correctly if it has every frame from the very beginning. The genesis chain is intact when ALL of these are true:

• wal.genesis marker is present (the first archive was also the store’s first-ever snapshot — no prior snapshot.bin existed, which rules out both prior truncating snapshots and legacy stores)
• wal.floor is 0 or absent (no archives have been pruned)

When the genesis chain is NOT intact (pruned archives, or archiving started after a prior truncating snapshot), open_at for archive-resident commits returns GraphError::CommitOutOfRange — a clean refusal, never silently wrong data. Live-WAL commits (commits in the current wal.bin) are always reachable.

Summary:

node_history/edge_history take no commit bound and scan whatever WAL/archives remain, so a pruned commit’s events are not in the list — the horizon field in their response says so. was_linked, edges_at, and open_at/query_at check the horizon floor explicitly and refuse, naming the range they accept.

Retention and pruning

Pruning removes archives whose end-frame index falls at or below a retention floor. This is controlled by wal.floor: when the floor is written, all archive files at or below that floor are deleted.

The floor is written before archive deletion (crash-safe ordering): if a crash occurs mid-prune, orphaned archive files above the new floor are cleaned up on the next GraphDb::open. No archive is ever deleted without the floor being persisted first.

Effect on as-of: pruning archives breaks the genesis chain. After any prune, open_at for pruned-archive commits returns CommitOutOfRange. History scans are unaffected.

How far back history reaches

Automatic snapshots (serve, git ingest) keep every WAL archive by default (AUTO_SNAPSHOT_RETENTION, crates/cli/src/lib.rs, is None unless configured): nothing is pruned unless a caller asks for it. Retention is opt-in — mushroomdb snapshot <db-dir> --retention N is how to bound the archives, and every read says where the remaining history starts once that bound has pruned something. Below that horizon (commit < wal_horizon_floor): was_linked, edges_at, and open_at/query_at return GraphError::CommitOutOfRange, whose message names the range it will accept and says that events before the floor are not retained. node_history and edge_history take no commit bound, so they return the events that survive together with horizon — the oldest commit still retained; anything before it was pruned.

Error reference