Snapshot & Proofs

cortex/snapshot manages the cryptographic commitment layer: a journal MMR (Merkle Mountain Range) over every write, per-namespace SMTs (Sparse Merkle Trees) over canonical memory and edge records, and SnapshotManifest values that seal a point-in-time OverallRoot. cortex.Proof builds multi-proofs for sub-agent scope shipping.

Source files: cortex/snapshot/snapshot.go, cortex/snapshot/mmr.go, cortex/snapshot/smt.go, cortex/snapshot/multiproof.go.

---

Design decisions

Two anchored namespaces: memories and edges. The SMT covers the canonical bytes of m/<id> (MemoryHead) and e/from/<src>/<t>/<dst> (EdgeRecord). Tombstones fold into the parent canonical bytes, a tombstoned Head carries Tombstoned set; the SMT value-hash changes. Salience, forms, embedding metadata, and checkpoint records are NOT anchored (derived state).

Cortex never signs. The SignedBy field in SnapshotManifest is populated by the agent runtime or tools/attest layer before chain anchoring. Cortex holds no key material (D4).

Pull-driven snapshots. cortex.Snapshot(reason) is called explicitly by the skill compiler, tools/attest, or a periodic snapshotter. There is no auto-snapshot. The reason is recorded in the manifest's Trigger field for ops visibility but does NOT feed OverallRoot.

MMR hook is always running. cortex.New installs a JournalHook on the store. Every AppendJournal call (Write, Update, Tombstone, AddEdge, Embed, Compact, Attest, LearnWeights, everything) atomically appends an MMR leaf in the same Pebble batch.

Schema version pinned in OverallRoot. SchemaVersion=1 is mixed into the root computation so adding a namespace at v2 forces an explicit re-anchor instead of silently re-rooting.

Counters not in OverallRoot. Per-namespace memory + edge counts ride along the manifest as a replay sanity check but are not hashed in. Counter drift from a corrupted accum/ would otherwise force a divergent root before the actual error surfaced.

---

OverallRoot

root, err := c.OverallRoot()  // [32]byte

OverallRoot is a single 32-byte hash that commits to the entire cortex state:

OverallRoot = sha256(
    OverallRootDomain  ||   // "matrix.cortex.snapshot.overall.v1"
    SchemaVersion(1B)  ||
    journal_root       ||   // sha256 over all MMR peaks
    nsCount(2B BE)     ||
    for each ns sorted ascending:
        lpString(ns)   ||   // length-prefixed namespace name
        stateRoot            // per-namespace SMT root
)

This is the cortex_snapshot_hash used in the D11 compiler seed:

D11 seed = sha256(intent.id || actor || OverallRoot || mtx_digest || model_digest)

---

Snapshot

manifest, err := c.Snapshot("pre-compile")

Persists a SnapshotManifest at snap/<seq> and returns it.

type Manifest struct {
    SchemaVersion uint8
    Actor         string
    SeqAtSnapshot uint64
    JournalSeq    uint64              // max j/<seq> covered (inclusive); == LeafCount
    JournalRoot   [32]byte
    StateRoots    map[string][32]byte // {"memories": root, "edges": root}
    OverallRoot   [32]byte
    CreatedAt     int64               // Unix nanoseconds
    Trigger       string              // "compile" | "attest" | "periodic" | "explicit" - NOT in OverallRoot
    SignedBy      string              // populated by agent runtime before chain anchor
    Signature     []byte              // populated by agent runtime
    Counters      Counters            // NOT in OverallRoot
}

type Counters struct {
    Memories   uint64
    Edges      uint64
    Tombstoned uint64  // memory tombstones (Head.Tombstoned set)
}

---

Journal MMR

The journal MMR is an append-only Merkle accumulator over every journal entry's leaf hash. Peaks are persisted under accum/mmr/.

leaf_hash = sha256("matrix.cortex.journal.v1" || canonical_CBOR_entry)

Each AppendJournal call (inside a WriteBatch) invokes the installed MMRHook, which stages the leaf into the accumulator in the same atomic Pebble batch. The journal_root component of OverallRoot is derived from the current MMR peak set.

BatchedReader

The MMR hook uses the BatchedReader interface to read previously staged (but not yet committed) nodes from the in-flight batch. This is required for cortex.Attest's two-entry pattern: the second leaf's cascade may need to merge with the first leaf which was just staged in the same batch.

---

Per-namespace SMTs

Two sparse Merkle trees track canonical state:

memories SMT: keyed by sha256(domain.memories.v1 || id[16]) with value sha256(domain.value.v1 || canonical_Head_CBOR). Updated on every Write, Update, Tombstone, UpdateHead, and embedder KindEmbed (the embedder rewrites m/<id> with EmbeddingRef set).

edges SMT: keyed by sha256(domain.edges.v1 || src[16] || type[1] || dst[16]) with value sha256(domain.value.v1 || canonical_EdgeRecord_CBOR). Updated on every AddEdge and RemoveEdge. Forward direction only - the reverse e/to record is byte-identical and would double-count.

SMT node cache lives under idx/smt/, which is in the derived (droppable) namespace. Rebuild re-derives the SMTs from the canonical m/ and e/from/ keys.

---

Multi-proofs for sub-agent scopes

// 1. Take a snapshot (fixes the root the proof verifies against)
manifest, _ := c.Snapshot("for-scope")

// 2. Build a multi-proof for the URIs the sub-agent needs
proof, err := c.Proof([]memory.URI{uri1, uri2}, manifest)

// 3. Include the proof in the sub-agent's CortexScope
scope := &scope.Scope{
    SnapshotHash: manifest.OverallRoot,
    Proofs:       proof,
    // ... other fields
}

Proof returns a snapshot.MultiProof - a compact proof bundle that lets the sub-agent verify its included keys against manifest.OverallRoot without trusting the API.

If the memories SMT root has drifted since manifest was taken (i.e. new writes happened), Proof returns ErrManifestRootMismatch. Re-Snapshot before re-Proof.

MultiProof structure

type MultiProof struct {
    Root      [32]byte
    Namespace string          // "memories" or "edges"
    Proofs    []SingleProof   // one per requested key
}

type SingleProof struct {
    KeyHash  [32]byte      // sha256(memory ULID bytes)
    Leaf     []byte        // canonical Head CBOR (nil = non-membership proof)
    Siblings [][]byte      // sibling nodes along the SMT path
}

func (mp *MultiProof) VerifyAgainstManifest(m *Manifest) error

---

Finding a snapshot by root

manifest, err := c.snap.FindSnapshotByRoot(hash)
// err == snapshot.ErrSnapshotNotFound if no manifest has that OverallRoot

Used by the scope verifier to resolve Scope.SnapshotHash against existing manifests. Returns the manifest whose OverallRoot matches hash.

---

CurrentRoots

journalRoot, stateRoots, overallRoot, err := c.snap.CurrentRoots()

Returns the live roots without persisting a manifest. Used internally by OverallRoot() and the rebuild verifier.

---

Storage cost

For 100k memories + 100k edges per actor:

• MMR nodes: approximately 2N = 400k x 32 B = 13 MB under accum/mmr/n/*
• SMT nodes per namespace: bounded by O(N . 256) but empty-subtree compression keeps the realized count near O(N . log2N). Rough estimate: 1.7M nodes per namespace x 32 B + key overhead = 80 MB per namespace, 160 MB total under idx/smt/{memories,edges}/n/*
• Snapshot manifests: 250 B each; at 1 per intent + periodic = 10 MB/month under snap/<seq>

Pebble compaction reclaims tombstoned/superseded nodes as the SMTs evolve.

---

Modifying snapshots