Store & Journal

Package matrix/cortex/store manages the per-actor Pebble database and the atomic write batch machinery. Package matrix/cortex/journal defines the append-only write log entry shape. Package matrix/cortex/keys encodes all Pebble keys. Together they are the foundation every other cortex subsystem builds on.

Source files: cortex/store/store.go, cortex/store/writebatch.go, cortex/keys/keys.go, cortex/journal/journal.go.

Design decisions

One Pebble DB per actor. §11.1 requires that journal + head + version + indexes commit atomically. Pebble batches cannot span databases, so a single Pebble DB per actor is mandatory. All namespaces are key prefixes inside that DB — not separate files or databases.

Every mutation is journaled. WriteBatch.Commit returns ErrBatchNoJournal if the caller never called AppendJournal. This is load-bearing: the journal is the replay anchor, and a write that bypasses it silently breaks the Rebuild invariant.

Journal sequence is per-actor monotonic, gap-free. Allocation happens inside BeginWrite, which holds seqMu until Commit or Abort. Gap-free invariant: if seq=N exists, seq=N-1 must exist too. The embedder and replay harness depend on this.

Store

s, err := store.Open(root, actor, nil)
defer s.Close()

Opens (or creates) the Pebble DB at <root>/<actor>/store/. The actor string must not contain / — enforced by keys.ValidateNoSeparator.

// Point read
value, ok, err := s.Get(key)

// Prefix scan
err = s.PrefixIter(keys.PrefixIdxType, func(k, v []byte) error { ... })

// Journal seq (next-to-allocate)
nextSeq := s.NextSeq()

JournalHook

The snapshot layer installs a hook on the store so every AppendJournal call simultaneously stages an MMR leaf in the same atomic batch:

s.SetJournalHook(c.snap.MMRHook())

The hook fires per AppendJournal call, inside the same WriteBatch. It receives the raw Pebble batch and the leaf hash so it can stage additional keys without a separate commit.

WriteBatch

The atomic write transaction. Every mutating cortex operation (Write, Update, Tombstone, AddEdge, UpdateHead, Compact, Attest) creates one WriteBatch, stages all its keys, appends at least one journal entry, then commits.

wb := s.BeginWrite()
defer wb.Abort()  // no-op if Commit already succeeded

wb.Set(key, value)          // any key: m/, mv/, idx/, salience/, …
wb.Delete(key)              // for UpdateHead idx/tag/idx/frame removals
wb.AppendJournal(entry)     // at least once; may be called twice (Attest)
seq := wb.Seq()             // seq of the most recently appended entry

if err := wb.Commit(); err != nil { ... }

BeginWrite acquires seqMu. Only one WriteBatch may be open per Store at a time. The backing Pebble batch is an indexed batch so that staged writes within one batch are visible to sibling reads in the same batch — required for cortex.Attest's two-entry pattern (KindAttest at seq=N, KindLearnWeights at seq=N+1).

Key encoding

All Pebble keys are defined in cortex/keys. Invariants:

• All numeric components are big-endian fixed-width so byte-sort == numeric-sort.
• IDs are 16-byte binary ULIDs (textual only at API boundaries).
• Versions and seq numbers are uint64, 8 bytes BE.
• The path separator '/' is forbidden inside path components.

Namespace prefixes

Important meta/ keys

Sidecar posture: meta/salience_weights, meta/compile_cache/*, and meta/goal_state/* are derived / runtime-policy state. They are NOT inputs to OverallRoot. The Rebuild operation drops and re-derives them from the journal.

Journal

The journal is the cortex's write-ahead log and audit ledger. Every WriteBatch.Commit writes at least one entry at j/<seq>.

Entry shape

type Entry struct {
    Seq       uint64   // per-actor monotonic, gap-free
    Kind      Kind     // string: "write" | "update" | "tombstone" | …
    CreatedAt int64    // Unix nanoseconds
    CreatedBy []byte   // agent ref
    Payload   []byte   // canonical CBOR — kind-specific payload
}

Leaf hash

leafHash = sha256("matrix.cortex.journal.v1" || canonical_CBOR_entry)

Domain-separation prefix prevents cross-protocol hash collisions. The leaf hash feeds the journal MMR.

Journal kinds

Iterating the journal

err = s.IterJournal(fromSeq, func(e *journal.Entry) error {
    // process e
    return nil
})

Used by the embedding worker, replay rebuild, and audit tooling. The gap-free invariant means every seq from 0 to s.NextSeq()-1 has a corresponding entry.

Modifying the store or journal