Memory Taxonomy

Package matrix/cortex/memory defines the typed record schema for every memory that lives in the cortex. Nine types, a shared Head + Version structure, auto-generated forms at three granularities, and write-time validation. This is the type system everything else operates on.

Source files: cortex/memory/types.go, cortex/memory/data.go, cortex/memory/codec.go, cortex/memory/validate.go, cortex/memory/verb.go, cortex/memory/edge.go.

Design decisions

Nine types, closed vocabulary. The type discriminator is a 1-byte integer (0x01 through 0x09). Adding a tenth type is a schema migration, not a code change. Existing persisted bytes are tied to the byte values and must not be renumbered.

Two-record layout per memory. Head is the mutable record (tags, frames, visibility, tombstone, salience pointer). Version is the immutable data record (one per Write or Update). They share an ID but are stored under separate key prefixes (m/ and mv/) so UpdateHead can rewrite m/<id> without touching mv/<id>/v/<n>.

Forms are computed at write time and stored. Short and medium forms are rendered once by forms.Render, persisted in both Head.Forms and Version.Forms, and read on every Find without a live recompute. Full form is only rendered on demand. This keeps query latency predictable.

The 9 memory types

All nine implement the TypedData interface. memory.TypeOf(data) returns the type byte for a concrete value.

ID

Memory IDs are 16-byte binary ULIDs (80 bits of entropy + 48 bits of milliseconds). Textual rendering uses Crockford-base32 via oklog/ulid. IDs in Pebble keys are stored as raw 16-byte binary for compact storage and natural byte-sort ordering.

type ID [16]byte

id := memory.NewID()        // random ULID with current wallclock
s := id.String()            // Crockford-base32 textual form
id, err := memory.ParseID(s) // parse textual back to binary

Head

Head is stored at m/<id:16> and rewritten on every Write, Update, Tombstone, and UpdateHead.

type Head struct {
    ID                 ID
    Type               Type
    CurrentVersion     uint64
    ActorScope         string           // actor name; DID once tools/registry lands
    Visibility         Visibility       // VisPrivate | VisScoped | VisActorPublic
    DeclaredImportance uint8            // 0..10; feeds salience.D factor
    Tags               []Tag            // actor-meaningful labels, max 16, max 64 chars each
    Tombstoned         *Tombstone       // nil = live
    LastUpdatedAt      time.Time
    EmbeddingRef       *VectorRef       // nil until async embedder runs
    Forms              Forms            // latest short + medium, mirrored from Version
    Frames             []FrameRef       // idx/frame entries, Frame-tier routing
}

Mutable fields

Only Tags, Frames, DeclaredImportance, and Visibility are mutable via UpdateHead. All other fields are auto-managed. Attempting to set ID, Type, CurrentVersion, Tombstoned, EmbeddingRef, or Forms via UpdateHead is rejected at the API boundary.

Visibility

VectorRef

Lives on Head so a Find result can decide "embedded?" without fetching vec/meta. The full vector bytes live in VectorMeta at vec/meta/<id>.

type VectorRef struct {
    VertexID uint64
    Model    string     // "<name>@<digest>"
    Dim      uint16
    Stale    bool
}

Version

Version is stored at mv/<id:16>/v/<n:8> and is immutable once written.

type Version struct {
    ID            ID
    Version       uint64
    Type          Type
    Data          []byte      // canonical CBOR-encoded typed Data
    CreatedAt     time.Time
    CreatedBy     string
    ExpiresAt     *time.Time  // optional TTL; nil = no expiry
    Confidence    float32     // 0..1; 1.0 default
    Provenance    Provenance
    Forms         Forms
    FormsOverride bool        // true if Forms.Short/Medium were skill-supplied
    Hash          [32]byte    // sha256("matrix.cortex.memory.v1" || Type || Data)
    ValidFrom     *time.Time  // bi-temporal valid-time start (nil = defaults to CreatedAt)
    ValidUntil    *time.Time  // bi-temporal valid-time end (nil = still valid)
}

Bi-temporal valid-time interval

ValidFrom and ValidUntil form a half-open interval [ValidFrom, ValidUntil). ValidFrom defaults to CreatedAt when nil; ValidUntil is nil for currently-valid assertions. The query engine treats the interval as half-open so a supersession that stamps ValidUntil = successor.ValidFrom leaves no overlap at the boundary.

Closing ValidUntil on supersession is a NEW journaled Version (via cortex.CloseValidity), never an in-place Head mutation, preserving replay byte-safety.

Old versions remain readable after Update - they are never deleted. The Resolve(uri) method requires an explicit version number; ResolveLatest(id) is provided for convenience but is discouraged in compiler paths (D13).

SourceKind

Tags how a memory entered the cortex:

Provenance

type Provenance struct {
    Source       SourceKind
    DerivedFrom  []string     // matrix://cortex/... URIs
    Attestations []string
    SignedBy     []byte       // optional; Phase 2 callers leave nil
    SignedAt     *time.Time
    Sig          []byte
}

Data schemas

Each of the 9 types has its own *Data struct encoded as canonical CBOR into Version.Data.

Key schemas

FactData

type FactData struct {
    SchemaVersion uint8
    Subject       string
    Predicate     string
    Statement     string
    Confidence    float32
    Source        SourceKind
    ObservedAt    time.Time
}

PreferenceData

type PreferenceData struct {
    SchemaVersion uint8
    Topic         string
    Polarity      Polarity   // prefer | avoid | neutral | do | dont
    StrengthVal   float32    // 0..1
    Rationale     string
}

ConstraintData

type ConstraintData struct {
    SchemaVersion uint8
    Statement     string
    Polarity      Polarity
    StrengthVal   Strength   // soft | firm | hard
    Source        ConstraintSource
}

GoalData

type GoalData struct {
    SchemaVersion uint8
    Statement     string
    Status        GoalStatus   // active | paused | completed | abandoned
    Horizon       time.Time
}

PatternData

type PatternData struct {
    SchemaVersion uint8
    Statement     string      // "neo.pattern.v1:" + JSON PatternSpec
    Strength      float32
    Coverage      int
    DerivedFrom   []string
}

Forms

Three render granularities, computed by forms.Render:

Token budget uses the bytes/4 heuristic (memory.CountTokens): tokens = ceil(utf8_bytes / 4). Switching to a real BPE tokenizer would change the snapshot hash, it is a spec-level migration.

Per-type short scaffold templates:

Medium adds detail: rationale, evidence counts, confidence, durations, horizon dates, counterparty, asset amounts.

FormsOverride

Callers that want a custom rendered form can set WriteMeta.FormsOverride=true and supply their own Forms. The override bypasses auto-generation but is still validated against the token budgets by memory.ValidateMemory. Exceeding budget returns ErrFormTooLong.

Tombstone

type Tombstone struct {
    Reason string
    At     time.Time
    By     string
}

Tombstoned memories keep all their version records (audit trail). Find excludes tombstoned memories by default; Query.IncludeTombstoned=true opts in. Update on a tombstoned memory returns ErrTombstoned. Salience is zeroed at tombstone time (factor inputs are preserved for a hypothetical un-tombstone path).

Tags

Tags are bounded-length actor-meaningful labels. Max 16 tags per memory, max 64 characters per tag. Tags are immutable across Update; changing tags requires UpdateHead.

Verbs and FrameRefs

Verb is the closed 10-value enum from D7, stored as a 1-byte integer:

FrameRef binds a memory to a (verb, object-kind, object-ref) triple. These are indexed under idx/frame and drive the Frame-relevant tier in Context.

type FrameRef struct {
    Verb    Verb
    ObjKind ObjKind
    ObjRef  string
}

func (fr FrameRef) Hash() [ObjHashSize]byte // sha256(ObjRef)[:16]

ObjKind is also a closed 1-byte enum. FrameRef.Validate() enforces all three at Write time. Frames are immutable across Update; changing frames requires UpdateHead.

Memory and URI

// Memory is the ergonomic combined view (NOT the on-disk shape)
type Memory struct {
    Head    Head
    Version Version
}

// URI is the canonical pointer
type URI string  // matrix://cortex/<Type>/<id>#<version>

Modifying the taxonomy