Write-back Consolidation

Package matrix/neo/internal/writeback is Neo's automatic background consolidation pass (the frozen spec's write-back, option B): after each turn a cheap model sweeps the transcript and promotes durable learnings into cortex -- objective facts (semantic), task outcomes (episodic), and reusable how-to patterns (procedural). The main agent never has to consciously call remember(); this keeps the durable store current so compaction only has to capture the ephemeral story-so-far.

Source file: neo/internal/writeback/consolidator.go.

---

Design decisions

Option B: automatic background consolidation. The frozen spec considered two options: (A) agent-must-remember and (B) automatic background pass. Neo chose B. The main agent never has to consciously call remember().

Dual-model pipeline. The consolidator uses two LLM clients: extract (a stronger model for durable-learning extraction, because extraction quality directly sets memory quality) and classify (a cheap model for the rare similar-neighbor relation check). A nil classify falls back to extract.

Best-effort, bounded queue. Jobs are enqueued on a channel of depth 8. If the queue is full, the job is dropped; cortex stays eventually-current. The live transcript is ground truth for the turn anyway.

Synchronous mode for compaction. ConsolidateSync runs the same extraction on the caller's goroutine. Called from compact() before evicting older turns so durable facts/events/patterns reach cortex before the turns are lost.

Very selective. Most interactions yield nothing, and that is the correct, common answer. The prompt explicitly instructs the model to return empty arrays when nothing is durable.

Auto-propose skills. When a pattern's coverage crosses MinPatternSuccesses, the consolidator proposes it as a skill (P2-2). Proposed skills are deduped by pattern name and tracked with a reinforcement count.

---

Consolidator

type Consolidator struct {
    cfg      config.Config
    extract  *llm.Client // durable-learning extraction (stronger; quality sets memory quality)
    classify *llm.Client // cheap relation-classify on the rare similar-neighbor path
    pager    *memory.Pager

    jobs chan string
    done chan struct{}

    proposeMu      sync.Mutex
    proposed       []string
    proposedSet    map[string]struct{}
    reinforceCount map[string]int // tracks reinforcement per pattern name
}

wc := writeback.New(extractClient, classifyClient, pager, cfg)
wc.Start()
defer wc.Stop()

// In the agent loop, after a turn completes:
if a.consolidator != nil {
    a.consolidator.Consolidate(renderTranscript(a.working))
}

// Before compaction evicts older turns:
if a.consolidator != nil {
    a.consolidator.ConsolidateSync(ctx, renderTranscript(oldTurns))
}

---

Consolidation prompt

The extraction model reads the transcript and extracts ONLY durable learnings:

Return STRICT JSON:
{
  "facts": ["..."],
  "user_facts": ["..."],
  "patterns": [
    {
      "name": "...",
      "trigger": "...",
      "preconditions": ["..."],
      "steps": ["..."],
      "gotchas": ["..."],
      "success_criteria": ["..."]
    }
  ],
  "outcome": {"summary": "...", "status": "success|failure|partial"}
}

Rules:

• facts: objective, durable truths about repo/environment/domain (NOT transient chit-chat)
• user_facts: durable truths about the USER (name, role, preferences) - pinned to every future conversation
• patterns: reusable how-to recipes (name, trigger, preconditions, steps, gotchas, success_criteria)
• outcome: include ONLY if a concrete task was completed or failed; otherwise null
• Copy identifiers verbatim
• If nothing is durable, return {"facts": [], "patterns": [], "outcome": null}

---

Processing

For each extracted category:

Facts (up to 5)

_, _ = pager.RememberFact(ctx, statement)

Stored as FactData with subject matrix://knowledge/neo. Semantic dedup prevents near-identical duplicates.

User facts (up to 5)

_, _ = pager.RememberUserFact(ctx, statement)

Stored as FactData with subject matrix://knowledge/user. Deduped by normalized statement before writing. These are pinned to every future conversation via UserProfile.

Patterns (up to 3)

_, _ = pager.ReinforcePattern(ctx, spec, nil)

If a pattern with the same dedup identity (name, trigger, steps) already exists, it is reinforced (coverage++, strength nudged up). Otherwise a fresh low-confidence candidate is written. When coverage crosses MinPatternSuccesses, the pattern is added to the proposed skills list.

Outcome (1)

_, _ = pager.RecordOutcome(ctx, summary, mapOutcome(status), "")

Stored as EventData with EventObservation kind.

---

Loose JSON parsing

The model may wrap JSON in prose or code fences. parseLooseJSON extracts the outermost {...} object before unmarshaling:

parseLooseJSON("```json\n{...}\n```", &out)
parseLooseJSON("Sure! Here is the result:\n{...}\nHope that helps.", &out)

---

Proposed skills

The consolidator tracks patterns that cross the success threshold and exposes them:

func (c *Consolidator) ProposedSkills() []string

Returns the names of proposed skills (deduped, ordered by proposal time). The session injects these into the agent's stable system prefix as a names-only skill index.

---

Modifying write-back