Architecture

Matrix is a polyglot monorepo built around a shared execution stack. Neo handles conversational interaction through a recursive LLM tool-calling loop. Cortex provides per-actor persistent typed memory on Pebble. MCL compiles natural-language intents into a typed Intent IR through MatrixScript. The executor walks plans and dispatches MCP tools. The surrounding modules handle settlement, scheduling, routing, and service integration.

Layered model

                         user message
                              |
                              v
          +-------------------------------------------+
          |              Neo (agent loop)              |
          |  pack window -> compact -> call model ->   |
          |  run tools -> loop (ErrIncomplete = retry) |
          +--------------------+----------------------+
                               |
            core_execute       |  memory_recall
            (MCL delegation)   |  (cortex retrieval)
                               v
          +-------------------------------------------+
          |               MCL compiler                 |
          |  lexer -> parser -> validator -> canonical  |
          |     \                                  /    |
          |      -> interpreter <- LLM <- grammar      |
          |              |                             |
          |              v                             |
          |          Intent IR  (closed verb, kind)    |
          +--------------------+----------------------+
                               v
                         +-----------+
                         | executor  |  lifecycle + plan walker
                         +-----+-----+
                               |
                               v
   +----------------+    +-----------+    +------------------+
   |   Cortex       |    |  bridge   |    |   MCP servers    |
   |  (Pebble DB)   |<-->| (adapter) |    |  (subprocess)    |
   +-------+--------+    +-----------+    +------------------+
           |                                      |
           +---- events ----> attest + EMA loop --+

The core Go modules

Each is independently go build/go testable with its own go.mod. Cross-module imports use replace directives during development and explicit versions on publish.

Surrounding service modules: gateway (metered LLM proxy with PAX ledger), router (Fly Machine provisioning), deus (service marketplace/registry), layerx (agent-native settlement with USDX/USDL, Merkle receipts, DID-based accounts), chronos (scheduler with dispatch worker and wake delivery), tachyon (smart contract engine), and uwac (web connectors).

Neo: the conversational agent

Neo is the default user-facing agent. Its core loop in matrix/neo/internal/agent implements a recursive LLM tool-calling cycle:

• Transcript-as-state. The conversation IS the state. The model sees the working window and decides what to do next. No hidden state machine, no plan tree.
• Two-model architecture. A main model (conversational, tool-calling) and a cheap model (compaction, validation, write-back). Sub-agents use the main model with extended reasoning OFF to cut latency.
• Context budget with thresholds. Soft (80%) triggers cooperative compaction at a clean boundary; hard (92%) forces it as a runaway backstop.
• ErrIncomplete recovery. When the loop stalls or exhausts its step budget, it returns ErrIncomplete. The supervisor respawns a fresh agent over durable state rather than surfacing a fake "Done".
• Swarm sub-agents. spawn_subagents fans tasks out to concurrent headless agents, each in its own isolated context window. Heavy tool work stays in sub-agent windows; only distilled results return.
• Automatrix. Autonomous opportunity execution: Chronos wakes Neo on a recurring schedule, the governor picks opportunities, and a supervised run executes on the restricted (no-money) tool surface.
• Memory pager. Pins identity + rules + active goal every turn; page-faults top-K relevant cortex records via HNSW vector search or salience ranking.
• SSE event stream. POST /chat + GET /events contract with per-run replay buffer, live tool-call transparency, and workspace surface classification (terminal, browser, editor, search, media, action).

Cortex: the persistent memory store

Cortex is the ground truth for all durable agent memory. One Pebble DB per actor; all namespaces are key prefixes. 14 phases of implementation:

• Phase 1-3: Pebble shell, typed memory taxonomy (9 types), Write/Update/Tombstone, predicate-indexed Find with secondary index (idx/tag), salience-cold ranking.
• Phase 4: Auto-form generation (short/medium/full), budget-aware Find rendering with token counting.
• Phase 5: Async embedding pipeline + pure-Go HNSW vector index + Find Near / Find NearURI.
• Phase 6: 14 typed edge types, AddEdge/RemoveEdge with forward+reverse atomic writes, bounded BFS graph traversal (Find From/Find Follow).
• Phase 7: Journal MMR + per-namespace SMT-256 + OverallRoot + SnapshotManifest + multi-proofs.
• Phase 8: Three-tier cold-start Context composer (Pinned + Frame-relevant + Outcomes), idx/frame + idx/actor_obj secondary indexes.
• Phase 9: Budget-aware compaction (Compact), checkpoint records, summarize-and-link.
• Phase 10: Sub-agent CortexScope (Merkle-proof-bounded reads/writes), UpdateHead for Head-only mutations.
• Phase 11: Replay invariant (Rebuild): drop derived, walk journal, byte-identical OverallRoot.
• Phase 11.5: Salience instrumentation: AccessCount bumped by Find (late-binding), Citations bumped by Attest.
• Phase 12: EMA per-actor salience weight learning: KindLearnWeights journal entries, ColdScoreWith with learned weights.
• Phase 14: Token-bucket rate limiting on scope violations and attest entries (DoS surface guards).

The load-bearing invariant: drop the derived indexes, replay the journal, expect a byte-identical OverallRoot. Every store mutation journals atomically; a write without a journal entry fails with ErrBatchNoJournal.

MCL: the compiler

MCL (Matrix Communication Layer) is the compiler that turns prose + a SKILL.mtx + a verb hint into a typed Intent IR. The language is MatrixScript (.mtx): a declarative DSL where compiler logic, skill procedures, and the IR grammar itself are written as .mtx files. The Go runtime in mtx/ interprets them; it does not contain compile logic.

Pipeline: lexer (CRLF normalization, §SECTION headers, 2-space INDENT, matrix:// URIs) -> parser (recursive descent following EBNF in mtx/grammar.bnf) -> validator (V1-V12 rules: required sections, version-pinned tool URIs, closed vocab sets) -> canonical hash (AST-hashed, comments excluded) -> interpreter (first-match-wins on-blocks, prompt interpolation, cortex slot resolution) -> Frame extraction under grammar-constrained decode.

Closed vocabularies: 10 verbs (D7: find acquire build modify deliver analyze negotiate schedule monitor delegate) and 8 object kinds. Extensions use an x: prefix. Adding either requires a journaled migration.

Executor: the plan walker

The executor turns an Intent IR into executed work. It owns:

• Lifecycle state machine (lifecycle/): 8 states from drafting to completed/failed/cancelled. Each transition is a signed (ed25519) envelope.
• Plan-tree walker (runtime/): DFS walk of PlanTree with sequential/parallel branches (parallel fans out goroutine-per-child), tool_call dispatch through the MCP registry, step nodes hitting the executor LLM, and gate nodes for user approval.
• MCP tool dispatch (mcp/): JSON-RPC 2.0 client with stdio + streamable HTTP transports. Per-agent persistent MCP server processes with health-pinging, auto-reconnect, and graceful drain.
• Materiality classification (materiality/): D9 classifier enforces that material plan modifications halt execution until re-accept.
• 22 locked design decisions covering module boundaries, replay determinism, tool URI scheme, credential handling, and MCP server version pinning.

Cross-cutting flows

Production topology

• Compute: one Fly Machine per user, auto-suspended when idle, with a per-Machine Volume at /data.
• State: a dedicated box hosts MinIO (per-user snapshots) + Postgres (user to machine mapping) + matrix-router.
• Network: WireGuard mesh between Machines and the box; only the router's :443 is public.
• Auth: Supabase Auth -> JWT -> router validates -> wakes the user's Machine via the Fly Machines API -> reverse-proxies.