LLM Client

The MCL LLM client provides a provider-agnostic interface for calling language models. It supports 6 providers, 3 API shapes, grammar-constrained decoding, streaming, and a 4-tier model router with step-kind sub-routing.

Source files: llm/llm.go, llm/messages_api.go, llm/responses_api.go, llm/model.go, llm/identity.go.

---

Providers

Provider auto-detection is based on the model string. Callers can override with Config.Provider.

---

API shapes

The client supports three wire formats, auto-detected from the endpoint URL suffix:

func DetectAPIShape(endpoint string) APIShape {
    // /v1/messages -> ShapeMessages
    // /v1/responses -> ShapeResponses
    // /v1/chat/completions -> ShapeChatCompletions
}

All three shapes implement interpreter.LLM and interpreter.StreamingLLM.

---

Config

type Config struct {
    Model           string          // provider-specific model ID
    Provider        Provider        // override auto-detection
    APIKey          string          // override env var lookup
    Endpoint        string          // override provider default
    Temperature     float64         // 0 = deterministic (compiler default)
    Seed            int64           // D11 seed; 0 = no seed param
    MaxTokens       int             // default 8192
    EnableThinking  bool            // extended reasoning toggle
    Timeout         time.Duration   // default 90s
    GrammarMode     GrammarMode     // None, JSONSchema, or EBNF
    Grammars        map[string]*GrammarDef // grammar ID -> schema/EBNF
    InjectIdentity  bool            // prepend IdentityPreamble (Forge)
    Shape           APIShape        // force API shape (auto-detect if zero)
    // Gateway routing fields (sess#32)
    GatewayURL      string
    ActorDID        string
    IntentID        string
    SlotLabel       string
    KindRoute       string
    OnResponseHeaders func(http.Header)
}

---

Grammar-constrained decoding

The compiler slot uses GrammarJSONSchema with the intent_frame@1 grammar so the model can only emit valid Frame JSON.

---

Model router (Session 31a)

The model router is a 4-tier system with step-kind sub-routing for the executor tier:

Tiers

Executor step kinds

ModelRegistry

type ModelRegistry struct { ... }

func NewModelRegistry(fallback Config) *ModelRegistry
func (r *ModelRegistry) Register(k RouteKey, cfg Config) *ModelRegistry
func (r *ModelRegistry) Resolve(k RouteKey) Config

The registry resolves (Slot, Kind, LongCtx) to a Config. LongCtx variants are preferred when LongCtx=true is set; unknown executor kinds fall back to KindReason.

RouteKey

type RouteKey struct {
    Slot    ModelSlot
    Kind    StepKind
    LongCtx bool
}

Kind is only meaningful when Slot == SlotExecutor. For other slots, it is ignored.

---

Identity injection (Forge Phase 1)

When Config.InjectIdentity is true, the IdentityPreamble is prepended as the first system message at every Decode and Stream call:

You are Matrix -- an agent framework for LLM execution.
Your codebase lives at /root/matrix. This conversation is part of your
ongoing self-maintenance. Every action you take should serve improving
Matrix itself.

The IdentityVersion constant ("matrix-identity-v1") is mixed into model digest computation so the compile-cache invalidates cleanly across preamble migrations.

---

Streaming

All three API shapes implement interpreter.StreamingLLM:

type StreamingLLM interface {
    Stream(ctx context.Context, messages []Message, grammar string,
        onDelta func(delta string)) (string, error)
}

Each shape parses its provider's SSE event format:

---

Gateway routing (sess#32)

When GatewayURL is set, all requests are routed through the Matrix Gateway. The client stamps X-Matrix-Actor-DID, X-Matrix-Intent-ID, X-Matrix-Slot, and X-Matrix-Kind-Route headers so the gateway can authenticate the actor, route against per-slot whitelists, and track costs. Cost telemetry flows back via X-Matrix-Cost-Pax / Daily-Spent / Daily-Remaining response trailers captured by OnResponseHeaders.