LLM Client

Package matrix/neo/internal/llm is Neo's OpenAI-compatible function-calling transport. It reuses matrix/mcl/llm's Config, provider detection, and model registry for gateway metering and provider routing, but owns its own tool-calling message shape.

Source files: neo/internal/llm/client.go, neo/internal/llm/message.go.

---

Design decisions

Chat-completions shape only. v1 supports the OpenAI chat-completions API with native tools/tool_calls/tool role. Anthropic Messages and OpenAI Responses shapes are rejected with a clear error; their tool schemas differ and would break the loop.

Gateway metering path. When Config.GatewayURL is set, calls are rewritten to ${GatewayURL}/v1/chat/completions with MATRIX_GATEWAY_TOKEN bearer and X-Matrix-* metadata headers. Spend is attributed to the actor under slot "neo".

Reasoning channel extraction. Some providers inline chain-of-thought inside content as <think>...</think> or <thinking>...</thinking>. splitInlineThink moves this out of the visible channel and into the Reasoning field, so internal monologue never leaks into the chat.

Typed sentinel errors. ErrRequestTooLarge (HTTP 413, recoverable by compaction), ErrProviderRejected (4xx validation, non-retryable), and ErrRateLimited (429, retryable with hard backoff) let callers make precise recovery decisions.

Sub-agent model variant. Background sub-agents use a dedicated client (subMain) with extended reasoning OFF to cut latency and token burn. Only the user-facing Neo loop and the core MCL pipeline think.

---

Client

type Client struct {
    model           string
    provider        mcllm.Provider
    endpoint        string
    apiKey          string
    gatewayURL      string
    gatewayTokenEnv string
    actorDID        string
    intentID        string
    slotLabel       string
    temperature     float64
    maxTokens       int
    seed            int64
}

client, err := llm.New(mcllm.Config{
    Model:       "Qwen/Qwen3.7-Max",
    Temperature: 0.4,
    MaxTokens:   4096,
    GatewayURL:  cfg.GatewayURL,
    ActorDID:    cfg.ActorDID,
    SlotLabel:   "neo",
})

Provider is auto-detected from the model string. Endpoint defaults to the provider's canonical URL. API key is read from environment (FIREWORKS_API_KEY, TOGETHER_API_KEY, OPENCODE_API_KEY) unless overridden in Config.APIKey.

---

Chat

func (c *Client) Chat(ctx context.Context, req ChatRequest) (*ChatResult, error)

Sends the message list + optional tool schemas, returns the model's single assistant turn.

type ChatRequest struct {
    Messages   []Message
    Tools      []Tool
    ToolChoice string // "auto" (default), "none", "required"
}

type ChatResult struct {
    Message      Message
    FinishReason string
    Usage        Usage
}

FinishReason values: "stop", "length" (truncated), "tool_calls". Truncated generation is handled by the agent loop, never emitted raw.

---

Message types

type Message struct {
    Role       string     // "system" | "user" | "assistant" | "tool"
    Content    string     // text content
    ToolCalls  []ToolCall // assistant turn: requested calls
    ToolCallID string     // tool turn: which call this answers
    Name       string     // tool turn: function name
    Reasoning  string     // chain-of-thought (not serialized)
}

Constructors

llm.SystemMessage("be helpful")
llm.UserMessage("what is the PAX price")
llm.AssistantMessage("PAX is trading around $X") // seeding from history
llm.ToolResult("call-1", "paxeer__price", `{"pax": "0.42"}`)

ToolCall

type ToolCall struct {
    ID       string
    Type     string // always "function"
    Function FunctionCall
}

type FunctionCall struct {
    Name      string
    Arguments string // JSON-encoded args
}

ParseArgs() decodes the JSON string into a map[string]interface{}.

---

Tool schema

type Tool struct {
    Type     string      // always "function"
    Function FunctionDef
}

type FunctionDef struct {
    Name        string
    Description string
    Parameters  map[string]interface{} // JSON Schema object
}

tool := llm.NewFunctionTool("fs__read_file", "Read a file", map[string]interface{}{
    "type": "object",
    "properties": map[string]interface{}{
        "path": map[string]interface{}{"type": "string"},
    },
    "required": []string{"path"},
})

---

Reasoning channel

fromWireRespMessage handles three provider postures:

1. Separate reasoning_content field - copied directly to Message.Reasoning
2. Inline <think>...</think> or <thinking>...</thinking> in content - extracted by splitInlineThink
3. Unterminated opening tag - the whole remainder is reasoning (truncated generation safety)

The Reasoning field is never serialized onto the wire and never treated as the answer. It is surfaced as a distinct channel only.

---

Gateway headers

When gatewayURL is set, the request carries:

Authorization: Bearer ${MATRIX_GATEWAY_TOKEN}
X-Matrix-Actor-DID: <actor DID>
X-Matrix-Intent-ID: <intent ID>
X-Matrix-Slot: neo

This matches the daemon/router environment key MATRIX_GATEWAY_URL and the MCL compiler's gateway posture.

---

Error handling

Sentinel errors

var ErrRequestTooLarge = errors.New("neo/llm: request body too large")       // HTTP 413, recoverable
var ErrProviderRejected = errors.New("neo/llm: provider rejected request (non-retryable)") // 4xx validation
var ErrRateLimited      = errors.New("neo/llm: provider rate limited (429)")  // retryable with backoff

Callers detect with errors.Is():

• ErrRequestTooLarge: the agent can compact the window and retry
• ErrProviderRejected: stop the retry ladder immediately (the request shape is bad)
• ErrRateLimited: back off hard; the task supervisor reads this to lengthen its respawn backoff

HTTP errors

HTTP errors are parsed for structured error bodies:

neo/llm: fireworks http 429: Rate limit exceeded (type=rate_limit)

Empty choices, parse failures, and API errors all return wrapped errors with the provider name for attribution.

---

Modifying the client