Operations FAQ

Q: How do I force-wake a user's machine?

The router exposes POST /internal/wake with a constant-time bearer token (ROUTERWAKETOKEN). Admin user management is available at /admin/users when ROUTERADMINTOKEN is set. These are internal endpoints, not part of the public API.

Q: How does the gateway handle outages?

The gateway has kill switches at multiple levels. Setting MATRIXGATEWAYDISABLED=true returns 503 to every request including /healthz. Setting an actor's dailypaxmax to 0 in dailybudgetcaps returns 429 instantly. If the upstream provider returns non-2xx, the gateway forwards the body verbatim with no ledger debit. Restarting the gateway clears the disabled flag.

Q: What monitoring endpoints are available?

The daemon exposes GET /healthz for liveness checks (includes SSE broker statistics), GET /events for Server-Sent Events streaming of real-time agent transcripts, and GET /me for per-user settings and identity. The gateway exposes its own GET /healthz for liveness.

One daemon per user on a Fly Machine, auto-suspended when idle. A single public router validates a JWT, wakes the target machine, and reverse-proxies the request. State lives on a shared box (MinIO + Postgres + router) over a WireGuard mesh. The gateway runs as a separate service on the shared box.

The user's machine is auto-suspended when idle to conserve resources. On the first request, the router wakes the machine and waits for daemon readiness before proxying. This is a one-time cold-start cost. Docker images install all dependencies at build time (no runtime apt-get or npm install) to keep cold starts minimal.

No. The daemon is single-flight: one user per process. A concurrent POST /messages while another is in progress returns 409 Busy. Isolation is per-machine, giving each user a dedicated runtime environment.

Dockerfiles install all dependencies at build time. Entrypoint and bootstrap scripts are idempotent: run them twice and you get the same state. No runtime package installation occurs. The entrypoint.sh handles credential resolution from environment variables, Cortex initialization, and MCP server setup.

The router exposes POST /internal/wake with a constant-time bearer token (ROUTER_WAKE_TOKEN). Admin user management is available at /admin/users when ROUTER_ADMIN_TOKEN is set. These are internal endpoints, not part of the public API.

The gateway has kill switches at multiple levels. Setting MATRIX_GATEWAY_DISABLED=true returns 503 to every request including /healthz. Setting an actor's daily_pax_max to 0 in daily_budget_caps returns 429 instantly. If the upstream provider returns non-2xx, the gateway forwards the body verbatim with no ledger debit. Restarting the gateway clears the disabled flag.

The daemon exposes GET /healthz for liveness checks (includes SSE broker statistics), GET /events for Server-Sent Events streaming of real-time agent transcripts, and GET /me for per-user settings and identity. The gateway exposes its own GET /healthz for liveness.