Matrix logo

Tester

The tester wraps forge test and parses JSON output into structured suite/case results with pass/fail counts, gas usage, and failure reasons.

Source file: internal/tester/tester.go

The tester wraps forge test and parses JSON output into structured suite/case results. It supports path, contract, and test name filtering, and returns partial results when some tests fail.


Design decisions

Forge test with JSON output

The tester runs forge test --json and parses the output. It supports both single-object JSON and NDJSON (one line per suite) formats, handling the output variation across forge versions.

Filtering

Three optional filters narrow the test run:

FilterForge flagExample
match_path--match-pathtest/MyToken.t.sol
match_contract--match-contractMyTokenTest
filter--match-testtestTransfer

Structured output

Test results are parsed into a hierarchical structure:

TestResponse
  ├── Passed: int
  ├── Failed: int
  └── Suites: []TestSuiteResult
        ├── File: string
        ├── Passed/Failed/Skipped: int
        └── Cases: []TestCaseResult
              ├── Name: string
              ├── Status: "Success" | "Failure"
              ├── Reason: string (failure details)
              ├── Gas: uint64 (unit gas or fuzz mean gas)
              └── Duration: string

Partial results on failure

When tests fail, the tester returns the envelope with ok: false but populates data with the suite results. This lets agents see exactly which tests failed and why without a second round-trip.

Timeout

The forge subprocess has a 30-minute timeout, generous for large test suites.


Error codes

CodeMeaning
TEST_FORGE_FAILEDforge test subprocess failed (non-zero exit, timeout, no output)
TEST_ASSERTION_FAILEDOne or more tests failed (partial results in data)
INVALID_REQUESTMissing project_root

Modifying the tester

What to changeWhere
Add filter flaginternal/tester/tester.go - Test method args
Change timeoutinternal/tester/tester.go - RunWithTimeout call (30 min)
Add gas reportingpkg/types/test.go - TestCaseResult
Parse coverage outputNew method in internal/tester/