Matrix logo

Standards: Testing Rules and Project-Specific QA Guidance

Formal testing expectations and language-specific QA guidance for matrix-core. Covers the shared baseline, per-language testing rules, and project QA reports.

Overview

This section collects the repository's formal testing expectations and the project-specific QA guidance. The baseline policy comes from rules/common/testing.md, and the language-specific files refine that baseline for Go, C++, C#, Java, Kotlin, Dart, Rust, Swift, PHP, Perl, Python, TypeScript, web frontends, and Chinese.

The most consistent theme across the files is disciplined verification: write tests before implementation, keep coverage at or above 80%, prefer behavior-focused test names, and use the framework that matches the language and runtime.

Repository-Wide Testing Baseline

rules/common/testing.md establishes the minimum quality bar:

  • Minimum test coverage is 80%.
  • All three test types are required:
    • Unit tests for individual functions, utilities, and components.
    • Integration tests for API endpoints and database operations.
    • E2E tests for critical user flows.
  • The required workflow is RED > GREEN > IMPROVE:
    1. Write the test first.
    2. Run it and confirm it fails.
    3. Implement the smallest change that passes.
    4. Refactor.
    5. Verify coverage.
  • Tests should follow Arrange-Act-Assert structure.
  • Test names should explain behavior, not implementation details.
  • The tdd-guide agent is used proactively for new features and for troubleshooting failures.
  • When tests fail, check isolation, verify mocks, and fix implementation issues rather than changing the test unless the test is wrong.

Test naming conventions

test('returns empty array when no markets match query', () => {})
test('throws error when API key is missing', () => {})
test('falls back to substring search when Redis is unavailable', () => {})

Web QA Priorities

rules/web/testing.md changes the emphasis for browser-facing work:

  • Visual regression first: Screenshot key breakpoints at 320, 768, 1024, and 1440. Cover hero sections, scroll-driven sections, and meaningful states. If both themes exist, test both.
  • Accessibility second: Run automated checks, verify keyboard navigation, verify reduced-motion behavior, verify color contrast.
  • Performance checks: Run Lighthouse or equivalent on meaningful pages. Keep Core Web Vitals targets aligned with the performance guidance.
  • Cross-browser coverage: Chrome, Firefox, and Safari.
  • Responsive behavior: Check at 320, 375, 768, 1024, 1440, and 1920.
  • Flaky timeout-based assertions are explicitly warned against; prefer deterministic waits.

Language-Specific Testing Rules

FileMain guidanceTools/commands
rules/golang/testing.mdTable-driven tests, race detectiongo test -race ./..., go test -cover ./...
rules/cpp/testing.mdGoogleTest with CMake/CTest, coverage, sanitizerscmake --build build && ctest --test-dir build --output-on-failure, lcov, -fsanitize=address,undefined
rules/csharp/testing.mdxUnit, FluentAssertions, Testcontainersdotnet test, WebApplicationFactory<TEntryPoint>
rules/java/testing.mdJUnit 5, AssertJ, Mockito, Testcontainers@DisplayName, @ParameterizedTest, JaCoCo
rules/kotlin/testing.mdMultiplatform, coroutines, flows, local fakeskotlin.test, Turbine, runTest, MockEngine
rules/typescript/testing.mdPlaywright for E2E critical flowse2e-runner agent
rules/python/testing.mdpytest with coverage and markerspytest --cov=src --cov-report=term-missing
rules/rust/testing.md#[test], rstest, proptest, mockall, asynccargo test, cargo llvm-cov --fail-under-lines 80
rules/swift/testing.mdSwift Testing frameworkswift test --enable-code-coverage
rules/php/testing.mdPHPUnit or Pest, separate unit/integrationvendor/bin/phpunit --coverage-text
rules/perl/testing.mdTest2::V0, Devel::Coverprove -l t/, cover -test

Go Testing

rules/golang/testing.md focuses on the execution model:

  • Use go test with table-driven tests.
  • Always run tests with -race.
  • Use go test -cover ./... for coverage reporting.
  • Path-scoped to **/*.go, **/go.mod, **/go.sum.

C++ Testing

rules/cpp/testing.md is built around GoogleTest and CMake/CTest:

cmake --build build && ctest --test-dir build --output-on-failure

Coverage with lcov:

cmake -DCMAKE_CXX_FLAGS="--coverage" -DCMAKE_EXE_LINKER_FLAGS="--coverage" ..
cmake --build .
ctest --output-on-failure
lcov --capture --directory . --output-file coverage.info

Sanitizers in CI:

cmake -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined" ..

Java Testing

rules/java/testing.md defines a classic JVM testing stack:

  • JUnit 5, AssertJ, Mockito, Testcontainers.
  • Test structure mirrors src/main/java under src/test/java.
  • Behavior-oriented names with @DisplayName.
  • 80%+ coverage with JaCoCo.
  • Integration tests use PostgreSQLContainer<?> and PGSimpleDataSource.

Rust Testing

rules/rust/testing.md covers unit, integration, async, parameterized, and property-based testing:

  • #[test] with #[cfg(test)] modules for unit tests.
  • rstest for parameterized tests and fixtures.
  • proptest for property-based testing.
  • mockall for trait-based mocking.
  • #[tokio::test] for async tests.
  • cargo llvm-cov for coverage with a hard floor of 80%.

Kotlin Testing

rules/kotlin/testing.md covers multiplatform and reactive patterns:

  • kotlin.test for multiplatform tests.
  • Turbine for Flow and StateFlow testing.
  • kotlinx-coroutines-test with runTest for coroutine work.
  • Hand-written fakes over mocking frameworks when dependencies are complex.
  • Minimum coverage: ViewModel + UseCase for every feature.

PHP Testing

rules/php/testing.md sets a default and fallback:

  • Use PHPUnit by default; if Pest is configured, prefer Pest for new tests.
  • Keep unit tests separate from framework and database integration tests.
  • Use factories or builders for fixtures.
  • If Inertia.js is present, prefer assertInertia with AssertableInertia.

Practical QA Expectations

Across all files, the contributor policy is consistent:

  • Write tests before implementation when adding new behavior.
  • Keep test coverage at or above 80%.
  • Use the language-appropriate framework and runner.
  • Prefer behavior-focused test names.
  • Include integration coverage for endpoints, storage, and infrastructure-heavy paths.
  • Use visual regression and accessibility checks for web work.
  • Use real-infrastructure tests where the file explicitly asks for them (e.g., Testcontainers, HTTP-through-middleware validation).