The problem
MCP Inspector doesn't belong in CI
Exploration is manual. Production needs deterministic, repeatable, code-first tests on every push — with JUnit, SARIF, and HTML evidence.
Three modes
Functional · Regression · Performance
assert_tool_call, assert_capabilities, assert_tool_schema, protocol version gates
assert_snapshot, mask_patterns, assert_tool_idempotent — catch silent drift
assert_latency p95/p99, assert_throughput, SLO gates in CI
Productivity
Record → Generate → Gate
mcp-test record — live MCP calls become tests + snapshots (RFC-001)
mcp-test generate — scaffold suites from tool definitions
pre-commit hook, GitHub Action, conformance badge, SARIF export
Reports
Console · JUnit · JSON · HTML · PDF
Self-contained HTML dashboard with charts, chaos panels, MCP trace timeline, security findings, and PDF export via Save as PDF or mcp-test export-pdf.
Resiliency & security
Chaos · Experiments · SARIF · Bastion pairing
@marker chaos_faults — delay, 503, truncate, schema drift
mcp-test experiment run --suite core with scorecards
assert_tool_denied, authorization boundaries, OWASP MCP SARIF
Test defenses in CI; enforce at runtime in production
