Discover, run, and report on MCP server tests automatically. Functional + regression + performance testing in one tool. Replace manual MCP Inspector validation with repeatable CI-native automation.
MCP Inspector is great for exploration. But you can't put a GUI in CI. You need deterministic, repeatable, code-first tests that run on every push.
One workflow validates both answer quality and time-to-answer in CI.
assert_tool_call, assert_resource_read, assert_prompt, assert_capabilities, assert_tool_schema, assert_protocol_version. Direct MCP protocol testing.
assert_snapshot with ignore_fields and mask_patterns. assert_tool_idempotent for determinism checks. Catch silent regressions before they ship.
assert_latency with p95/p99/mean/median over N runs + warmup. assert_throughput for concurrent load and RPS. SLO-style gates in CI.
If you know pytest, you know MCP Test Harness. Same conventions, same discovery, same feel. The harness handles server lifecycle, MCP handshake, and session injection.
pip install mcp-test-harnessmcp-test init --server-command "python my_server.py"mcp-test# tests/test_my_server.py
from mcp_test_harness import assert_tool_call, assert_capabilities
async def test_server_has_tools(mcp_server):
"""Verify the server advertises tool capabilities."""
await assert_capabilities(mcp_server, {"tools": {}})
async def test_echo_tool(mcp_server):
"""Call the echo tool and check it works."""
result = await assert_tool_call(
mcp_server, "echo", {"message": "hello"}
)
assert result is not None
async def test_echo_latency(mcp_server):
"""Ensure echo responds within 200ms at p95."""
await assert_latency(
mcp_server, "echo", {"message": "hi"},
p95_ms=200, iterations=20, warmup=3
)
Finds test_*.py files and test_ functions automatically. pytest conventions. Broken files log warnings, don't crash the run.
Compare responses against stored snapshots. ignore_fields for volatile data, mask_patterns for dynamic strings. Update with --update-snapshots.
Multiple workers, each with its own server instance. Tests from the same file stay on one worker for fixture correctness.
mcp-test --watch re-runs when test files change. Configurable poll interval with debounce for rapid saves.
@marker(timeout=60, retry=3, tags=["smoke"]). Filter with -m smoke or -k pattern. @skip(reason="...") for conditional skips.
JUnit XML for GitHub Actions/Jenkins/GitLab. JSON with full metadata. HTML reports. Console summary always printed.
Custom assertions, fixtures, reporters, transport adapters. Load via config or Python entry points (auto-discovered).
stdio for local servers, SSE for remote, streamable HTTP with auth headers. Test local and remote servers identically.
Pre-built images on ghcr.io. Runtime (:latest) and dev (:dev) targets. No local Python needed for CI.
assert_tool_denied, assert_authorization_boundary. MCP-aware security assertions. Pairs with MCP-Bastion for runtime defense.
JSON-RPC envelope checks. Post-connect validation of initialize, tools/list, resources, prompts. Best-effort call_tool probe.
Built-in mcp_server (per-test) and mcp_server_session (per-module). Custom fixtures with setup/teardown. Cycle detection.
No complex setup. No Java classpath. No separate test runner. One command and you're testing MCP servers.
JUnit XML, JSON reports, GitHub Action, Docker images. Designed for automated pipelines, not manual clicking.
MIT licensed. No per-seat, per-test, or per-server pricing. Use in any commercial product.
MCP Test Harness gives teams repeatable evidence that MCP behavior is safe, reliable, and governable.
Reduce unexpected behavior with automated JSON-RPC and MCP spec validation on every test run.
Catch common misuse patterns early. Authorization boundary assertions. Pairs with MCP-Bastion for runtime defense.
Auditable artifacts for internal governance and external frameworks like the EU AI Act. Full metadata in every report.
Works on Python 3.11+. Linux, macOS, Windows. Or use the Docker image.
Core harness. Lightweight: mcp + YAML + anyio. No heavy ML dependencies.
Pre-built runtime image. No local Python needed. Tags: latest, 2.0.0, dev.