Docs / Security testing

Testing mode

Security testing

MCP Test Harness owns the CI gate: deterministic security payloads, cyber suites A–D, incident E–K, audit-chain verify, load resilience, authz hooks, and AGENTS.md Unicode scan. Runtime enforcement stays with MCP-Bastion. Green means the encoded corpus is caught reproducibly — not novel adaptive adversaries.

What to enable

quality_gate:
  require_security_tests: true
  fail_on_severity: high
manifest_gate:
  enabled: true
  path: __snapshots__/mcp_manifest.snap
agents_md_gate:
  enabled: true
  fail_on: high

Cyber suites (shipped)

• Suite A — semantic egress quarantine/allow, covert-channel neutralization, general-exec tools absent
• Suite B — known encodings + capacity/media (assert_capacity_enforced, signed/unsigned media)
• Suite C — attestation polarities (assert_envelope_verified, unattested deny, tamper reject)
• Suite D — quarantine / eviction / cross-tenant (assert_untrusted_context_quarantined, …)
• Incident E–K — recorded-verdict incident conformance fixtures
• Load resilience — deny-codes, control engage, no-fail-open, fairness, latency SLO until shed
• Authz / schema — confused-deputy hooks, RCE-precondition schema rules
• Assurance — evidence / metrics / OSCAL assessment-results helpers

Payload & suite helpers

• assert_injection_blocked / assert_path_traversal_blocked / assert_no_secret_leak
• assert_egress_quarantined / assert_egress_allowed — Suite A semantic egress
• assert_covert_channel_neutralized — known encodings only
• assert_general_exec_tools_absent — capability-reduction config check
• assert_audit_chain_intact — hash-linked audit chain
• assert_agents_md_clean — hidden Unicode in AGENTS.md / agent rules
• assert_control_engages_under_load / assert_no_fail_open_under_load / assert_fairness_under_load
• assert_schema_preconditions_clean / assert_schema_bounds_clean / assert_schema_bounds_enforced
• assert_tool_definitions_clean — hidden Unicode, instruction-override text, destructive read-only hints
• assert_unsafe_url_fetch_blocked — URL scheme, userinfo, and literal non-public addresses (no DNS)
• assert_idempotency_key — replay one result; reject a changed payload under the same key
• assert_remote_requires_auth — Streamable HTTP initialize without credentials must fail closed

from mcp_test_harness import (
    marker,
    assert_egress_quarantined,
    assert_audit_chain_intact,
    assert_control_engages_under_load,
)

@marker(tags=["security", "semantic-egress"])
async def test_egress(mcp_server):
    await assert_egress_quarantined(mcp_server, "send_message")

@marker(tags=["security", "audit"])
def test_audit(chain_path):
    assert_audit_chain_intact(chain_path)

Commands

mcp-test -m security
mcp-test --sarif-output findings.sarif
mcp-test manifest check
mcp-test scan-agents
mcp-test audit verify chain.json
mcp-test --report-format html --report-output report.html

Markers semantic_live and adaptive are reserved for nightly detection-quality runs — they must not gate PRs.

Lifecycle

Scan / config → Test / CI (this harness) → Enforce / runtime (Bastion)

Canonical guide: SECURITY_TESTING.md · Cyber suites: CYBER_SECURITY_SUITES.md · Tutorial: TUTORIAL.md