# How to Test an MCP Server: Contracts, Failures, and CI

> Test MCP servers beyond the Inspector: schemas, protocol versions, auth, timeouts, cancellation, retries, failure injection, conformance, and agent behavior.
> Author: Roman Belov · Published: 2026-08-21 · Source: https://futurecraft.pro/blog/mcp-server-testing/

An MCP server can pass a demo and still fail the first real workflow. The tool appears
in `tools/list`, one happy-path call returns JSON, and the Inspector looks green. None
of that proves what happens when a token has the wrong audience, an upstream hangs, a
client retries a write, or an agent chooses the wrong tool.

Test the server as three things at once:

1. a normal application with business logic;
2. a versioned protocol implementation;
3. a capability surface used by a probabilistic caller.

This guide assumes you already know the
[MCP architecture](/blog/mcp-servers-explained/). For deployment and recovery patterns,
use the [production MCP server guide](/blog/mcp-production-custom-servers/); for access
control and adversarial cases, keep the
[MCP security guide](/blog/mcp-security-guide/) beside this test plan.

## Use a test stack, not one tool

Each layer catches a different class of failure:

| Layer | Runs against | Best at finding |
| --- | --- | --- |
| Handler unit test | Tool function and fake dependencies | Validation, policy, mapping, business rules |
| Contract test | Real MCP client and server | Advertised schemas, error shapes, serialization |
| Transport test | Spawned stdio process or HTTP endpoint | Framing, auth, cancellation, lifecycle, cleanup |
| Conformance test | Official harness | Violations of a dated MCP specification |
| Agent scenario | Real host/model with controlled fixtures | Tool selection, sequencing, recovery, task completion |

Do not replace the first four with model evaluation. Deterministic bugs deserve
deterministic tests. Conversely, a unit test cannot tell you that two plausible tool
descriptions cause the model to select the destructive one.

## Start with a contract inventory

Before writing cases, list every exposed surface:

- tools, resources, prompts, and advertised capabilities;
- supported transports and protocol revisions;
- authentication modes and scopes;
- external APIs, databases, filesystems, queues, and secret stores;
- state handles, subscriptions, or multi-round-trip input;
- limits: request size, result size, concurrency, rate, deadline, and retention.

Turn that list into a small matrix. A server with two transports, two protocol eras,
and three identity roles already has twelve meaningful boundary combinations. You do
not need every business case in every combination; you do need one smoke path and the
version-specific behavior for each claimed combination.

## Treat `tools/list` as a public API

Tool metadata is not decoration. It is both a machine-readable contract and part of
the model's routing context. A casual description edit can change which tool gets
called even when the handler code is untouched.

For every release, verify:

- tool names and deterministic order;
- descriptions that state action, constraints, and side effects;
- required versus optional arguments;
- `additionalProperties` behavior;
- formats, enums, bounds, and maximum lengths;
- input schema dialect and valid `$ref` resolution;
- output schema and structured result shape;
- annotations, while remembering they are untrusted hints;
- the tool set returned for each authorization level.

Normalize the response before snapshotting it. Remove timestamps, generated IDs, and
other volatile metadata. Keep a human review on semantic changes:

```json
{
  "name": "close_issue",
  "description": "Close one issue after explicit user confirmation.",
  "inputSchema": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "issue_id": { "type": "string", "pattern": "^iss_[a-z0-9]+$" },
      "reason": { "type": "string", "minLength": 3, "maxLength": 500 }
    },
    "required": ["issue_id", "reason"]
  }
}
```

Then test that the implementation enforces the same contract. A beautiful schema with
a handler that accepts extra fields is documentation, not validation.

The current [tools specification](https://modelcontextprotocol.io/specification/2026-07-28/server/tools)
requires valid JSON Schema, separates protocol errors from tool execution errors, and
asks clients to validate results before passing them to an LLM.

## Test handlers without the protocol first

A tool handler should be callable with injected dependencies: repository, upstream
client, clock, ID generator, authorization context, and audit sink. That makes the
fastest tests ordinary application tests.

For a write tool, cover at least:

```text
valid request
missing or malformed field
unknown field
object not found
object belongs to another tenant
caller lacks permission
invalid state transition
upstream timeout
upstream rate limit
duplicate idempotency key
audit sink unavailable
```

Assert side effects, not only returned text. Did the repository receive exactly one
write? Was the tenant derived from verified identity rather than a model argument? Was
the upstream request aborted? Did a failed audit write block a high-risk mutation or
degrade safely according to policy?

Use seeded clocks and IDs. Avoid sleeping in unit tests; a fake clock can cross a
deadline or expiry instantly.

## Verify protocol errors and tool errors separately

**Protocol errors** cover malformed JSON-RPC, an unknown method, or a request that does
not match the protocol shape. They are returned as JSON-RPC errors.

**Tool execution errors** cover invalid business input, an API failure, or a forbidden
state transition the model may be able to correct. They return a tool result with
`isError: true` and useful, bounded feedback.

Test both. Also test that errors do not leak stack traces, SQL, internal URLs, tokens,
or full upstream bodies. A useful error tells the caller what can be changed without
exposing the system behind it. Do not turn every failure into successful text such as
“Something went wrong.” Hosts lose the signal they need for retries and recovery.

## Run the real transport

In-process tests miss the boundary where many MCP bugs live.

### stdio

Spawn the built artifact as a child process, then assert:

- stdout contains only framed MCP messages and diagnostics go to stderr;
- split and back-to-back messages are parsed correctly;
- malformed input does not corrupt the next request;
- closing stdin ends the process within the shutdown budget;
- SIGTERM cancels or drains in-flight work according to policy;
- the process does not inherit unrelated secrets.

Run at least one test from a directory containing spaces and with a minimal environment.
That catches path and implicit-shell assumptions hidden by a developer machine.

### Streamable HTTP

Exercise the deployed HTTP handler, not only the tool function:

- method and content-type handling;
- authorization on every request;
- token issuer, audience, expiry, and scope failures;
- origin and host validation where the deployment requires it;
- body and response size limits;
- concurrent calls from different principals;
- request-scoped SSE response and disconnect cleanup;
- reverse-proxy timeouts, forwarded metadata, and CORS.

The current transport model sends each HTTP message as a POST and may return either a
JSON object or a request-scoped SSE stream. That stream is also part of cancellation,
so a proxy buffering or swallowing disconnects can change server behavior.

## Test every protocol era you claim

This became non-negotiable with the July 2026 revision. The two current behavior
families differ materially:

| Behavior | Through 2025-11-25 | 2026-07-28 |
| --- | --- | --- |
| Connection start | `initialize` handshake | `server/discover`, no initialize |
| Client metadata | Session-scoped | `_meta` on every request |
| Streamable HTTP cancellation | `notifications/cancelled` | Close the request stream |
| Change events | Unsolicited notifications | `subscriptions/listen` |
| Liveness `ping` | Defined | Not defined |

The official TypeScript SDK calls these the legacy and modern eras in its
[protocol version guide](https://ts.sdk.modelcontextprotocol.io/v2/protocol-versions).
Do not run a test with one wire version and label the server compatible with both.

For each supported revision, verify negotiation, required metadata, cancellation,
errors, and version-specific features. Also test the refusal path: a client that pins an
unsupported revision should get a clear failure rather than silent partial behavior.

## Make timeouts and cancellation observable

Create a fake upstream that can behave on command:

```text
respond immediately
delay before headers
stream one chunk, then hang
return malformed JSON
close mid-response
ignore cancellation
return 429, 500, or 503
```

For each slow path, assert four things:

1. the caller gets a bounded result before its deadline;
2. cancellation reaches the handler through the correct transport/version mechanism;
3. the upstream request and acquired resources are released;
4. telemetry distinguishes timeout, cancellation, and upstream failure.

A progress notification must not extend a hard deadline forever. Set separate budgets
for queueing, upstream work, serialization, and total task time. Test the boundary
values, not just a comfortably short and comfortably long case.

## Retries require an idempotency decision

Read operations are often safe to retry. Writes are not automatically safe because
the transport disconnected before the response arrived.

For every mutating tool, choose one policy:

- accept an idempotency key and return the original result on replay;
- expose a read-back operation so the caller can reconcile state;
- make the operation naturally idempotent;
- mark it non-retryable and require a fresh confirmation.

Test a write that commits upstream and loses the response. Then replay the exact call.
The expected outcome must be explicit: one object, one charge, one message, or a clear
conflict—not “usually one.”

## Use Inspector for discovery and CI smoke tests

The [MCP Inspector](https://modelcontextprotocol.io/docs/2026-07-28/tools/inspector)
remains the quickest way to inspect schemas and reproduce a failure. The current tool
offers web, CLI, and terminal interfaces. Use the web UI while developing and the CLI
for a small deployment smoke test:

```bash
npx @modelcontextprotocol/inspector --cli \
  node build/server.js \
  --method tools/list \
  --format json

npx @modelcontextprotocol/inspector --cli \
  --server-url https://mcp.example.com/mcp \
  --transport http \
  --method tools/call \
  --tool-name health \
  --format json
```

Pin the Inspector version in CI and keep secrets in the CI secret store. Do not expose
the Inspector proxy to an untrusted network: it can launch local processes and connect
to server targets.

## Add official conformance checks

The official
[MCP conformance framework](https://github.com/modelcontextprotocol/conformance)
drives a running server, captures protocol traffic, and validates messages against the
wire schema. For an HTTP server:

```bash
npx @modelcontextprotocol/conformance server \
  --url http://127.0.0.1:3000/mcp \
  --requirements 2026-07-28
```

Use the frozen `--requirements` set for the revision you claim. A rolling suite can
gain scenarios after that revision shipped; it answers a different question. If you
support both eras, run both requirement sets at their actual wire versions.

Conformance is necessary protocol evidence, not a product certification. It does not
know that `close_issue` skipped tenant authorization or that its description causes an
agent to pick it for “archive this note.”

## Finish with agent scenarios

Build a small fixed dataset of user tasks and fixtures. Include:

- a task with one obvious tool;
- two similar tools where only one is permitted;
- missing information that should trigger clarification;
- a recoverable tool error;
- an irreversible action that must pause for approval;
- untrusted content containing an instruction to call another tool;
- an upstream outage where the agent should stop rather than loop;
- a request the server and agent must refuse.

Score the final task state and the tool trajectory: tools called, order, arguments,
approvals, retries, and stop condition. Do not require exact prose or hidden chain of
thought. The [AI agent testing guide](/blog/ai-agent-testing-evaluation/) covers dataset
versioning, judge calibration, and release gates.

## A practical CI gate

**Every pull request**

- schema and normalized contract diff;
- handler and in-process protocol tests;
- one stdio or HTTP transport smoke test;
- secret scan over logs and fixtures.

**Before release**

- all supported protocol versions;
- authorization and tenant-isolation suite;
- failure injection, cancellation, shutdown, and duplicate-write cases;
- frozen conformance requirements;
- fixed agent regression dataset.

**After deployment**

- Inspector CLI health and read-only canary;
- metrics for error class, latency, cancellation, retry, and result size;
- rollback check with a known compatible client.

Keep the gate boring. A server is ready when the same failures are caught repeatedly,
not when someone can complete a perfect demo in the Inspector.