mcp-schema — Methodology v1

Version 1 · 19 Aug 2026 · the source and population are fixed at first collection. A change to any rule below is a new methodology version (v2), never an edit to this file, and the affected series is republished in full with a restatement note.

What is measured

The set of tools each remote MCP server publishes via tools/list, as observed by a real streamable-HTTP handshake. This is ephemeral and unarchived: a server can add, rename, remove, or re-schema its tools at any moment, and neither the Wayback Machine nor any index records what the schema was. Only a dated history of handshakes can.

Source and roster

The population is the official MCP registry — https://registry.modelcontextprotocol.io/v0.1/servers — paginated with its cursor to a full listing before any probe. This is the single source of truth for which servers exist; no directory or union of directories is merged in. Changing this source is a methodology change (v2), because it redefines the population and breaks comparability across the archive's history.

The raw registry response is the record, captured before any handshake: it is content-addressed (same rule as artifacts — hash, dedupe, write once) and referenced by a dated <slug>/roster/<day>.json. If pagination is cut short, complete: false marks the listing as truncated, so a short roster is never mistaken for a shrinking registry. A server that disappears from the registry is recorded as retired, never silently dropped — that would bake survivorship bias into the dataset.

• Target identity is the registry server name (stable across endpoint moves).
• Endpoint is the first streamable-http remote declared by the server.
• Servers declaring no streamable-http remote are recorded (no_endpoint), not skipped.

Probe

For each target, in order:

1. initialize (protocolVersion 2024-11-05, no capabilities).
2. notifications/initialized (best-effort; a failure here is not an error).
3. tools/list, with the session id returned in the Mcp-Session-Id header.

The raw tools/list HTTP response body is the artifact, retained verbatim. Timeout is 20 seconds per request.

Error classification (error_class)

A failed poll writes a row with its class; it never writes nothing. The last known-good head is advanced only on success.

Partial runs are not a population sample

--limit N probes the first N targets in roster order, not a random sample, and the registry is not homogeneous in that order: its head is auth-heavy and its tail is no_endpoint-heavy. Confirmed on the first full collection — a 250-target partial run measured auth 38% and no_endpoint 26%, while the full 23,231-target population measured auth 13% and no_endpoint 51%. A --limit run exists to exercise the pipeline; it must not be used to estimate population-level error-class distributions.

Canonicalisation (frozen, archive/canonicalise.py)

Identity hashes are computed over canonical bytes: JSON payloads are re-serialised with sorted object keys, no insignificant whitespace, and arrays of objects are sorted by their canonical serialisation. This collapses field ordering and array reordering so a publisher reordering its response is not recorded as drift. Raw bytes are retained separately; only the hash uses the canonical form.

Drift

A target has drifted iff its canonical hash changed since the last successful observation. The human-readable description (summarise_diff) reports added, removed, and changed tool names with their before/after inputSchema.

Schema half-life

The published "schema half-life" is the median survival time of a contract, estimated with the Kaplan-Meier estimator. A contract's lifetime is the interval between observed changes; the interval still in force at the end of the window is right-censored (it is a lower bound on that contract's survival). Censoring matters: a naive median over the observation window would just report the window length forever, because most contracts have not changed yet.

• An observation interval ending in a change is an event.
• An interval still unchanged at the last observation is censored.
• The half-life is reported only once the survival curve actually crosses 0.5; before that the tile shows — ("insufficient observation window"), and a pointwise Greenwood 95% confidence interval accompanies the point estimate.

Versioning

• prober_version = the archive package version that performed the probe.
• methodology_version = v1 (this document).
• Both are recorded on every observation; either is a dimension of restatement.

Cadence

Daily, at 03:15 UTC with a randomised delay. A coverage drop below 90% of active targets marks the run degraded; a run with zero attempts fails loudly.