Recording and Replaying Real API Traffic

Hand-written handlers encode what you think the API returns. A recording encodes what it actually returned, including the four fields nobody documented and the header your client depends on without knowing. This page covers capturing traffic through a proxy, scrubbing it safely, replaying it deterministically, and keeping the recordings from rotting.

Context: fidelity versus intent

Recording and authoring solve different halves of the same problem, and treating them as competitors leads to picking the wrong one.

A recording has perfect fidelity. Every header, every status code, every unexpected null in a field your types declare non-nullable is preserved, because nobody had to notice it to capture it. What a recording cannot do is express a case that did not occur — the 409 on a concurrent edit, the empty list, the rate limit — because those never happened during the session.

An authored handler has perfect intent. It expresses exactly the scenario a test needs, including ones that are hard to provoke in a real system. What it cannot do is contain a detail its author did not know about.

The productive arrangement uses both: record to discover the true shape, then author the scenarios, using the recording as the source of truth for the payload structure. That is the same division of labour that proxy versus inline mocking describes at the architectural level.

What each source of truth is good at A four-row comparison. Recordings win on payload fidelity and on discovering undocumented fields. Authored handlers win on covering rare scenarios and on expressing intent in review. Recordings carry a rot risk that authored handlers do not, and authored handlers carry an accuracy risk that recordings do not. The final row recommends using recordings to discover the shape and authored handlers to express the cases. Property Recorded cassette Authored handler payload fidelity exact, including surprises only what the author knew rare scenarios only if they happened any case you can describe reviewability large diffs, hard to read intent visible in the diff principal risk goes stale without a refresh policy encodes a wrong assumption Record to discover the shape; author to express the cases. Neither is a substitute for the other.

Solution

1. Record through a proxy

WireMock’s record-and-playback mode proxies to the real API and writes a mapping per interaction:

# Start WireMock in recording mode, proxying everything to the real API.
docker run --rm -p 8080:8080 \
  -v "$PWD/cassettes":/home/wiremock/mappings \
  -v "$PWD/cassettes/__files":/home/wiremock/__files \
  wiremock/wiremock:3.13.2 --verbose

# Turn recording on, targeting the upstream you want captured.
curl -sf -X POST http://localhost:8080/__admin/recordings/start \
  -H 'Content-Type: application/json' \
  -d '{
        "targetBaseUrl": "https://api.example.com",
        "captureHeaders": { "Accept": {}, "Content-Type": {} },
        "persist": true,
        "repeatsAsScenarios": true,
        "transformers": ["response-template"]
      }'

# ... drive the application against http://localhost:8080 ...

curl -sf -X POST http://localhost:8080/__admin/recordings/stop | jq '.mappings | length'

Two options in that payload do real work. captureHeaders limits what is recorded as match criteria — without it, every recorded stub demands an exact match on every header the browser happened to send, and replay misses constantly. repeatsAsScenarios turns a repeated request with differing responses into a scenario rather than silently keeping only one, which is what preserves a POST-then-GET sequence.

2. Scrub at capture time, not before commit

A secret that reaches disk has already leaked as far as your backup system is concerned. Scrub as part of the capture step:

// scripts/scrub-cassettes.ts
import { readdirSync, readFileSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';

const DIR = 'cassettes';

const DROP_HEADERS = ['authorization', 'cookie', 'set-cookie', 'x-api-key', 'proxy-authorization'];
const VOLATILE_FIELDS = ['requestId', 'traceId', 'generatedAt', 'expiresAt'];

function scrub(node: unknown): unknown {
  if (Array.isArray(node)) return node.map(scrub);
  if (node && typeof node === 'object') {
    const out: Record<string, unknown> = {};
    for (const [k, v] of Object.entries(node as Record<string, unknown>)) {
      if (DROP_HEADERS.includes(k.toLowerCase())) continue;
      if (VOLATILE_FIELDS.includes(k)) { out[k] = `<${k}>`; continue; }
      if (k === 'email' && typeof v === 'string') { out[k] = '[email protected]'; continue; }
      out[k] = scrub(v);
    }
    return out;
  }
  return node;
}

for (const file of readdirSync(DIR).filter((f) => f.endsWith('.json'))) {
  const path = join(DIR, file);
  writeFileSync(path, JSON.stringify(scrub(JSON.parse(readFileSync(path, 'utf8'))), null, 2) + '\n');
}
console.log('scrub-cassettes: done');

Normalising the volatile fields to a placeholder does double duty: it removes noise that would otherwise produce a diff on every re-record, and it makes an accidental real value stand out in review. Run the detector from mock data privacy and anonymisation over the directory afterwards so nothing depends on this script being exhaustive.

3. Replay, and fail loudly on a miss

Replay is the easy half — the discipline is refusing to fall through:

# docker-compose.replay.yml
services:
  mock-api:
    image: wiremock/wiremock:3.13.2
    # No --proxy-all: a request with no recording must 404, never reach the internet.
    command: ["--global-response-templating", "--disable-banner"]
    ports: ["8080:8080"]
    volumes:
      - ./cassettes:/home/wiremock/mappings:ro
      - ./cassettes/__files:/home/wiremock/__files:ro

The absence of a proxy fallback is the whole safety property. A replay stack configured with --proxy-all silently reaches the real API whenever a recording is missing, which means an offline demo works on the office network and fails on a train — and nobody knows why, because it never failed during testing.

For MSW-based replay, the same rule expressed in code:

// src/mocks/replay.ts
import { http, HttpResponse } from 'msw';
import cassettes from '../../cassettes/index.json';

interface Interaction {
  method: string;
  path: string;
  query?: Record<string, string>;
  status: number;
  body: unknown;
  headers?: Record<string, string>;
}

const key = (method: string, path: string) => `${method.toUpperCase()} ${path}`;
const INDEX = new Map<string, Interaction>(
  (cassettes as Interaction[]).map((i) => [key(i.method, i.path), i])
);

export const replayHandlers = [
  http.all('https://api.example.com/*', ({ request }) => {
    const url = new URL(request.url);
    const found = INDEX.get(key(request.method, url.pathname));
    if (!found) {
      // Loud, specific, and actionable — never a silent pass-through.
      return HttpResponse.json(
        { error: 'no_recording', message: `No cassette for ${key(request.method, url.pathname)}` },
        { status: 501 }
      );
    }
    return HttpResponse.json(found.body, { status: found.status, headers: found.headers });
  }),
];

A 501 with the missing key in the message turns “the app is broken offline” into a one-line fix. This is the replay mode referenced in swapping mock and live APIs with env vars.

From live capture to offline replay A pipeline in four stages. The application drives traffic through a recording proxy to the real API, producing raw captures. A scrub step removes credentials and normalises volatile fields before anything is committed. The cassettes are committed and reviewed. In replay the stack serves only from cassettes, and a request with no recording returns a 501 rather than falling through to the network. Application driven by hand Recording proxy captureHeaders limited Real API once, deliberately Scrub — fails closed credentials out, volatiles normalised Cassettes in git reviewed like code Replay — fails closed no recording means 501, never the network The two heavy-bordered stages are the ones that must never be softened: a scrub that warns, or a replay that proxies, defeats the whole arrangement.

Verification

# Replay works with the network unavailable
docker compose -f docker-compose.replay.yml up -d
curl -s --max-time 3 http://localhost:8080/orders/ord_1 | jq '.id'

# No credentials survived the scrub
grep -rniE 'authorization|bearer |api[_-]?key|set-cookie' cassettes && exit 1 || echo 'cassettes clean'

# Every cassette records what it was captured against
jq -r 'select(.recordedAt == null) | input_filename' cassettes/*.json   # expect no output

The middle command belongs in CI. Recordings are the single most common way a real token reaches a repository, and a grep costs nothing.

Gotchas and edge cases

  • Recorded stubs match too strictly by default. A capture that records every request header produces stubs that only match the exact browser, version and locale that recorded them. Limit captureHeaders to the headers that genuinely vary the response — usually Accept and Content-Type and nothing else.

  • Timestamps in bodies make every re-record a full diff. If generatedAt is not normalised, re-recording changes every file and the review becomes unreadable, so nobody reviews it. Normalise volatile fields as part of the scrub, and the diff shrinks to the fields that actually changed.

  • A cassette with no expiry becomes a fossil. Stamp each with recordedAt and the API version, and run a scheduled job that re-records and fails on a structural difference — the same classification described in versioning mock contracts alongside API releases. Without it, the recording quietly contradicts the live contract and every test that depends on it is testing history.

What to drive during a recording Three things worth deliberately exercising while recording, because none of them will be captured otherwise: the empty and single-item cases, at least one error response, and the pagination boundary. A session that only walks the happy path produces a cassette that can only replay it. Empty and single-item cases drive them explicitly otherwise only the typical case is captured At least one error response trigger a 404 and a 4xx deliberately errors are never captured by accident The pagination boundary walk to the last page the short final page is where bugs live Twenty minutes of deliberate driving during the session saves writing those cases by hand afterwards.

← Back to Proxy vs Inline Mocking Strategies