Replay Testing with ReplayClient

The client is a swap point, and testing is where that pays off most: seat a ReplayClient in place of the real adapter and your endpoints run against recorded fixtures — offline, deterministic, no network. capi_test_framework makes this the default way to test a Capi client, and this chapter is the loop: write a test once, record it against the live service, commit the fixtures, replay them forever.

One test body, three runtimes

#[capi_test] writes a test once and emits the shape for whichever lane is building — exactly one compiles per configuration:

• native sync → #[test], with async and every .await stripped;
• native async → #[tokio::test];
• browser wasm → #[wasm_bindgen_test].

#[capi_test]
async fn test_get_models() {
    let api = GrokTestHarness::setup(current_test_name!());
    let models = api.query(GetModels::new()).await.unwrap();
    assert_eq!(models.len(), 3);
}

The .await you write is real in async builds and stripped in sync builds, so the same source verifies the client on every lane without a cfg in sight; a helper that must compile both ways takes #[maybe_async], the crate’s strip_await/identity pair aliased by cfg. current_test_name!() names the fixture directory after the enclosing function, so the literal and the name cannot drift apart.

The harness

A small harness in tests/common.rs wires a ReplayClient into the interface. grok’s is the shipped model:

#[derive(Debug, Copy, Clone, Default)]
pub struct BasicConfig;

impl From<BasicConfig> for ReplayConfig {
    fn from(_: BasicConfig) -> Self {
        ReplayConfig::default()
            .classify_header("cookie", FixtureClass::Omitted)
            .with_sensitive_env_prefix("GROK_")
            .with_sensitive_file("tests/sensitive.local.env")
            .with_sensitive("api_key", "xai-DUMMY_KEY")
            .with_sensitive("team_id", "00000000-0000-4000-8000-000000000002")
    }
}

pub fn setup_interface(client: ReplayClient) -> GrokApi<ReplayClient> {
    let context = client.context();                    // the recorded clock and RNG
    let mut api = GrokApi::new_with_defaults(client);
    api.set_base_context(context);                     // every request derives from it
    api
}

pub struct GrokTestHarness;
impl GrokTestHarness {
    pub fn setup<T: ToString>(name: T) -> GrokApi<ReplayClient> {
        let api = setup_interface(ReplayClient::for_test(BasicConfig, name));
        api.authenticate(api.sensitive("api_key"));   // the placeholder in replay, the live key when recording
        api
    }
}

Three things make it deterministic. ReplayClient::for_test(config, name) resolves the conventional root — tests/data, or CAPI_FIXTURE_ROOT — and loads tests/data/<name>/. client.context() is installed as the config’s base context, so every request derived from it inherits the recorded clock and RNG: a polling loop sleeps in mock time, a PKCE verifier or a multipart boundary is drawn from the recorded seed, and the replay matches byte for byte. And the config marker converts into a ReplayConfig that declares what each recorded item is — the next chapter has the full vocabulary.

api.sensitive("api_key") is the record-versus-replay idiom for a credential. with_sensitive(name, placeholder) registers the placeholder that fixtures hold and that replay returns; when recording, the live value comes from <prefix>API_KEY in the environment or the gitignored sensitive.local.env file, and a missing one fails before any traffic. An unregistered name panics in every mode, so a typo surfaces in replay too.

Record or replay, by fixture presence

There is no construction-time choice between the two. for_test scans the fixture directory: fixtures present, the client replays; none present, it records live traffic there. So the first run of a new test records and every run after it replays, and re-recording is deleting the directory. is_recording() reports which mode a run took, and .mock_only() forces replay even when fixtures are missing, for a test that must never reach the network.

Recording needs a live client, and which one is bundled depends on the lane:

reqwest and native-client are each mutually exclusive with single-threaded — the crate refuses the pair with a compile_error! naming the remedy — so a native recording suite runs --features std,async,reqwest and a current-thread suite runs --features std,async,single-threaded and replays committed fixtures. The fixture backend is a seam of its own: FsFixtureBackend on native, NodeFsFixtureBackend on browser wasm under Node, and InMemoryFixtureBackend for a fixture set built in the test itself.

The fixture format

Fixtures live in tests/data/<test_name>/, one numbered exchange per request, consumed in order:

tests/data/test_get_models/
  0001.req      # the request the client should make
  0001.rsp      # the response to play back
  0001.meta     # what replay needs and HTTP does not carry

A .req or .rsp holds the exchange and nothing else, in the canonical text form capi_http_types::text defines — the same rendering capi_debug_dump writes, so a dump and a fixture read alike. The head is a request line of method plus complete absolute URL, or a status line of version plus numeric status, then one lowercase name: value line per stored field, then the blank line the body follows. No host line (the URL carries the authority), no request version, no reason phrase:

GET https://api.x.ai/v1/models
accept: application/json
authorization: [REDACTED]

HTTP/2.0 200
content-type: application/json

{"data":[{"id":"grok-4"}]}

The body is the raw bytes after that blank line, so a fixture may carry arbitrary binary; a body kept in a file of its own (BodyEncodingMode::File, or Auto deciding from the content type), or captured chunk by chunk with the moment each arrived, is named by the sidecar instead. A response’s trailer block is a standalone 0001.trailers file, because it is only known once the body reaches EOF, and it replays as pending until the body does — which preserves the distinction a live trailer-capable client exhibits.

Everything else rides 0001.meta, a small versioned key=value grammar the test framework owns, so reading a fixture pulls no JSON or TOML codec into the graph:

capi-meta 1
request.date=2026-01-01T00:00:00Z
request.masking=masked
response.date=2026-01-01T00:00:00.150Z
response.masking=masked

The dates are the recorded wall clock that drives the injected context: the first request’s pins it, and each match advances it to that exchange’s response. Every other key has a default and appears only when it differs — where a body lives, the recording RNG’s seed, the URL fragment a head does not carry, whether the match key asserts the recorded authority, how a streaming request body ended, and the masking attestation and capture marker the next chapter explains.

Ordered consumption is what makes multi-step flows testable: a reconnect, a poll loop, or an OAuth2 exchange records as 0001, 0002, … and replays request by request, and the engine supports duplicate keys — create, modify, retrieve against one endpoint each consume the next matching entry. A request that matches nothing fails loudly, with with_write_unmatched_requests(true) writing the offending request to disk for comparison.

By default a recorded response keeps only the headers the response policy classes Plain — payload-shape headers such as content-type and content-length, caching validators, the WebSocket upgrade set. with_record_all_response_headers() keeps everything, for debugging.

Time in a replay

The mock clock is frozen by default — this and sleeping on it are the only things that move it — so two timestamps taken with nothing in between are the same instant, and an assertion about elapsed time is exact. The client exposes the knobs: advance_clock(d) moves it by hand, set_auto_advance(d) gives every read a step so successive timestamps differ, set_system_time(t) pins it, and use_live_clock(true) hands the wall clock back. A test that reads the clock many times without moving it trips a spin detector; set_stall_threshold raises the bar for one that legitimately does.

A polling test is the common case. capi_test_framework::wait_until(clock, spec, step) sleeps interval on the clock between iterations and gives up with WaitTimeout once timeout elapses — instantly in replay, since sleep advances mock time rather than waiting, and at real intervals when recording, so the fixtures pace as the service did. wait_until_done is the same loop over an endpoint that reports a status. And client.mock_timeout(d, future) races a future against the mock clock, so a test can prove a hang is caught without waiting for one.

Recording the twin

For an endpoint whose sensitive fields ride the body, record through api.query_redacted(endpoint) instead of query. It dispatches the live request as usual, but the fixture the recorder writes — and the request the replay engine compares — is a twin built from a redacted clone of the endpoint and finalized with its credentials and signatures redacted by the seal. Nothing in the twin ever held a live value, so nothing has to be found and masked after the fact; it asks the endpoint to be Clone and ApiRedact, which the wire derive provides (Redaction). Seat RedactedRunner under a runner chain to combine it with paging or polling.

Testing a decoder directly

For a decoder you want to exercise without fixtures at all — a 429 on the third call, a body truncated below its Content-Length, a server that hangs — capi_test_core’s MockClient scripts responses and the faults a recording cannot express:

let mock = MockClient::new()
    .on(Method::GET, "/shipments/shp_123")
    .respond(MockResponse::ok().bytes(body, "application/json"))   // first call
    .then_fault(Fault::Connect)                                    // second call: reset
    .default_response(MockResponse::not_found())                   // every other path
    .build();

Rules match in order, a rule’s chained outcomes script a per-call sequence whose last outcome repeats, and an unmatched request panics unless told otherwise. Fault covers Connect, ConnectionRefused, Timeout, Hang, Cancelled, and Empty; MockResponse::ok() / status(code) build a response with text, bytes, body_encoded, or event_stream, plus the delivery modifiers a recording cannot express — truncate_to(n), reset_after(n), malformed(), dribble(chunks, total), delay(d). MockClient implements ApiClient, so it seats in the same harness a ReplayClient does, and a MockClient::new() with no rules is the proof of a negative — hand it to a resumable decoder as FollowUpChannel::new(mock) and a stray reconnect trips the panic.

The gate

Every committed fixture attests how it was written, and one more test in the suite refuses the ones that must not land:

#[test]
fn fixtures_attest_masking() -> Result<(), Box<dyn std::error::Error>> {
    FixtureAudit::scan("tests/data")?.check()?;
    Ok(())
}

cargo capi fixtures runs the same audit from the command line, and cargo capi verify includes it. What the attestation says, and the three tiers that decide what a fixture may hold, is the next chapter; recording a WebSocket session as a .ws fixture is in WebSocket, and the per-message comparison a gRPC fixture gets — bodies to .bin sidecars, grpc-status in the trailer block, the twin as the route for a secret in a frame — is in gRPC.