Consuming and Patching a Client Downstream

This chapter is the outside view: you are using a client someone else built on Capi, not authoring one. The properties that make a client a description rather than code are also what make it pleasant to consume — and, when it lags the service or misdescribes an endpoint, to fix in your own crate without waiting on a release.

Consuming: the whole surface

Using a published client is three steps — pick a client adapter, construct the interface, authenticate, and query:

use some_api_rs::{SomeApi, endpoints::GetThing};

let api = SomeApi::new_with_defaults(capic_reqwest::ReqwestClient::default());
api.authenticate(std::env::var("SOME_API_KEY")?);
let thing = api.query(GetThing::new("id_123")).await?;

new_with_defaults takes the batteries-included codecs behind the client’s default-codecs feature; new takes your own. authenticate is the credential flow the client library chose — a bearer token here — and a client with several credential tiers exposes each as a role (api.management().authenticate(..)). Two calling styles exist and are one path:

let thing = api.query(GetThing::new("id")).await?;    // interface-first (inherent)
let thing = GetThing::new("id").query(&api).await?;   // endpoint-first (EndpointQuery, from capi_extras)

Anything that opens a scoped request works on the endpoint-first side too, so endpoint.query(api.with_timeout(d)) reads as well as the other way round.

Reading a failure

Errors arrive as one type-erased QueryError, and its accessors are the consumer’s whole triage vocabulary:

Every accessor is a reference into the same error, so a match arm can ask several questions of one value before deciding. to_log_string() is the one-line form for a log.

Runner verbs a consumer applies

Without touching the client, a consumer reshapes a call at the runner layer, and the verbs read as a sentence:

let raw = api.with_raw_response().query(ep).await?;       // ApiResponse — skip decoding, inspect bytes and headers
let (head, thing) = api.with_headers().query(ep).await?;  // (ResponseHead, T) — the decoded value plus its head
let wrapper = api.wrapped().query(ep).await?;             // the wire wrapper of a response-unwrap endpoint

let thing = api
    .retry(RetrySpec::default())            // RetryRunner: backoff on retryable failures; retry_if(spec, |e| ..) for your own predicate
    .retry_after(RetryAfterSpec::default()) // RetryAfterRunner: obey a 429/503's Retry-After
    .follow_redirects(3)                    // RedirectRunner: up to n hops, re-described at each
    .refresh_on_unauthorized()              // RefreshRunner: one credential refresh on a 401, then resend
    .with_headers()
    .query(ep)
    .await?;

let done = api.wait_until(WaitSpec::default(), |job: &Job| job.is_finished()).query(GetJob::new(id)).await?;

with_raw_response and wrapped seat a response terminal — one per chain — while with_headers mounts an observer over whichever terminal is seated, and the re-dispatching verbs (retry, retry_after, follow_redirects, refresh_on_unauthorized, wait_until) nest: each wraps the runner beneath it, and the order you write is the order they fold. progress(..) and throttle(..) seat a transfer stage on the same chain. The composition rules — what an observer sees, which terminal pairs with which endpoint — are the runners chapter’s.

A failed call keeps its head. with_headers() attaches its head clone to the error arm as well, and QueryError::response_head() reads it back — which is how a consumer gets a Retry-After off a 503, or the trailing metadata a gRPC peer attaches to an UNIMPLEMENTED, without dropping to with_raw_response():

match api.with_headers().query(ep).await {
    Ok((head, thing)) => { /* … */ }
    Err(error) => {
        if let Some(head) = error.response_head() {
            let retry_after = head.headers.get("retry-after");
            // for a gRPC refusal the trailer cell is shared with the head,
            // and it has filled by the time the error reaches you
        }
    }
}

The per-request knobs beneath the runners are on the scoped request — api.scoped() opens one — and they are the consumer’s without any verb: with_header / append_header / without_header, with_rate_limit / no_rate_limit, with_auth / no_auth / as_role, with_user_agent, with_middleware and its response and exchange forms, with_transfer_stage, with_notifications, and with_context_extension. Every verb above returns that same scoped request, so the knobs and the verbs interleave freely.

Bounding a call: timeouts and cancellation

Three per-request knobs give a call a budget — TimeoutExt, StallExt, CancelExt, all in the prelude — and they differ in what time counts:

use capi_rs::prelude::CancelToken;

// Whole call: "this must resolve by T". All wall time counts.
let thing = api.with_timeout(Duration::from_secs(10)).query(GetThing::new("id")).await?;

// Each wait on the peer: "no single silence longer than this".
let feed = api.with_stall_timeout(Duration::from_secs(5)).query(StreamEvents::new()).await?;

// Both, for both guarantees.
let page = api
    .with_timeout(Duration::from_secs(60))
    .with_stall_timeout(Duration::from_secs(5))
    .query(ListThings::new())
    .await?;

let token = CancelToken::new();          // fire token.cancel() from anywhere
let call = api.with_cancel(token.clone()).query(GetSlowThing::new("id"));

Unary and SLA-bound calls take with_timeout; long streams and pagination take with_stall_timeout; combine them when you want both guarantees.

with_timeout(d) is a whole-call budget: the duration becomes one absolute deadline when the send starts, and everything under the call — every retry attempt, backoff sleep, and follow-up page fetch — measures against that same instant. Because all wall time counts, so does the consumer’s own pace: a stream you read one line per minute exhausts a ten-second deadline however healthy the server is. Right for an SLA, wrong for a long download.

with_stall_timeout(d) bounds each individual wait on the peer — the dial, the response head, every body chunk — re-anchored at each one. Consumer think-time never counts, because nothing is armed while nobody is waiting; and a peer that trickles steadily never trips it, because each chunk does arrive inside d. Set both and every wait is bounded by min(stall, remaining).

A knob that fires ends the call with an outcome the framework owns rather than one the adapter invented, so the same code reads it whichever adapter is underneath:

match call.await {
    Ok(thing) => { /* … */ }
    Err(err) if err.is_cancelled() => { /* the token fired — do not retry */ }
    Err(err) if err.is_timed_out() => { /* the whole-call deadline passed */ }
    Err(err) if err.is_stalled() => { /* one wait on the peer ran long */ }
    Err(err) => match err.refusal() {
        Some(refusal) => { /* this client cannot enforce a knob you set */ }
        None => { /* an ordinary failure — triage with kind() */ }
    },
}

Each of the first three is true whether the knob fired before the response head arrived or while the body was still coming — the same event, noticed at different moments — and the predicates are the only way to tell a timeout or a stall from a dropped connection, since both keep ErrorKind::Transport and stay retryable. A fired token does not: is_cancelled() is ErrorKind::Cancelled, which is_retryable() rejects, so a retry runner stops at once instead of sleeping through its backoff schedule against a token that will never un-fire.

The knobs are gated on the client’s capability markers, so they only compile against a client that enforces them — the adapter table says who enforces what. A budget that reaches an adapter by a path the markers cannot see is refused rather than dropped: refusal() hands back a Refusal naming the client and the knob, the kind is InvalidRequest, and nothing was sent.

Patching an endpoint without forking

An endpoint is an ordinary struct bound to an ApiConfig, so a new endpoint in your crate can wrap an imported one and change only what is wrong. The tool is OverrideEndpoint: name the wrapped type, hand back the two accessors, and every request-construction method defaults to delegating to it, so the impl states exactly its departures. grok’s Deferred<T> is the shipped model — the same chat request, asked to return a polling handle instead of a completion:

#[derive(Debug, Clone, Default)]
pub struct Deferred<T: DeferrableChat>(pub T);

impl<T: DeferrableChat> OverrideEndpoint for Deferred<T> {
    type Endpt = T;
    type Output = GetChatDeferredCompletion;                    // a different response shape
    type Error = GrokError;
    type Decoder = BodyDecoder;
    type Requires = <Self::Endpt as Endpoint>::Requires;        // the lane is the inner endpoint's

    fn endpoint_ref(&self) -> &Self::Endpt { &self.0 }
    fn endpoint(self) -> Self::Endpt { self.0 }

    fn body(self, config: &InnerConfig<Self>, _context: &ApiContext) -> Result<impl ToBody, BodyError> {
        #[derive(Debug, WireModel, WireEncode)]
        struct DeferChat<T> {
            #[wire(flatten)]
            endpt: T,
            deferred: bool,
        }
        config.json_lib().encode_body(&DeferChat { endpt: self.0.wire_body_fields(), deferred: true })
    }
}

impl<T: DeferrableChat> DecodeBodyFn for Deferred<T> {
    fn decode_fn() -> BodyFnDecoder<Self> { GrokResponse::decode }
}

The five associated types are required — Output, Error, Decoder, and Requires forward with <Self::Endpt as Endpoint>::X when they do not change — and InnerConfig<Self> names the wrapped endpoint’s config, so an override speaks the same codecs and base URL. From this seam you add a body field the upstream crate omitted, change the URL or the method, add a header, apply a tighter rate limit, or swap the decoder. The override is an Endpoint in its own right through a blanket impl, so api.query(Deferred::new(chat)) needs nothing else; and because it wraps rather than copies, an upstream fix to the inner endpoint reaches you on the next update.

Derived inputs: adapting the call shape

A related seam reshapes the input rather than the endpoint. IntoEndpoint lets query() accept your own type — an app-specific struct with a fallible conversion into the real endpoint:

impl IntoEndpoint for ThingByName {
    type Endpoint = GetThing;
    type Error = LookupError;
    type Remap = remap::No;               // remap::Yes to also reshape the output through MapOutput
    fn into_endpoint(self) -> Result<GetThing, LookupError> { /* resolve name → id */ }
}
// now: api.query(ThingByName { name }).await?

With remap::Yes and a MapOutput impl the input also transforms the output (StreamIterator::try_map is the streaming equivalent); conversion and mapping failures surface as QueryError::convert_error and map_output_error. The flag is sealed to those two markers, so a third state can never fall outside the pass-through blanket. Reach for OverrideEndpoint to change what an existing endpoint sends, and IntoEndpoint to adapt a foreign input (or output) into the call; both keep the adaptation in your crate.

Patching the crate, not the endpoint

Sometimes the fix belongs upstream and you want to build against your own checkout of the client — or of a framework crate — while it is in review. Family crates depend on each other by plain version requirement, never by path, so a checkout resolves each dependency from crates.io unless a patch table above it says otherwise. cargo capi patch (from capi_verify’s cargo-capi binary) writes that table:

cargo capi patch

Run it at the directory holding the checkouts, never inside one — it refuses a directory that is itself a cargo workspace, so the table sits above every repo and inside none of them — and --only <dir>,<dir> narrows it to the checkouts you name. Cargo reads .cargo/config.toml from the current directory upward, so one table reaches every repo beneath it, rust-analyzer and trybuild scratch projects included. The table is never committed: an entry names a filesystem path, so a committed one would point a consumer’s clone at directories that do not exist and suppress the registry resolution that should have happened. And one table serving several repos over-patches by construction — cargo reports the entries a given build does not reach as unused on every command, which is expected.

Extending

When the fix is not a patch but a new piece — a codec, a transport, an auth scheme or signer, a middleware, a whole client — capi-extension-template is the starting point: cargo generate --git https://github.com/capi-rs/capi-extension-template renders a crate with the family’s feature lanes, the alignment guard, the documentation skeletons, and a [package.metadata.capi] table already in place, so cargo capi verify is its gate from the first commit. The capi-rs-author skill covers the authoring patterns — config, interface, auth, endpoints, decoders, errors, and replay tests — and is where a new client library starts.

Next: testing — how a client author, and you, verify all of this offline.