A fallback is another attempt, not a clean success

A model gateway can return a successful response after one provider times out and another rate-limits. If the record keeps only the final provider, the request looks healthy while the retries, changed model, and accumulated estimate disappear.

This guide includes an original zero-network lab with invented providers, models, durations, and estimate units. It called no model or provider, used no account or credential, measured no latency, and incurred no cost.

Keep one request and every attempt

The request is the reader-visible unit of work. An attempt is one bounded try against one selected provider/model pair. Preserve both:

• a stable request ID and ordered attempt number;
• selected provider and model for each attempt;
• the declared outcome that caused success or fallback;
• a duration field with an honest clock and unit;
• an estimate plus its source and basis;
• the cumulative estimate after each attempted call; and
• the final request outcome and selected provider, if any.

Do not overwrite the failed attempt with the successful one. Do not call a synthetic tick “milliseconds.” Do not label an estimate as billed cost unless it has been reconciled to authoritative billing evidence.

What the fixture demonstrates

The first invented request has a seven-unit budget. Provider A times out after three synthetic ticks with a two-unit estimate. Provider B reports a rate limit after one tick and one estimated unit. Provider C succeeds after two ticks and three estimated units.

The final response is a success, but its lineage is three attempts and six estimated units:

Those units come from the fixture’s invented fixture-rate-card; they are not currency, tokens, invoices, or real provider prices. The ticks express ordering and declared duration only, not measured wall-clock latency.

Stop before an over-budget attempt

The second request starts with a two-unit budget. Its first attempt consumes the full two-unit estimate and times out. The next candidate has a one-unit estimate, so the dispatcher records not-attempted-budget-stop without invoking it.

That distinction matters. “Provider B failed” would be false: the budget policy prevented the call. The record keeps the candidate’s estimate provenance while leaving cumulative usage at two units and synthetic duration at zero.

The lab uses a strict greater-than check: an attempt whose cumulative estimate equals the budget is allowed; one that would exceed it is stopped first. This is a declared fixture policy, not a universal recommendation. A production design must decide how to handle missing estimates, estimate error, streaming usage, concurrent requests, reservation, and post-response reconciliation.

Run the offline fixture

Prerequisite: Node.js 24 or newer. Download all three files into one directory, rename the script from .txt to .mjs, then run the shown command. The script uses Node.js built-ins only and makes no network request.

• Gateway attribution lab source, served as plain text
• Synthetic request fixture
• Expected result

node gateway-attribution-lab.mjs gateway-attribution.synthetic.json result.json

The script creates result.json only when that path does not already exist. Compare the parsed JSON with the expected result; whitespace formatting is not evidence. Repository tests verify the deterministic data, refuse a second write to the same evidence path, assert the budget stop happens before a call, and reject malformed or ambiguous fixtures.

What the validator refuses

The small schema fails closed on unknown fields, unsafe or duplicate identifiers, unsupported outcomes, negative or fractional budget/estimate values, missing estimate provenance, duplicate providers within one request, and too few or non-array requests.

That validation does not authenticate a caller, enforce a real budget, verify provider usage, prevent secret leakage, retry a network call, or emit an OpenTelemetry trace. It proves only that this synthetic input satisfies this lab’s contract and produces the recorded deterministic result.

Relate the record to traces carefully

OpenTelemetry defines a span as one operation within a trace and supports attributes and timestamped events. Its semantic conventions aim to give telemetry fields consistent meaning across implementations. A gateway can model the overall request and provider attempts with spans or events, but this fixture does not claim to implement OpenTelemetry or prescribe a final schema.

The current OpenTelemetry GenAI attribute registry warns that input messages, output messages, instructions, and retrieval queries may contain sensitive information. Attempt attribution rarely requires recording prompt or response bodies. Use trace or request IDs for correlation; keep span names and aggregation dimensions bounded and low-cardinality. Opt in to content only under a reviewed data policy.

Primary references:

• OpenTelemetry tracing API
• OpenTelemetry trace semantic conventions
• OpenTelemetry GenAI attribute registry

The original research leads were two public gateway projects. Their captures were treated as untrusted prompts for the question, not as evidence or recommendations. This article makes no claim about either product and carries forward no private topology, usage, account, or migration detail.

A review checklist for a real gateway

1. Can one request be correlated with every provider attempt without storing sensitive content?
2. Does each attempt retain provider, model, outcome, duration definition, and estimate provenance?
3. Are fallback reasons distinguishable from policy stops and caller cancellation?
4. Is an estimate visibly separate from metered usage and reconciled billing?
5. Is the budget checked before an attempt, and what happens when the estimate is unavailable?
6. Can operators see that a “success” required retries or changed provider/model behavior?
7. Are high-cardinality and sensitive telemetry fields intentionally bounded?

A gateway is an attribution system before it is a router: if it cannot explain the attempts behind a result, fallback may improve availability while making behavior and estimated consumption harder to understand.