Idempotency

Retries are safe here: send an Idempotency-Key header, and the same logical request returns the same logical response — no duplicate generation, no double billing. We shipped this for inference because nobody else has it, including OpenAI.

The problem it solves

Inference is not a read. If a network timeout makes your client retry a chat completion, the retry executes a second generation — you pay twice, and your user may see the answer twice. Payment processors solved this decades ago with idempotency keys; we ported the model to inference.

Semantics

Send a unique string of 8 to 256 bytes as the Idempotency-Key header:

• Same key, identical request body → the stored original response is replayed. The replay carries the original x-request-id (the ledger row the first execution wrote stays the correlation point) and an Idempotency-Replayed: true header so you can tell a replay from a fresh execution.
• Same key, different request body → 409 idempotency_key_reused. A key names one request; reusing it for a different one is a client bug we refuse to paper over.
• Keys are cached for 10 minutes today. That window only ever grows — it will never shrink under a live integration.
• The store is single-instance today; multi-instance replay with a longer window is scheduled next sprint. Same-key requests that race the cache simply execute — at-least-once, never double-billed beyond the executed turn.

Replay — run it twice

The second curl replays the stored response (watch for Idempotency-Replayed: true and the original x-request-id in the headers; -i shows them):

curl -sS -w '\n%{http_code}\n' $IRONSTRATUM_BASE_URL/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -H "Idempotency-Key: docs-replay-demo-0001" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Reply with the word: original"}],
    "max_tokens": 24
  }'
curl -isS -w '\n%{http_code}\n' $IRONSTRATUM_BASE_URL/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -H "Idempotency-Key: docs-replay-demo-0001" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Reply with the word: original"}],
    "max_tokens": 24
  }'

Both calls return 200 with the same response body — the second one never touched a model.

Conflict — same key, different body

Reusing a key with a different body is a 409:

curl -sS -w '\n%{http_code}\n' $IRONSTRATUM_BASE_URL/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $IRONSTRATUM_API_KEY" \
  -H "Idempotency-Key: docs-replay-demo-0001" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{"role": "user", "content": "Reply with the word: different"}],
    "max_tokens": 24
  }'
# expect: 409

What to do

• Generate a key per logical operation (a UUID or a hash of the user action) — not per session, not a constant.
• Retry the same request (same body, same key) on any timeout, connection reset, or 5xx.
• Treat 409 idempotency_key_reused as a client bug: your key generation collided across different payloads.