Prompt caching

On a 30K-token system prompt with Claude Sonnet 4.6, a cache read costs $0.12 / 1M instead of $1.2 / 1M. For an agent that re-sends the same context 50 times a session, the cache saves ~90% of the input bill.

How to enable it

Add a cache_control: { type: "ephemeral" } breakpoint on any system block, message, or tool definition. The model caches the span up to that breakpoint on the first call; subsequent calls that share the same prefix bytes hit the cache.

from openai import OpenAI

client = OpenAI(
    api_key="sk-kn-...",
    base_url="https://api.kunavo.com/v1",
)

# Mark the large, stable prefix once with cache_control.
# Every subsequent call that sends the same prefix reads it back
# from cache at ~10% of the input rate on Anthropic models.
SYSTEM = [
    {
        "type": "text",
        "text": open("long_system_prompt.txt").read(),
        "cache_control": {"type": "ephemeral"},
    },
]

resp = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": "Summarize the doc above in 3 bullets."},
    ],
)

u = resp.usage
# OpenAI-compat fields populated by Kunavo:
# - prompt_tokens_details.cached_tokens — read from cache
# - cache_creation_input_tokens          — written to cache this call
print(u.prompt_tokens_details.cached_tokens,
      getattr(u, "cache_creation_input_tokens", 0))

Identical wire-format on the native Messages API — cache_control passes through to Claude untranslated:

# Native Anthropic Messages API — cache_control passes through untranslated.
curl https://api.kunavo.com/v1/messages \
  -H "Authorization: Bearer $KUNAVO_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "system": [
      {
        "type": "text",
        "text": "<your large stable system prompt>",
        "cache_control": { "type": "ephemeral" }
      }
    ],
    "messages": [
      { "role": "user", "content": "Summarize the doc above in 3 bullets." }
    ],
    "max_tokens": 1024
  }'

Pricing

Cache rates derive from each model's input price, at the vendor's own ratios. Cache reads bill at 0.10× input on Anthropic, OpenAI and Gemini models alike. Cache writes bill at 1.25× input on every Claude model and on GPT-6 Astra, GPT-5.6 Sol, GPT-5.6 Terra and GPT-5.6 Luna. Kunavo bills Claude's 1-hour-TTL writes at the same 1.25×, below Anthropic's 2×. Every other model has no separate write price: its writes bill at the plain input rate. Either way, a prefix that is reused even once costs less cached than sent fresh. Every class carries the model's own discount, so its cache prices sit at the same discount off the vendor's list as its input and output.

Affinity routing keeps the cache warm

Upstream prompt caches are per-API-key. If every request lands on a random upstream key, the cache hit rate is ~1/N. Kunavo derives a stable hash from your system prompt + first user message and routes all calls with the same prefix to the same upstream key — using weighted rendezvous hashing, so only ~1/N keys are remapped when the pool changes.

You don't need to configure anything. Affinity routing applies automatically to all chat calls. (Image / video / TTS use random routing — there is no per-key cache for those modalities.)

Dashboard metrics

Once your calls start hitting the cache, /app/usage surfaces two rolled-up metrics for the selected window:

• Hit rate — cached input tokens ÷ total input tokens. Exact, computed from per-call token counts.
• Saved — USD saved this window, computed per model with its real cache-read rate (no flat-average approximations). This is an estimate at catalog rates — since real bills are floored at catalog and cached calls can land below the floor, the number is a tight upper bound on actual savings. For typical chat workloads the two are within a few percent.

Each call's detail page (/app/usage/<id>) shows the cached + cache-write token counts when they're non-zero, and the actual billed amount.

Response fields

Kunavo surfaces cache tokens in both response shapes. On the OpenAI-compatible /v1/chat/completions endpoint:

On the native Messages API at /v1/messages, the Anthropic-original fields pass through:

Where to go next

• Messages API reference — native Anthropic shape with worked cache examples.
• Claude prompt caching — the usage-object numbers that diagnose a cache, and the three ways routing through a gateway breaks a hit rate.
• Billing & the ledger — how the cache-aware max() floor interacts with upstream cost.
• Full pricing table — fresh input / output prices for every chat model, with the cache sub-table at the bottom.