Benchmarks

Every number on this page came from a real battle-test against a live local Zerocache instance with real provider keys — not a synthetic benchmark.

A support-triage agent suite, re-run

A real ReAct tool-calling agent (search_kb / get_order /resolve_ticket over in-script data) resolves a 3-ticket suite — 10 model calls plus 9 tool calls per run — against the completion cache, live-smoke-tested through Gemini's OpenAI-compatible endpoint with a real key.

100%cache hits on the second run — zero upstream calls
byte-identicalticket resolutions across both runs, every tool-call turn included
~5.1k + ~370prompt + completion tokens saved per 3-ticket suite

The demo/completion-cache/harness (battle-test.mjs and agent.mjs) reproduces this. The largerdemo/agent-showcase/— an architect, three parallel coders, a Claude reviewer over /v1/messages, and a fixer shipping a rate-limiting feature — is built to run cold, then cached, then reworded (semantic tier); run it yourself for a live trace.

Re-indexing an edited corpus

A 9-document corpus, re-indexed after editing one document and adding another. The other 7, byte-identical between versions, were served from cache for free.

Miss vs. hit latency

Measured in the Mastra agentic battle-test, same corpus, before and after caching.

Ingest (per doc) — 196× faster

983ms5msbeforeafter

Query — ~600× faster

~600ms~1msbeforeafter

Consumer battle-tests — three independent frameworks

18/18Mastra agentic checks passed
18/19LlamaIndex/Python real-key tests passed (1 external rate-limit)
17/17LangChain/TS real-key tests passed

LangChain/TS (a raw embedding client, OpenAI) found the bare-string input gap and the first measured coalescing number. LlamaIndex/Python (Gemini, a second language ecosystem) confirmed the fixes held against a second provider. Mastra (an agentic pipeline — an LLM deciding which tool to call, not a direct embedding client) confirmed the same behavior holds when Zerocache is several layers removed from the caller, and exercised theDELETE routes end-to-end for the first time.