Demo
demo/agent-showcase/is a bespoke multi-agent coding team — an architect, three parallel coders, a Claude reviewer over /v1/messages, a fixer — that ships a real rate-limiting feature to a sample repo and proves it with node --test. It then runs the identicaltask again, served entirely from Zerocache: same diffs, same passing tests,X-Zerocache-Completion-Hit: true on every model call. A third run rewords the task and is designed to still hit, through the opt-in semantic near-match tier.
What a cached re-run looks like
Run 1 is cold — every model call is a real, billed upstream request. Run 2 is the same task, unchanged: the cacheable chat-completion and /v1/messages calls return from the store, the diffs applied to the sample repo are byte-identical, and node --testpasses exactly as it did on run 1. Run 3 rewords the brief; with a--features semantic build and ZEROCACHE_SEMANTIC=1, the paraphrased prompts match previously-served completions above the cosine threshold.
This demo is built to be run against your own keys for a live trace — it does not ship pre-recorded numbers. For a measured result, see the support-triage agent suite on the Benchmarks page: a second run was 100% cache hits, zero upstream calls, ~5.1k prompt + ~370 completion tokens saved per suite.
What it exercises
| Stage | Agent | Route | Zerocache surface |
|---|---|---|---|
| Retrieve | Architect | POST /openai/v1/embeddings · POST /gemini/v1/images/embeddings | embeddings cache · image-embeddings cache |
| Plan | Architect | POST /openai/v1/chat/completions | chat completions cache · multi-provider |
| Repo brief | 3× Coder (parallel) | one identical POST /openai/v1/chat/completions, fired concurrently | in-process coalescing (3 misses → 1 upstream call) |
| Implement | 3× Coder (parallel) | POST /openai/v1/chat/completions | chat completions cache |
| Review | Reviewer | POST /anthropic/v1/messages | native Anthropic messages cache (exact-match) |
| Fix | Fixer | POST /openai/v1/chat/completions | chat completions cache |
Every request is BYOK — Authorization: Bearer <that provider's key>, never a Zerocache key. temperature: 0 throughout (also what the cache gate requires).
Run it yourself
git clone https://github.com/shramanb113/ZeroCache
cd ZeroCache/demo/agent-showcase
npm install
cp .env.example .env # OPENAI_API_KEY required; ANTHROPIC + GEMINI optional
# in another shell — semantic build so the reworded run can hit
ZEROCACHE_SEMANTIC=1 cargo run -p zerocache-http --features semantic
npm run warm # pre-fills the cache (runs 1 → 2 → 3)
npm run -- run --run=2 # the hero shot: identical task, fully cached
npm run -- run --run=3 # the reworded task, semantic hits