THE RETRIEVAL TAX

We ran a controlled benchmark on live Tavily — not a hypothetical bill.

125 agent-style queries. Naive always-fanout (5 searches each) burned 625 Tavily calls ($5.00). Adaptive probe-first governance cut that to 173 searches ($1.38) — 72.3% savings.

Method: reproducible GAIA-style corpus · 973 total Tavily API calls · task file in repo (tasks-tavily-study.json)

THE HIDDEN COST MULTIPLIER

Agent loops compound three things:

1. Paid searches — Tavily, Exa, Brave, SerpAPI: priced per query, not per agent run
2. Fanout — one question becomes five sub-queries “for better coverage”
3. Repetition — every step, every branch, every retry

Do the math: 12 agent steps × 5 fanout queries × $0.008 = $0.48 per run in search alone. Run that a thousand times a month and you’re at nearly $500 — for retrieval, not reasoning.

Teams fixed the token side first. Helicone, LangSmith, Portkey — good tools, real category. Retrieval FinOps doesn’t exist yet. It’s still if uncertain: search() copy-pasted into agent code.

INTRODUCING RETRIEVAL FINOPS

Retrieval FinOps is the policy and infrastructure layer that governs:

• When your agent searches
• How hard it fans out
• When it stops
• What evidence is good enough

…all under a budget. This is not a search API. It’s a control plane between your agents and whatever search providers you already use.

You’re not buying a better index. You’re buying governed retrieval.

TAVILY FIELD STUDY (JUNE 2026)

125 production-style queries across factual, procedural, comparative, news, and multi-hop categories. Naive baseline: always fan out to 5 sub-queries. Governed: adaptive probe-first with research policy.

Tasks: 125
Naive searches (5× fanout): 625
Governed searches (adaptive): 173
Naive cost @ $0.008/search: $5.00
Governed cost: $1.38
Savings: $3.62 (72.3%)
Blocked under research policy: 29/125
Blocked under legal policy: 75/125

Not every query fans out less — multi-hop and thin-evidence queries still expand. Factual queries drove the largest absolute savings (270 → 64 searches in the factual bucket). That’s the point: govern spend without banning search when you need it.

Under a legal policy on the same corpus, 75/125 answers would return allow_answer: false (authority and source-count gates) — even when search spend is low. Cost routing and risk gating are separate decisions.

Legal block rates are from a controlled web-search benchmark with default policy thresholds — illustrative of gating behavior, not a compliance certification or production SLA.

Reproduce locally: scripts/run-tavily-study.sh · full write-up in docs/tavily-study-2026.md

SAME POLICY, DIFFERENT BACKENDS

We re-ran the identical 125 tasks on live Exa and Brave with the same naive vs governed setup.

Takeaway: you cannot quote Tavily’s 72.3% for every stack. On Tavily, many factual queries stopped after 1 probe — different snippet/confidence signals, same control plane.

Exa: adaptive fanout still saves (40.0%), but governed runs mostly settled at 3 searches/task (probe + partial expand). Under legal, Exa blocked 48/125 vs Tavily’s 75/125 — authority scores clustered lower on Exa URLs in this run.

Brave: same 40.0% savings pattern as Exa (625→375, $0.005/search). Governed runs also settled at 3 searches/task — Brave probe confidence rarely stops after 1 call. Under legal, Brave blocked 76/125 (nearly Tavily’s rate, but with 2× the searches).

Legal block rates are from a controlled web-search benchmark with default policy thresholds — illustrative of gating behavior, not a compliance certification or production SLA.

Tune presets per workload; don’t assume lab numbers transfer to prod.

Reproduce: EXA_API_KEY=... bash scripts/run-exa-study.sh · docs/exa-study-2026.md · 1375 Exa API calls · BRAVE_SEARCH_API_KEY=... bash scripts/run-brave-study.sh · docs/brave-study-2026.md · 1375 Brave API calls

SAMPLE AUTOPSY (8 EVENTS)

Smaller bundled log slice for the free design-partner autopsy workflow:

See sample autopsy report → (40 → 30 searches, 25% modeled savings on example logs)

WHAT THE ARCHITECTURE LOOKS LIKE

┌─────────┐     ┌──────────────────────────┐     ┌─────────────────┐
│  Agent  │────▶│  Retrieval Control Plane │────▶│ Tavily / Exa / Brave / … │
│         │◀────│  policy · cache · audit  │◀────│ Wikipedia / …   │
└─────────┘     └──────────────────────────┘     └─────────────────┘

The agent doesn’t call Tavily directly. It asks the control plane for evidence — cited, scored, budgeted.

• Ranked evidence with confidence scores
• A synthesis your team can read (for demos and internal champions)
• Conflict flags when sources disagree
• A cost receipt: queries run, queries avoided, dollars spent

Full pipeline diagram →

WHAT THIS IS NOT

• Not competing with Tavily on index quality — use whatever search API you want
• Not claiming truth — surfacing evidence and uncertainty, not certifying answers
• Not replacing RAG — governing live retrieval in agent loops, not your vector DB

If you need a Perplexity clone, build one. If you need predictable retrieval spend with an audit trail, that’s Retrieval FinOps.

TRY IT YOURSELF

pip install query-fanout
query-fanout-autopsy examples/sample_agent_logs.jsonl

query-fanout "how does photosynthesis work" --preset wikipedia

Export format for your own logs — one JSON object per retrieval event:

{"agent_id": "research-bot", "run_id": "run-001", "query": "what is photosynthesis", "searches_run": 5, "backend": "tavily"}