Sakana Fugu vs. LLM Council: Two Very Different Bets on Multi-Model Intelligence

"One model to command them all." — that's how Sakana AI pitches Fugu. "A Saturday hack to read books with LLMs together." — that's how Andrej Karpathy pitches LLM Council.

Same underlying instinct — the best answer isn't from one model, it's from a coordinated team of models — but two radically different execution philosophies. One is a productized, research-grade orchestrator sold behind an OpenAI-compatible endpoint. The other is a 99% vibe-coded local web app that fits on a laptop.

This post is a deep dive into what Sakana Fugu actually is, how it works, and how it compares with LLM Council.

Executive Summary

Multi-model orchestration is quietly becoming the next frontier in Generative AI — the belief that no single model, however large, will consistently beat a well-coordinated team of models. Two projects, released within months of each other, put this thesis to the test from opposite ends of the spectrum:

Sakana Fugu (2026) — a productized, research-backed system from Sakana AI that hides a learned coordinator (Trinity + Conductor) behind a single OpenAI-compatible API. It dynamically assembles frontier models into Thinker / Worker / Verifier roles, loops until a verifier is satisfied, and bills you a single blended rate. It targets autonomous, long-horizon agentic workloads — coding, paper reproduction, security assessments, Kaggle-style research.
LLM Council (2025) — Andrej Karpathy's tiny, transparent, local web app that fans a query out to a panel of frontier LLMs, has them anonymously peer-review each other, and lets a Chairman model synthesize the answer. It targets humans doing hard thinking — reading, evaluating, comparing.

Fugu optimizes for answer quality on autonomous tasks with opaque routing. Council optimizes for transparency and human insight. Both are early signals that the "one giant model" era is peaking and the next axis of progress is coordination — but each is pointed at a fundamentally different job-to-be-done.

If you're building an agentic product and want frontier quality without single-vendor lock-in, Fugu is the more serious answer today. If you're a human trying to reason through a hard question and want to see where the frontier disagrees, Council is unbeatable. This post breaks down how each works, where each breaks, and how to choose between them.

1. The problem both are trying to solve

Frontier LLMs — GPT-5.x, Claude 4.x, Gemini 3.x, Grok 4, etc. — are astonishingly capable, but each one:

has different strengths (Claude tends to reason carefully, GPT tends to code confidently, Gemini tends to search and synthesize, Grok tends to be bold)
has different failure modes (hallucination patterns, refusal patterns, long-context regressions)
comes with vendor lock-in, price shocks, and geopolitical/export-control risk

If you combine them intelligently — let them critique each other, delegate to each other, verify each other — you can plausibly beat any single one. That's the shared thesis. Everything else is engineering.

2. What is Sakana Fugu?

Sakana Fugu is Sakana AI's new offering (2026). It packages a multi-agent orchestration system behind a single OpenAI-compatible API. From the outside it looks like just another model endpoint. Inside, it's a learned coordinator that dynamically assembles a team of frontier LLMs for each query.

Two SKUs:

Model	Positioning
Fugu	Balanced latency / quality. Drop-in default for coding, code review, chatbots. Supports opting individual providers out of the agent pool for compliance.
Fugu Ultra	Deeper pool, maximum quality. Used for Kaggle competitions, paper reproduction, cybersecurity assessments, patent landscape analysis. Pool is fixed.

Both models are billed and served through one endpoint — you switch by changing the model name, not your SDK.

2.1 The pool metaphor

Sakana's own visual for Fugu is a fish (the fugu) picking teammates out of an LLM pool — closed-source and open-source models sitting alongside Sakana's own model.

The important bit: Fugu is not a router that picks one model per request. It picks a team, assigns roles, runs multi-turn coordination, and returns one answer.

2.2 The architecture: coordinator, roles, verifier

Under the hood, Fugu is grounded in two ICLR 2026 papers from Sakana:

TRINITY: An Evolved LLM Coordinator — a lightweight evolved coordinator that hands out Thinker / Worker / Verifier roles across turns.
Learning to Orchestrate Agents in Natural Language with the Conductor — an RL-trained "Conductor" that discovers natural-language coordination strategies (which agent talks to which, and with what prompt).

Put together, the runtime loop looks roughly like this:

Three things worth calling out:

The coordinator sometimes solves directly. Not every query needs a committee — trivial ones are answered inline. This is the "fugu solves it itself" path.
The verifier can loop. If the verifier isn't satisfied, control goes back to the coordinator, which can re-team, re-prompt, or escalate. This is why Fugu Ultra can spend hours on a single hard problem (paper reproduction, Kaggle).
The coordination strategy is learned, not hand-written. This is the biggest differentiator from every "multi-agent framework" you've seen on GitHub. Nobody wrote "for coding, ask Claude, then have GPT critique." A coordinator was trained (evolutionary search + RL) to discover such patterns.

2.3 What Sakana claims

From the Fugu benchmarks page:

Beats publicly accessible frontier models on SWE-Bench Pro, LiveCodeBench (Pro), GPQA-D, TerminalBench 2.1, Humanity's Last Exam, CharXiv Reasoning, and more.
Shoulder-to-shoulder with non-public frontier "Fable 5" and "Mythos Preview" — while being reachable via a normal API and outside export-control chokepoints.
On an AutoResearch-style GPT training loop (Karpathy's setup), Fugu-Ultra reached the best mean BPB (0.9774) across 123 experiments in ~14 hours on a single H100, beating three anonymized frontier baselines.

2.4 Pricing model (worth understanding)

Fugu's pricing is subtly clever:

Fugu — you pay the standard rate of whichever model is active. When multiple agents are active, fees don't stack — you pay a single blended rate based on the top-tier model in the pool.
Fugu Ultra — fixed pricing: $5 in / $30 out per 1M tokens, jumping to $10 / $45 for contexts >272K.
Subscription tiers ($20 / $100 / $200 per month) for casual use.

Translation: enterprises don't get hit with N× cost for using N models. That's a real go-to-market wedge.

3. What is LLM Council?

LLM Council is Andrej Karpathy's small, local, "99% vibe coded" web app (22.3k stars at time of writing) that he built on a Saturday to help him read books with LLMs. It looks like ChatGPT, but every query fans out to a panel of frontier LLMs — via OpenRouter — who then critique and rank each other, and finally a Chairman LLM writes the final answer.

Default council in backend/config.py:

COUNCIL_MODELS = [
    "openai/gpt-5.1",
    "google/gemini-3-pro-preview",
    "anthropic/claude-sonnet-4.5",
    "x-ai/grok-4",
]
CHAIRMAN_MODEL = "google/gemini-3-pro-preview"

3.1 The three-stage flow

Key design choices:

Anonymization for judging — models don't know whose answer they're reviewing, to reduce brand bias.
All opinions are visible — the tabs let you be the real judge; the Chairman is a convenience.
Chairman is configurable — pick whichever model you trust to synthesize.
OpenRouter as the transport — one API key, many providers.

3.2 What it's for

It is explicitly not a production system. Karpathy's README literally says:

"I'm not going to support it in any way, it's provided here as is for other people's inspiration."

It's for humans in the loop — reading books, exploring hard questions, wanting to see which model disagrees with which. The value is transparency, not throughput.

4. Sakana Fugu vs. LLM Council — side by side

Dimension	Sakana Fugu	LLM Council
Origin	Sakana AI research + product, 2026	Karpathy Saturday hack, 2025
Delivery	Hosted, OpenAI-compatible API (`fugu`, `fugu-ultra`)	Local web app (FastAPI + React + OpenRouter)
Coordination strategy	Learned (evolutionary + RL) — Trinity / Conductor	Hand-coded three-stage pipeline (fan out → review → chair)
Roles	Dynamic: Thinker / Worker / Verifier per turn	Fixed: Members + Chairman
Loops	Verifier can send work back to coordinator over many turns	Single-shot pipeline (no revision loop)
Model pool	Frontier closed + open + Sakana's own; opt-out for Fugu, fixed for Ultra	User-editable list in `config.py`
UX	One answer, one endpoint; routing is opaque by design	All opinions shown as tabs; full transparency
Judge bias mitigation	Learned; not documented as anonymized	Explicit anonymization for peer ranking
Best for	Agentic coding, paper reproduction, security assessments, long-running autonomy	Exploring a hard question yourself, reading, evaluating models side-by-side
Cost model	Blended single-model rate; no fee stacking; Ultra is fixed $/token	You pay OpenRouter for every call to every council member (fees do stack)
Vendor lock-in	Sakana becomes the abstraction layer	You own the code and the router config
Data governance	Opt-in training; can opt out of specific providers on Fugu	Fully local, your keys, your machine

4.1 The philosophical fork

The two systems answer "how do we combine LLMs?" differently:

Fugu says: the coordination policy is itself a machine-learning problem — train it, evolve it, hide it, sell it. The API caller shouldn't (and can't) see which model did what.
Council says: the coordination policy is a human-reasoning tool — expose every opinion, let the human see disagreement, and use a simple Chairman synthesizer as a courtesy.

Fugu optimizes for answer quality on autonomous long-horizon tasks. Council optimizes for human insight on hard subjective questions.

Neither is wrong. They're pointed at different jobs-to-be-done.

4.2 Where each one breaks

Fugu weaknesses

Opaque by design. You can't see which model answered — a real problem in regulated industries where you need to attribute which model produced which claim.
Latency for Ultra is high (deep pool + verifier loops).
You depend on Sakana's coordinator quality; if the learned policy regresses, your agent regresses invisibly.
Not available in EU / EEA at launch.

Council weaknesses

Costs stack: 4 members + 1 Chairman = ~5× the tokens of a single model, every query.
No revision loop — if all four members are wrong in the same direction (very common on frontier-adjacent problems), the Chairman just averages the wrongness.
Peer review is one-shot; models don't get to argue back after being critiqued.
Not designed for autonomy — it's a chat UI, not an agent runtime.

5. When to use which (practical guide)

Reach for Sakana Fugu when:

You're building an agentic product — coding assistant, code review bot, security assessment agent, research automation — and you want frontier quality without picking a single vendor.
You care about resilience: if one provider goes down or gets export-restricted, Fugu keeps working.
You want one bill, one endpoint, one SDK.

Reach for LLM Council when:

You're a human doing hard thinking (reading, writing, analyzing) and you want to see how the frontier disagrees before deciding.
You want full local control, your own API keys, your own model list.
You're using it as an evaluation harness to compare models on your own prompts.

Reach for neither when:

The task is trivial and one frontier model already crushes it — don't pay the coordination tax.
The task is highly domain-specific and a fine-tuned small model would beat any generalist committee.

6. The bigger picture

Both projects are pointing at the same thing everyone in the field has quietly agreed on: the "one giant model" era is peaking, and the next axis of progress is coordination.

Karpathy is showing the shape — fan out, review, synthesize, keep humans in the loop.
Sakana is showing the product — hide the shape behind an API, learn the policy, and sell the outcome.

If Fugu's benchmarks hold up in independent evaluation and if the "no fee stacking" pricing survives contact with reality, this is a real category, not a demo. And if it does become a category, expect every hyperscaler to ship their own "council behind one endpoint" within a year.

Meanwhile, the LLM Council repo will keep being what it always was — a tiny, transparent, hackable tool that shows you the idea in 800 lines of Python and React.

Both are worth understanding. Only one is worth putting behind your production traffic today. Which one depends entirely on whether you're building for a human or for an agent.

References

About the Author

Siddhesh Prabhugaonkar is a Generative AI & Agentic AI Enablement and Adoption Specialist with two decades as an Architect, Consultant, and Trainer across IT, Cloud, and Generative AI. He is a Microsoft Certified Trainer, a Pluralsight Instructor, and helps enterprises move from GenAI curiosity to production adoption at scale.

His consulting and training practice spans GenAI, Azure, Microsoft Foundry, Claude, GitHub Copilot, Cursor, Windsurf, and modern full‑stack engineering (.NET, MEAN, MERN). Notable engagements include GenAI enablement for ADP, IoT platform consulting for IIT Bombay's E‑Yantra program, and early work on Microsoft's Repository platform (which later became Entity Framework).

Empowering organizations and individuals to adopt, build, and scale with Generative AI, Cloud, and Modern Software Engineering.

Connect & explore:

💼 LinkedIn — linkedin.com/in/siddheshprabhugaonkar
📝 Blog — azureauthority.in
📬 Newsletter — cloud-authority.com
🎥 YouTube — youtube.com/c/SiddheshPrabhugaonkar
🤝 Book a 1:1 on Topmate — topmate.io/siddheshp
🎓 Research Papers (Google Scholar) — scholar.google.com

Sakana Fugu vs. LLM Council: Two Very Different Bets on Multi-Model Intelligence

Executive Summary

1. The problem both are trying to solve