Multi-agent LLM consensus isn't verification — it's a shared echo
Three LLM agents agree on the code — and all fail the same way. Why consensus is an echo, not proof, and what to use instead of voting.
Three model instances "voting" on one block of code is no verification at all. It looks solid: the task goes out to three agents, the agents compare results, and if they agree we tick the box marked "confirmed fact." On a demo it's impressive. The trouble is that the confidence you feel right then is fake. And that fakery is exactly what's dangerous, because it nudges you to ship the result onward without a second look.
A pattern that only looks like verification
I've seen this pattern for the hundredth time with AI agents. It always runs the same way:
- The task goes to three model instances. Same prompt, same context, three separate runs.
- The agents compare answers. They look for the spots where they disagree.
- If they agree, the result ships as a "confirmed fact." And it moves deeper into the system.
It all seems to work. But "works on screen" isn't the same as "gives you certainty." Multi-agent consensus looks like an independent check. It isn't. Three runs of the same model aren't three independent sources, just one source run three times.
Agents don't seek truth, they seek probability
That's the heart of the consensus trap. A model doesn't aim at the truth, it aims at the most probable continuation. If there's an error buried in the training data, or an important file got missed in the loaded context, all three agents make the same mistake in the same way.
They reinforce the same hallucination in unison, because that's what the statistics tell them. Three models failing identically don't give you three independent votes. They give you one vote repeated three times, only louder.
On a demo you can't see this. Three matching answers look like a triple check, so nobody digs in. The problem only surfaces in production, where the same error repeats on every call and sounds just as confident each time. The smoother the answer, the harder it is for anyone to stop and question it.
WHEN MULTI-AGENT IS JUST AN ECHO
- A shared error in the training data. All three inherit the same distortion and repeat it in agreement.
- A file missed in the context. Nobody sees what they were never handed.
- The same prompt, the same frame of thought. When the input converges, the output converges too.
This is what I do hands-on — advising on AI strategy and building agents that survive the demo.
Consensus is an echo, not proof
This brings us back to the problem I unpacked in the post on model self-evaluation. A model grading its own work can't step outside its own assumptions. Going from one agent to three fixes nothing if they all see the world through the same probability distribution.
Treating model agreement as proof is asking for trouble. In production, that kind of system lets through errors that sound remarkably convincing, because three models signed off on the same falsehood in unison.
"Agreement is not verification. Verification needs an external counterweight — something that doesn't vote along with the models."
Confrontation instead of a mutual-admiration circle
Instead of building a mutual-admiration circle, build confrontation. This is where adversarial verification starts. The agents get an explicit instruction: poke holes in each other, swap roles, hunt for faults. It genuinely helps. A model in the critic's seat finds what the same model, as author, was defending a moment earlier.
The difference is subtle, but it changes everything. In a mutual-admiration circle every agent has an incentive to agree. On an adversarial panel each one has an incentive to prove the other wrong. The point isn't to argue for its own sake, it's to make the third and fourth pass actually come from a different angle than the first.
- Adversarial roles. One agent writes, the other has a single job: break it.
- Forced role swaps. The author becomes the reviewer, the reviewer becomes the author.
- A mandate to disagree. You reward the hole that gets found, not the nod of approval.
Confrontation between models still isn't enough to sleep soundly. In the next post in this agentic engineering series I'll break down how to design an adversarial panel and how to beat the judge panel's position bias — a judge favoring an answer purely because of where it sits in the running order.
What that "external weight" actually is
An external counterweight is something the model can't talk its way around. The code either compiles or it doesn't. The tests pass or they don't. The types line up or the build throws. It's not one more opinion inside a probability distribution, it's a mechanical verdict that doesn't care how convincing the answer sounds.
That's why a deterministic gate beats three agreeing models. Three models can be wrong together. A compiler doesn't vote. An integration test doesn't look for a compromise. It's red or it's green. That's the reference point consensus can never produce on its own.
Without a hard test, you have no quality system
Confrontation raises the bar, but it doesn't close the case. If at the end of the process you don't have a hard, deterministic test that ruthlessly rejects a wrong result, you don't have a quality system. You have well-organized guessing. Would you trust code signed off by three different models, knowing it passed no external test at all?
