Model self-evaluation: the judge who signs off on its own verdict
A model grading its own work always gives itself an A. The most common agentic engineering mistake — and the two-layer verification that fixes it.
A judge who reviews their own verdict always acquits. That's the shortest definition of the mistake that breaks 80% of agentic projects. You build a system where a model generates an output, and then the same model — or its twin — gets handed the task: "check whether this is correct." And the model checks. Always in its own favor.
LLM self-evaluation is a conflict of interest
The outcome is predictable. The model waves through its own hallucinations, because in its own logic they sound convincing. Same network, same weights, same blind spots. You're asking an author whether their own text is any good. What did you expect to hear?
This is the classic LLM-as-judge pointed the wrong way. The judge and the defendant are one model. Nobody shouts, no red alert fires — the reports glow green, and the worst kind of failure is exactly the one you can't see.
Take an agent that pulls data off invoices and writes it to a database. It got an amount wrong, but it got it wrong consistently: the same bad number went into the "amount due" field and into the summary. You ask the model whether the extraction is correct. It sees internal consistency and answers "yes," because from where it stands everything lines up. The error sits in the input the judge never got to see.
The dashboard shows "Status: OK," but garbage is landing in your database. The system runs — it just quietly poisons the product from within.
A "second pass" doesn't fix it
The natural reflex is to run the output through the model one more time. Or stand a twin next to it and have it check its colleague. Except the twin inherits the same blind spots. If the original made up an API endpoint that doesn't exist, a copy with the same training will judge it plausible and rubber-stamp it. You've added cost and latency, and confidence only goes up on paper. That isn't verification, it's an echo.
This is what I do hands-on — advising on AI strategy and building agents that survive the demo.
AI agent verification in two layers
In agentic engineering I don't ask the model whether it slipped up. I split verification into two layers that don't trust each other.
TWO LAYERS OF VERIFICATION
- Hard. You verify in code — schema, test, whether the fact checks out. Deterministic, binary. If something can be checked in code, never send it to an LLM.
- Soft. You stand up a separate model, with a different prompt. It takes no part in generating. It only audits.
The hard layer is boring, and that's exactly why it works. A JSON schema either validates or it doesn't. A test passes or it fails. A number matches or it doesn't. There's no room for "sounds convincing" here. Most teams push things into an LLM that a plain validator would settle in microseconds, and then they pay for it with hallucinations in production. You check whether the code compiles, whether the URL responds, whether the ID exists in the database. You don't ask a model about any of it, because the model guesses and the compiler knows.
The soft layer is everything you can't check in code: whether the answer stays on context, whether the tone fits, whether the reasoning holds. Here you do need a model, but a separate one. Different prompt, different role, zero hand in producing the output. An auditor, not a co-author. If the same model writes and grades, you've got theater, not control.
Order matters too. The hard layer screens out anything that's unambiguously wrong first, and only what survives reaches the soft auditor. That way the expensive model doesn't burn tokens catching typos in your JSON, and you get two independent gates instead of one that roots for itself.
The Agentic Engineering Radar
Over the last few months I tested these patterns in the field, not on slides. I kept 28 that actually hold up in production. The rest I threw out. I arranged them into the Agentic Engineering Radar: 4 sectors and 4 maturity rings, from clean production out to "Silent Failures" on the far edge. The closer to the center, the more predictable the system. The closer to the edge, the more things break quietly, without a single red alert.
Model self-evaluation is only the first pattern. In the posts that follow I show why multi-agent consensus isn't verification and how to spot silent failures in agentic systems before they poison production. One post, one concrete problem, one fix. I'll walk the whole radar live, then put it up for download.
The simplest test you can run today
Go into your pipeline and find the spot where a model grades its own output. Then ask one question: what here can be checked in code? Move that to the hard layer today. Hand the rest to a separate auditor that never touched the generation. That's enough to make "Status: OK" mean something again. The rule is simple, and it has no exceptions: nobody signs off on their own verdict. Which of these silent failures have you already caught in your own system?
