The Agentic Engineering Radar — report (Q2 2026 edition)

A companion to the Agentic Engineering Radar — 28 practices for building LLM systems across 4 sectors × 4 maturity rings. Author: Szymon Paluch. „Deterministic control over non-deterministic AI — from the trenches, not the slides."

Sections 1–3 are open; the deep-dive (4–8) sits behind an email gate. Take the „AI-Loops Readiness" assessment to find out where you land on the radar.

Executive summary + 5 takeaways

This document unpacks the Agentic Engineering Radar — a map of 28 practices for building LLM systems, sorted not by what sounds good on a conference stage but by what actually delivers in production. Each of the 28 entries sits in one of four maturity rings (Production / Pilot / Experiment / Silent-failure trap) and in one of four sectors (Building 6, Orchestration 6, Closing the loop and trust 8, Operations and HITL 8). The radar was built from the trenches, not the slides: it ranks techniques by the two axes that really decide an implementation's fate — who holds the control flow (the model or you) and whether there's a loop that verifies the output instead of taking it on faith. The red thread is single: AI scales the mess, engineering scales the quality. Stop prompting — build loops. Then one person delivers like a team, because the output is production, not slop.

Mind two different dimensions you must never confuse: the maturity ring ≠ the X/Y map quadrant. The ring tells you how battle-tested a given mechanic is. The quadrant tells you who holds control and whether a closing loop exists. An entry can sit in the Production ring as a mature mechanic and still not close the quality loop — and then it lands in Fragile control (Y<50), not in Production operator. That's the case with Structured output („Production with an asterisk" — the schema validates the shape, not the quality), Context engineering, Skills and gated MCP: solid building blocks, but on their own they don't verify the output.

The five takeaways the edition rates as most important:

• Everything that goes „wow" on stage sits in the trap or experiment ring. The mega-prompt, multi-agent „consensus", full ReAct/autoplanning, computer-use, the model grading itself — these are demos that look great live and quietly blow up on Thursday at 2 a.m. Stage appeal is inversely correlated with production readiness. The more something wows the audience, the more carefully you should treat it in a real system.
• Eight production entries from the Orchestration, Closing-the-loop and Operations sectors describe one and the same pattern: a human holds the control flow + a loop closes the quality. Deterministic workflow, HITL-gate, durable execution, RATCHET, JUDGE-PANEL, WATCHDOG, verify-before-judge and the append-only audit log — none of these entries hands control flow to the model and none trusts the output without external closure. That's no accident: those exact eight sit in the Production operator quadrant (X≥50, Y≥50) — the only one where you can sleep soundly. This is NOT the same as the whole Production ring: the remaining production entries (Skills, Structured output, Context engineering, gated MCP) are mature mechanics that validate the shape or supply building blocks, but don't close the quality loop themselves — and some of them sit in Fragile control (Y<50). Production maturity isn't enough; the loop decides the quadrant.
• The most dangerous traps fake success — status=ok, output=garbage. A silent fallback with no alert (per Datadog, ~1/3 of agent errors are plain rate-limits nobody saw), a model/prompt change with no versioning, a mega-prompt regression after editing one fragment. The system doesn't shout, the dashboard glows green, and the quality dropped long ago. That's why a separate „Silent-failure trap" ring exists on the radar — because the absence of a red light is not the same as correctness.
• Verification without an external yardstick is theater. The model grading itself (self-preference bias — the judge likes its own answers), trusting the confidence-score (confidence ≠ correctness), multi-agent „consensus" (agents agreeing reads like verification, but it's an echo of the same hallucination) — all three simulate control without delivering it. Real closure needs an immutable reference point from outside: RATCHET (immutable yardstick, keep-if-better/revert — proof: 0.83 Spearman on OffBall), a golden dataset as a regression gate, an adversarial judge with position swap. If a system grades itself, it has no verification — it has a mirror.
• A model swapped underneath you is a hidden bomb. The provider quietly bumps the version, you swap in a „faster" model to save money, someone tweaks the prompt „just a little" — and without versioning + a golden dataset in CI, a silent regression ships to production without a single log line. It's the cheapest disaster to avoid (an eval gate is enough) and the most common one to miss.

The map to the four methodological quadrants:

The rest of the document breaks each of the 28 entries down to first principles and shows how the three closing patterns — RATCHET, JUDGE-PANEL, WATCHDOG — move a system from the left half of the map to the quadrant where one person delivers a team's quality.

Methodology: two axes (how I read the radar)

The Agentic Engineering Radar is 28 entries laid out on a two-axis map. The map doesn't say „this is good and that is bad". It says something harder: where a given technique lands if you treat AI as production, not as a conference demo. Placement comes from trench experience — from systems that run at a client's and that someone has to fix when, at 3 a.m., they return garbage with a status of ok. Not from an analyst's survey by someone who never deployed.

X axis — Execution control (who holds the control flow)

The horizontal axis answers one question: who decides what happens next — the model or you?

• Left (X < 50): the model decides. You hand control to the LLM. It picks the next step, the next tool, the next branch. Full ReAct, autoplanning, „give the agent a goal and watch". Flexible, demo-friendly, great on stage. In production — non-deterministic and hard to reproduce when something goes wrong.
• Right (X ≥ 50): you hold the control flow. The flow is yours — code, graph, workflow. The model is called for a specific step, gets a narrowly defined task, returns a result, and the control logic stays on your side. This is exactly „stop prompting, build loops": one person delivers like a team, because the output is repeatable.

The X axis doesn't measure intelligence. It measures where the non-determinism sits. The further right, the fewer places where the model can quietly drift off course.

Y axis — Closure / verification (do you take it on faith, or check it)

The vertical axis answers the second question: what happens to the output before you call it done?

• Bottom (Y < 50): taken on faith. The model returned something, status ok, we move on. No verifying loop. You assume that since no exception fired, the result is good. This is where silent failure lives.
• Top (Y ≥ 50): the loop verifies. The output passes through closure — a deterministic check, a judge panel, a ratchet, a watchdog, a golden dataset in CI. Before anything moves on, something checked it against a rule the model can't bend.

The Y axis is the heart of the thesis „AI scales the mess; I scale the quality". Without closure you only scale the speed of generating slop.

Color — silent-failure risk (100 − Y)

Every entry has a color computed straight from the Y axis: silent-failure risk = 100 − Y. The lower a technique sits (less verification), the hotter it glows. It's not decoration — it's a reminder that the most dangerous thing isn't what loudly throws an exception, but what returns status=ok with output=garbage. Red on the map = „nobody here is watching the model's hands".

The four quadrants

The intersection of both axes at 50/50 gives four quadrants. This is how I read each of the 28 entries.

The arrow the whole radar points along runs from the bottom-left corner (Roulette) to the top-right (Production operator). The point isn't to get every entry into the top-right corner — it's to be aware of where you stand and exactly what holds you there.

„From the trenches, not surveys" — how this differs from Gartner

A classic radar/Magic Quadrant is built from surveys, vendor interviews and an aggregation of market opinion. Great for an exec slide, useless when you have to deploy an agent loop that won't blow up. Here placement comes from production experience — from what really survived contact with a client, not from what respondents check most often. This is deliberately anti-Gartner: I don't ask the market what's trendy; I say where a technique lands when you're on the hook for its output. That's why, for example, „multi-agent consensus as a gate" doesn't get promoted for hype — agents agreeing reads like verification, but it's often an echo of the same hallucination, so it lands in the Trap, not in Production.

The four maturity rings

Regardless of X/Y position, each of the 28 entries gets a ring — an answer to „how much can you rely on this today, in June 2026":

The last one matters most. The Silent-failure trap ring is the only one you won't find on a radar glued together from surveys — because nobody in a survey will admit that their „verification gate" is a hallucination checked by another hallucination. It's the place for techniques that sound mature but are a trap in production: self-grading as the only verification, trusting the confidence-score, a mega-prompt with no evals, a silent fallback with no alert. Recognizing them is half the engineering of agent loops — because silent failure doesn't shout. You have to catch it before it reveals itself on the client's invoice.

Silent failure — why the outer ring exists

The worst AI error isn't the one that throws a stack trace. It's the one that comes back with status: ok and looks done — while inside it's garbage. I call it silent failure: the system reports success, the output is false, and nobody finds out until someone makes a decision on that garbage. That's why, across the whole radar, silent failure isn't one of the maturity rings — it's a separate, signature outer ring. It's not „a less mature version of production". It's a different category of risk that can touch any of the 28 entries — including the ones that otherwise look mature.

Definition: two states that have to agree

A classic software bug breaks one thing: the program crashes or returns an error. That's easy to catch — there's an exception, an alert, a red log. An AI failure breaks the gap between two axes at once:

The crux: status measures whether the pipeline ran. It does not measure whether the output is true. Those two only get confused in a deterministic world, where „it ran" implies „it did what it was supposed to". In the world of non-deterministic AI they're two completely different guarantees — and most teams monitor only the first.

Why pilots die right here

Every company I come to has already tried AI. Almost none will say „it didn't work". They'll say something worse: „it worked, but we can't rely on it". That is exactly the signature of silent failure and it's the moment where most pilots die.

The mechanism is always the same. The demo is built on three pretty examples — and it works. It hits production, where the data is ugly, edge-case and adversarial. The system keeps returning ok on everything, so nobody notices the quality dropped. A few weeks later someone in the business catches a result that's obvious nonsense — and that's enough. Trust breaks not because AI makes mistakes (people do too), but because it makes them with full confidence and no warning. After one incident like that the project gets labeled a „slop generator" and lands in the freezer. Not because of a bug someone fixed, but because of a class of bug nobody saw.

That's why I sell loop closure, not another model. A company that's already been hit by slop won't buy a „better prompt". It will buy proof that this time the garbage gets caught before it reaches a decision.

Three mechanisms (anonymized)

I'll show three real patterns I've run into. Different domains, the same signature: green status, false content.

1. Hallucinated tool argument. An agent with API access calls a function with a correct schema but a made-up parameter — e.g. a record ID that doesn't exist, or a date range built from plausible-looking numbers. structured output lets it through without blinking, because the schema validates the shape, not the truth — the field is a string, so it's „valid". The tool returns an empty or random result, the agent wraps it in a confident answer, the pipeline ends ok. There's no error anywhere. There's just a wrong answer served as fact.

2. Silent fallback after a rate-limit. Under load the provider bounces some calls (429). The code has a „defensive" fallback: when the model doesn't answer, substitute a default value / the last good result / an empty object and keep going. No alert. The system reports 100% success while in reality a third of the outputs are placeholders. (In Datadog's production data — March 2026 — rate-limits are close to ⅓ of all LLM call errors, and a fallback hides them by definition.) Looks stable. It's fiction.

3. Model/prompt change with no versioning. Someone bumps the model version or „fixes" one paragraph in the mega-prompt to handle one case. They fix that one — and quietly break five others nobody tests anymore. Without a golden dataset and evals as a gate, the regression is invisible: the output still looks good, the status is still ok, and quality slipped a notch. It's not a failure at the moment of change — it's a mine that goes off weeks later, when nobody connects it to the cause.

The common denominator of all three: in none of them is there an exception to catch. Monitoring glows green.

Why the most dangerous things fake success instead of throwing an error

Because a non-deterministic model is optimized for plausibility, not for truth. Its job is to produce a plausible-looking output — and it does that just as readily when it's right as when it's making things up. A hallucination isn't an exceptional state you can catch with try/except. It's a normal, correctly executed output that just happens to be false. From the state machine's point of view, everything went perfectly.

And here's the trap I see most often: teams add „verification" that is itself silent. The agent grades its own work (self-preference bias — the model likes what it wrote itself). Or a confidence score is taken for correctness, even though confidence is not truth — the model is often most confident exactly when it's hallucinating. Or three agents are set up for „consensus", and their agreement reads like verification, while it's an echo of the same hallucination copied three times. All these moves look like closure. None of them close the loop. It's verification on paper that generates even more silent failure — because now you have a false output plus a false „verified" stamp.

That's why silent failure is its own ring

If silent failure were just „immaturity", you'd keep maturing and it would go away. But it doesn't go away with maturity — it migrates. A mature entry with missing verification (e.g. a production pipeline with no gate, multi-agent „consensus" as the gate, trusting the confidence-score) is more dangerous than a raw experiment, because it has more trust and more reach. That's why, on the 2-axis map, I compute color = silent-failure risk as 100 − Y, where Y is closure/verification. High execution control (the X axis) doesn't save you — you can have an iron, deterministic control flow and still feed it unverified garbage. The only thing that puts out the red is a loop that checks (RATCHET / JUDGE-PANEL / WATCHDOG), not order in the orchestration alone.

Hence the whole point of the radar and its red thread: stop prompting, start building loops. The outer „Silent-failure trap" ring exists to name the thing that shows up on no dashboard — and to show that it, not the crash, is the reason AI scales the mess until someone deliberately starts scaling the quality.