AI Researchers & Engineers - Agent Self-Review Failure Modes

🪞 There's a piece of AI advice that sounds obviously right and is often dangerously wrong: "Just have the agent check its own work." In practice, asking a model to critique itself can make it more confidently wrong — not less. Here's why, and when it actually works. The appeal is real. Two well-known patterns — Reflexion (Shinn et al., 2023) and Self-Refine (Madaan et al., 2023) — let an agent improve without retraining. The agent attempts a task, gets feedback, writes itself a short note about what went wrong, and reads that note before the next try. No weight updates. Just words. And when it works, it's striking. In the Reflexion paper, an agent paired with this loop solved 𝟭𝟯𝟬 𝗼𝗳 𝟭𝟯𝟰 AlfWorld tasks — improving over roughly a dozen trials, each time writing itself a short lesson from the last failure and reading it before the next attempt. Nothing in the model changed. Only the context it carried in did. 🪤 But here's the trap teams walk into. Self-critique only works if the model can actually tell good output from bad. And a model that is confidently wrong is usually just as confident that its wrong answer is right. Ask it to review itself and it says "looks correct to me" — and refines nothing. The loop doesn't catch the error. It launders it into something that looks more polished and is exactly as wrong. The thing that makes these loops reliable isn't the model's opinion of its own work. It's an 𝗲𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝘀𝗶𝗴𝗻𝗮𝗹 𝗶𝘁 𝗰𝗮𝗻'𝘁 𝘁𝗮𝗹𝗸 𝗶𝘁𝘀 𝘄𝗮𝘆 𝗮𝗿𝗼𝘂𝗻𝗱: → a failing test → a retrieved fact that contradicts the claim → a schema/validator that rejects the output → a human saying "no" Give the critique something real to push against, and it improves. Leave it to grade its own homework, and it reinforces whatever it already believed. 🏦 This is why, in the regulated systems I build, "the agent reviews its own output" is never a control by itself. The review has to terminate in something verifiable — a check that can fail — or it's theater. The takeaway: 𝗥𝗲𝗳𝗹𝗲𝗰𝘁𝗶𝗼𝗻 𝗶𝘀 𝗻𝗼𝘁 𝗶𝗻𝘁𝗿𝗼𝘀𝗽𝗲𝗰𝘁𝗶𝗼𝗻. It's only as good as the signal you anchor it to. If your agent's failures aren't diagnosable — if it can't see or name what went wrong — a reflection loop does nothing but add cost and false comfort. This is exactly the kind of trap I'm writing about in Agentic AI: The Practitioner's Guide — not another intro to LLMs, but the failure modes and design tradeoffs of shipping agents in regulated production. Reflexion and Self-Refine get the full treatment in Chapter 1. I'm publishing it free, chapter by chapter, as I write — follow along if you're building this for real, and tell me where your experience differs. 📘 Read along 👉 link in the comments. 👉 Have you seen an AI "self-review" catch a real mistake — or just polish a wrong answer? What was the signal that actually caught it? #AgenticAI #AIEngineering #LLM #GenAI #MachineLearning