Your AI Agent Didn't Crash. It Started Doing the Wrong Thing.
Synergised Consulting Ltd
Last Updated: 17 November 2026
When an AI agent breaks in production, it usually does not crash. The run completes, the health checks stay green, and the output is quietly worse. Two open-source releases this month attack exactly that: Google's AQuA watches an agent's real conversations, clusters what went wrong and traces it back to a line of code, and Cloudflare's Kitesurf gives the agent a browser you can inspect while it works. For the first time, the tools to diagnose a silent agent failure, not just detect one, are open source.
What Actually Shipped
Two tools shipped, and they cover two different halves of the same problem. AQuA, the Ambient Quality Agent from Google, is an open-source quality agent that runs beside your production agent, reads its conversation traces on a schedule, and turns failures into verified, code-anchored findings. Kitesurf, from Cloudflare, is an experimental browser engine built for agents: it renders pages without a Chrome container and speaks the Chrome DevTools Protocol, so whatever an agent browsing the web is looking at can be inspected with ordinary developer tools. One watches what the agent decided; the other shows what the agent saw.
According to Google's developers blog [1], AQuA samples up to 1,000 recent sessions per run from Cloud Trace, Cloud Logging or BigQuery, grades each against a nine-point checklist of common breakdowns, clusters the findings that share a mechanism, and then runs a separate verification model against full transcripts before anything enters the defect queue, a pipeline InfoQ's coverage of the release describes in detail [2]. Verified issues are tracked as NEW, RECURRING, or auto-RESOLVED after 14 days unseen, as Google's developers blog and InfoQ's coverage of the release both describe [1] [2]. It never sits in the request path and never applies a fix itself: diagnosis is proposed, a human decides.
Why Agent Failures Stay Invisible
A silent agent failure is hard to see because the system reports success at every layer that has monitoring. The process ran. The API returned 200. The deploy pipeline passed. None of that says the agent did the right work. Google's developers blog [1] names four layers that can look identical from the outside: the model hallucinated an argument or dropped a constraint from three turns earlier, the orchestration harness routed to the wrong sub-agent, a tool rejected an input because its valid values were never documented, or the instructions simply never contained a rule nobody thought to write. Those four failures have different owners and different fixes, and a raw log of what happened next to what does not tell you which one occurred.
A worked example, straight from the release
The clearest evidence comes from Google's own demonstration run [1]. In a 32-session sweep of a travel-planning agent across four scripted journeys, 27 sessions produced failures and 42 structured findings. Those clustered into 9 candidate issues, and the separate verification model rejected 3 of them as false positives, leaving 6 verified defects. One example: when a user named a seat number alongside their flight choice, the planning agent saved it directly into session state without ever calling the seat-availability agent. It returned success, HTTP 200, zero tool errors, on a seat it had never checked existed.
Two one-line prompt fixes later, a replay of the same 32 sessions showed the seat bypass falling from 15 sessions to 2, dietary-constraint dropping from 7 sessions to zero, and full-session passes more than doubling from 5 to 13. On an 87-trace internal benchmark, the same verification step rejected 4 of 24 candidate clusters [1]: the pipeline is designed to throw findings away rather than hand you unverified hypotheses that look like facts.
Kitesurf: Showing What the Agent Actually Saw
The second half of the problem is the agent's own eyes. When an agent drives a browser and the result is wrong, you currently debug whatever artefacts it left behind. Kitesurf, open-sourced by Cloudflare on 12-13 November under the Apache 2.0 licence [3], is a browser engine that runs entirely on Cloudflare Workers: it parses HTML, applies CSS, executes page JavaScript and renders screenshots with no Chromium process or container behind it. Because it speaks the Chrome DevTools Protocol, Puppeteer or an agent harness connects to it the same way it connects to Chrome, and DevTools can open on exactly the page the agent is processing. For an owner, the practical meaning is simpler than the architecture: you can watch what the agent watched, while it works, instead of reconstructing it afterwards from a screenshot it happened to save.
What These Tools Cannot Do
Both tools have named, documented limits, and Google's and Cloudflare's own material is unusually blunt about them [1] [3]. AQuA samples randomly, which means a defect that breaks 1% of a critical workflow can pass every structural check and never enter the sample; verification is capped at 50 clusters per run with 3 transcripts each, to keep cost predictable; and its replay feature re-sends recorded conversations but cannot reconstruct external state, so a session that depended on what a database held that morning will not reproduce. It diagnoses and proposes; it never edits code or opens a pull request on its own. Kitesurf's CDP endpoint, in Cloudflare's own words [3], is "not a full Chrome implementation", some DevTools features do not work yet, running it on Cloudflare requires a Workers Paid plan, and it ships with no authentication of its own: deploy it unprotected and anyone with the URL can browse the web on your account. An inspectable agent browser is not an authorised agent browser; someone still has to decide what the agent may touch.
What This Changes for a Business Running Agents
The operating lesson is that "the agent is running" was never the same statement as "the agent is working", and the gap between them is now measurable. In our own daily operation of scheduled production agents, the failures that cost the most have never been loud ones; they are the runs that complete on time and quietly drift, which is why the review step in front of anything published exists at all. Tools like AQuA formalise that instinct: a defect queue with citations to specific lines, each finding verified against transcripts before it reaches a human, turns agent supervision from spot-checking into a standing process. If you are running agents in a business you have bought or are preparing to sell, the diagnostic loop is becoming part of the checklist a careful reviewer expects to see, alongside the sandbox, the credentials and the approval points.
Sources
- [1] Google Developers Blog, The Outer Loop, Insights First: An Ambient Quality Agent That Diagnoses Your Production Agent (8 October 2026), InfoQ, Google Open Sources Ambient Quality Agent (AQuA) to Diagnose Production AI Agent Failures (13 November 2026), Cloudflare Blog, Kitesurf is now open source (13 November 2026),