Almost all of the 100+ orgs we’ve helped had the same gap in their systems.
Most (good) teams are now using agents to monitor errors and address them swiftly, but they’re skipping one step of well-built loops that would reduce their errors going forward each time it happens. Things like traces, errors and deploy records should all have agents that study them, to learn what went wrong and stop future sessions from making the same mistakes that break in production.
If production data and review findings get written back into the agents’ instructions, the whole setup improves itself.
Our setup#
An agent should be able to start from an alert, find the release it came from, find the commit in that release, and propose a root cause.
You need three things for that:
- Errors tagged with the release that produced them, so you can tie a regression to a change.
- Source maps for frontend traces. Without them, the agents can just point at a large bundle.
- A deploy log, so agents can identify what changes shipped.
Agents can’t use what they can’t query#
Agents have to be able to call a tool and get the data back. Access to the product database, code and tickets is a good start, but when something breaks, agents are limited to digging through git history by timestamp and guessing root causes.
If you deploy with GitOps, your deploy commits are already a perfect release history. Give your agents access to it.
Turning mistakes into rules#
We update instructions on a schedule:
- A skill collects findings from our review bots and from local agent sessions.
- It sorts each one into fixed, waived, deferred or false positive.
- Anything fixed twice becomes a rule, a review rubric or a line in the main instruction file.
- Any recurrent false positives or waived findings become updates to our review bot configs, so they stop raising the same false alarms.
- Each change goes up as a PR and gets reviewed like code, with a log of what changed and why.
What you can build once this works#
With release tags in place and agents able to read them, you can automate the next layer. A check after every deploy that watches error rates for that release. Then fix verification, where the agent confirms that error X stopped after release Y and writes that back into the instructions.