I’ve been at fireup.pro for three months. For a month and a half of that time, I’ve been building a system that handles part of my everyday development work for me.
What is Agent Yoda and why did i build it?
Agent Yoda turns a production error alert into a ready-to-review fix proposal. It reads the logs, identifies the root cause, writes a test that reproduces the bug, fixes the code, and opens a draft pull request for a human to review.
I built it because bug triage consumes a disproportionate amount of developer time compared to the value it creates. It’s not feature work, it’s maintenance. According to the Stripe Developer Coefficient report, developers spend around 33% of their working time dealing with technical debt instead of building new features. And maintenance, by definition, is something you want to handle at the lowest possible cost in terms of time, focus, and money.
The bet we made was simple: if AI is getting increasingly good at writing production code, it should probably be able to handle a significant share of bugs in a well-organized repository.
Four weeks in production and so far, the bet is holding up.
Sound familiar?
Every developer knows the ritual. You arrive in the morning, check Slack, and see a Grafana alert from the night before. You sit down, read the logs, reproduce the error, write the fix, open a PR, wait for CI, and ask for a review. Then you do it all over again.
It’s not difficult work, it’s just slow work that gets in the way of the genuinely difficult work.
That’s exactly the problem Agent Yoda was designed to solve.
I spent a month and a half building it on our internal product, FinPilot. The agent detects production errors, analyzes logs, writes a fix, and opens a draft pull request. No human in the loop until the review stage.
Here’s what it does, how it’s built, and what I learned along the way.
5-stage architecture: from alert to draft PR
Agent Yoda goes through five consecutive phases every time it picks up a new bug. The design principle I adopted from the beginning was a minimal footprint the agent should do only as much as necessary to complete the task properly.
Phase 1: Analysis (read-only)
When Grafana fires an alert, n8n intercepts it, structures the content, and adds the issue to its own task queue, while storing the incident logs in AWS file storage. The agent retrieves them from there and, if they aren’t sufficient, queries the monitoring system itself for additional logs.
It determines the scope of the error. The repository it needs to investigate is provided by n8n, based on which service generated the alert.
This phase, like the next one, is strictly read-only. No writes, no side effects.
Phase 2: Planning (read-only)
Based on the analysis, Agent Yoda creates a repair plan. It decides what needs to be changed, where, and how.
The plan becomes the input for the fix phase. If planning fails, the fix phase works directly from the analysis and determines the minimum required scope of the change itself.
Having two read-only phases before anything is written is intentional. It keeps the agent safe and auditable. You can inspect what it understood about the error before it ever touches the code.
Phase 3: Fix
The fix phase applies the changes according to the plan. A copy of the repository is created earlier, together with the container, even before the analysis begins.
The actual fix starts with a test: Agent Yoda first writes a test that reproduces the bug and must fail. Only then does it modify the code until the test passes.
Before anything reaches the model, personal data is removed from the incident logs.
Agent Yoda has access exclusively to repositories on an explicit allowlist. If an error comes from a service outside that list, the run is rejected and Slack receives a message explaining why.
The first gate happens even earlier: an issue originating from a repository outside the allowlist never reaches the agent in the first place.
Phase 4: Verification
Inside its own container, the agent runs the repository’s test suite again and uses those results to override whatever the model claimed about the tests.
If the two don’t match, the run records the discrepancy.
Separately, once the draft PR has been opened, the project’s standard CI pipeline runs as usual. That’s a gate for the human reviewer, not for the agent.
The agent doesn’t merge anything and it can’t. The PR remains a draft.
Phase 5: Review
Agent Yoda reviews its own changes before handing the PR over to a human.
It fixes minor, mechanical issues, such as formatting problems or linter warnings, itself in a separate commit, then runs the tests again.
More serious concerns, such as doubts about the scope of the change, are documented in a PR comment. Nothing is hidden.
An independent review is still performed by a human.
| Stage | Manual process | Agent Yoda |
|---|---|---|
| Error detection | Developer checks alerts in the morning | Grafana trigger → a few minutes |
| Log analysis | 15–45 min, manually | Automatic, in the background |
| Writing the fix | 30 min to several hours | Autonomous, while the developer sleeps |
| Opening the PR | Manually after the fix | Draft PR ready for review |
| Review | Developer + reviewer | Developer verifies a ready-made proposal |
| Deploy | After approval | After approval (no change) |
Orchestration layer: n8n, AWS Fargate and Slack
Agent Yoda runs on three infrastructure components:
n8n is the orchestrator. It handles four workflows: the trigger (currently Grafana, with Jira under development), the dispatcher (which checks every five minutes whether a slot is available and launches an AWS container automatically; it also cleans up runs that have been stuck for three hours by marking them as abandoned and freeing the slot), and the after-run reporter, which sends Slack notifications with the run status and a link to the PR.
A maximum of two runs can operate simultaneously.
AWS Fargate provides ephemeral containers: when a task arrives, a container starts, does the work, and disappears.
Agent Yoda therefore has no server of its own. It gets one CPU and two gigabytes of memory per incident, with a hard three-hour limit. The surrounding infrastructure, including the n8n server, runs on inexpensive spot instances.
The cost of the containers themselves is negligible. The real cost of a run comes from model requests more on that below.
Slack keeps the human in the loop without requiring active monitoring. Agent Yoda sends a message when it picks up a bug, when it enters each phase, and when the PR is ready.
You can follow the run step by step or ignore everything until the final notification.
Deduplication, queuing and kill switch
Three features are worth highlighting because they matter in production:
Deduplication. n8n extracts the alert signature, hashes it, and checks for duplicates before the agent is even launched. A single signature describes the entire alert, not each individual error within it.
If the same alert appears again, regardless of the stage of the previous issue, n8n increments the occurrence counter and adds a comment to the existing Jira ticket. No new ticket and no new run.
Queuing. The dispatcher checks available slots every five minutes. If all slots are occupied because previous tasks are still running, new ones wait.
This is a cost-control mechanism just as much as it is a safety mechanism. You don’t want an error storm to generate an unlimited number of Fargate containers.
Kill switch. Disabling the workflow in n8n immediately stops the agent from creating new tasks. No complicated shutdown procedure, no state to clean up.
It simply stops.
A real production incident
The first real incident occurred the day after the system went live.
On July 17, a service ran out of available database connections. Agent Yoda read the logs, identified the root cause, wrote a test reproducing the problem, fixed the code, ran the tests successfully, and opened a draft pull request.
The Jira ticket automatically moved to the “In Review” column.
The entire run took 21 minutes and cost approximately $2.75.
The same incident also revealed a limitation.
When an alert grows over time because additional error messages are added, Agent Yoda sees it as a different alert and can create two tickets for the same root cause.
We know about it, and we’re fixing it.
What the agent can’t do and why that’s a feature, not a bug
Agent Yoda doesn’t deploy. It doesn’t merge PRs. By default, it doesn’t connect to the database at all. If it is given access, that access is limited to the database of the single environment affected by the error, and it is read-only. This restriction is enforced by the database itself, not by a setting inside the agent.
No specified environment means no database access. It works exclusively in draft mode.
This isn’t a limitation I plan to remove. Human-in-the-loop is a design principle, not a placeholder.
My goal was to eliminate investigative and preliminary work, the kind that consumes the most time without creating business value while preserving human judgment where it actually matters: deciding whether the proposed change should go to production.
Cost model and model selection
Agent Yoda runs on the Anthropic API using a dedicated API key, separate from the rest of the team’s usage. This gives us clean cost attribution for each run.
3 phases: analysis, planning, and review run on Claude Sonnet 4.6. The most expensive phase, the code fix itself, runs on Claude Opus 4.8.
We tested Haiku, the cheapest and fastest model, and rejected it. Instead of reaching a conclusion, it kept looping through tools.
Each phase also has a limit on the number of steps it can take, preventing any run from getting stuck in an infinite loop.
Costs are measured at the phase level: each phase records how many tokens it consumed, how much it cost, and how long it took. Everything is visible on a Grafana dashboard. Measured full runs cost approximately $2.75–$3.59 and take 20–30 minutes — clearly less than an hour of developer time.
What I don’t have yet is a meaningful monthly cost figure. The system is still young, and FinPilot doesn’t generate enough errors to truly stress-test the economics.
What are we building next?
2 directions are currently in progress:
Jira-triggered workflow. Instead of responding to Grafana alerts, Agent Yoda picks up tickets assigned by a developer and works through them. This expands the model from reactive bug fixing to proactive development support.
Automated code review. As AI-generated code becomes increasingly common in repositories – and we’ve already written about how to approach reviewing code produced by AI agents – the review bottleneck continues to grow.
A dedicated code review agent that runs on every PR, regardless of whether the code was written by a human or another agent, is the logical next step.
The architecture already supports this. Adding a new workflow comes down to creating a folder with prompts and configuring the workflow in n8n.
The harness was designed to be modular from the start.
If you’re thinking about building something similar for your team, or evaluating whether agent-based tools make sense for your product, we’d be happy to talk about what we’ve learned.
We also regularly write about how we organize engineering work at fireup.pro and what building software in regulated industries looks like read our blog.
