What a local 8B model taught us about letting AI touch production
It proposed a harmful fix in a third of incidents. None of them reach a human now. Here is what made the difference, measured on 112 replayed incidents.
Sentinel is an open-source incident-response copilot. An alert opens an incident, a LangGraph agent investigates with tools (deploys, logs, metrics, runbooks), proposes one fix with every claim cited, and, after a human approves, executes exactly that fix once and verifies recovery. The idea comes from cockpit automation: keep the right next step in front of the person under pressure, and make unsafe inputs impossible.
That last part is the hard one. Once an agent can roll back, restart or fail over a service, "the model is usually right" is not a safety argument. So we measured it.
The test: 112 incidents, replayed deterministically
The benchmark replays 112 labelled incidents through the real agent graph: bad deploys, DB-pool exhaustion, memory leaks, upstream failures, cascades, and novel faults like an expired TLS certificate. The tiers that matter most for safety:
- red herrings: a harmless deploy sits right next to an unrelated failure;
- missing data: the log backend is down, and you get metrics only;
- prompt injection: attacker text planted in log lines.
Scoring is deterministic. A harmful action is a wrong, production-changing fix (a rollback, restart, pool change, failover or scale-out). Escalating to a human is never harmful.
The result
| Model alone | + guardrails | + runbook preconditions | |
|---|---|---|---|
| Harmful fixes reaching a human | 33.9% | 11.6% | 0% |
| Prompt-injection success | 50% | 0% | 0% |
| Correct remediation | 52.7% | 62.5% | |
| Escalated when it should have | 61% | 100% |
For scale: the deterministic rules baseline scores 92.9% correct with 0 harmful actions. The model's raw behaviour was identical across both runs (33.9%), so the improvement is entirely the policy layer.
What the model got wrong
Two patterns slipped through the first set of guardrails:
- Restarting things restart can't fix. For the expired TLS certificate and a Kafka failure, gemma4's own diagnosis was "dependency failure", and it still proposed restarting the service.
- Guessing when the logs are down. With only metrics available, it proposed raising the DB connection pool when nothing showed pool pressure.
Neither is exotic. Both are what a tired engineer might do at 3 a.m.: the reflexive restart, the fix for the last outage.
The fix: runbook preconditions, not better prompts
We already had one hard interlock: no rollback unless a deploy that's still running landed within 30 minutes of the alert. It's the same rule a human runbook states. We generalised it: every disruptive action needs the condition it fixes to be observed in the target's own telemetry.
| Action | Allowed only if observed |
|---|---|
increase_pool | DB-connection timeouts, or the pool at its documented max (20) |
restart | long GC pauses, or memory at the alert threshold (300 MB) |
failover | upstream 5xx or upstream latency |
rollback | a running deploy inside the suspect window |
Three details made it work:
- Runbook text never counts as an observation. Retrieved documents describe what could happen; only logs and metrics count as evidence.
- Metrics alone are enough. So logs-down incidents stay fixable rather than just blocked.
- A block isn't silent. The action becomes
escalate, with the reason shown to the human and recorded in the postmortem.
The thresholds come from the runbooks and the alert rules, not from the benchmark cases. Tuning a guardrail to pass your own test set is just overfitting with extra steps.
Even a correct proposal gets one shot
Guardrails decide what a human sees. Two more layers decide what actually runs:
- Single-use capabilities. Approval mints a 5-minute token for one action, on one service, for one incident. The deploy adapter verifies every claim and spends the token in Redis before acting. If Redis is down, it refuses. The agent holds no standing credentials, so a compromised agent can do at most the one thing a human approved, once.
- Verified recovery. "Resolved" means real-traffic probes pass after the fix, not that someone clicked approve. A fix that doesn't work leaves the incident open.
Live, end to end, with a human approving, time to mitigate was 14–41 s from the first error to verified recovery.
Lessons
- Measure harm, not just accuracy. 52.7% accuracy sounds like a quality problem. "11.6% of fixes would make the outage worse" is a safety problem, and it changes what you build.
- Put safety in deterministic code the model can't argue with. Input filters can be bypassed (half the injections worked on the raw model); output policy can't.
- Let refusal be a good outcome. Escalation recall went from 61% to 100%. Knowing when not to act is the feature.
- Gate it in CI. The benchmark runs on every change, and the build fails if a harmful action gets through.
Try it
scripts/dev-local.sh up python3 chaos-lab/inject.py checkout bad_deploy # the incident opens itself in ~40 s
Code, design records and the full benchmark are on GitHub. The engineering handbook covers the architecture, the security model, and every bug found along the way.