When Should an Agent Defer? Confidence Scoring and Human Authorization for Remediation

Most discussion of autonomous remediation focuses on capability. Can the agent find the root cause, can it write the fix, can it verify it. Those are necessary and increasingly solved. The part that decides whether you can actually run remediation in production is governance: a defensible rule for when the agent acts on its own and when it defers to a person.

The stakes are not abstract. Roughly 41% of codebases are now AI-generated, and industry research suggests around 45% of AI coding tasks introduce a critical flaw or security issue. The cost of poor software quality sits near $2.41 trillion. The fixes themselves are increasingly machine-authored, which means the same defect distribution that infected your codebase now infects your remediation. An agent fixing AI-generated code with more AI, unsupervised, is not a control. It is a faster path to the same failure.

So the governing principle is fixed: agents propose, humans authorize. The interesting question is how to make "authorize" cheap and selective rather than a blanket human gate on every fix. The answer is a decision surface that routes each proposed fix by two independent variables: how confident the agent is that the fix is correct, and how much damage it does if the agent is wrong.

Teams that bolt a single confidence threshold onto an agent learn the hard way that confidence alone is a useless gate. A 95%-confident fix to a feature flag and a 95%-confident fix to your payments authorization path are not the same decision. Confidence tells you how likely the agent is right. Criticality tells you what it costs when it is wrong. You need both, and they are orthogonal.

Criticality is a property of the system, not the change. It comes from what the change touches, computed against a live dependency model rather than the diff. A System Graph that maps services, dependencies, and CI/CD into one change-aware model is what lets you ask the right question: does this fix touch a node that fans out to revenue paths, handles regulated data, or performs an irreversible operation? Criticality is high when the answer is yes, regardless of how small the patch looks. A three-line change to a shared auth library is high-criticality. A six-hundred-line change to an isolated internal tool is not.

Confidence is a property of the evidence behind the fix, not the model's self-reported certainty. This distinction is where most implementations go wrong, so it deserves its own section.

A language model will happily emit a confidence number. That number is close to worthless as a gate because it is uncalibrated: the model is often most confident exactly where it is wrong. If you wire an agent's self-rated confidence to auto-apply, you have built a system that fast-tracks its own blind spots.

Confidence that can gate a production change has to be assembled from evidence the agent did not author:

Confidence, in other words, is a computed score over real artifacts. When the evidence is thin, the score is low, and the matrix sends the fix to a human. That is the design working as intended, not a failure.

Cross the two axes and you get the decision surface. Keep it coarse. Three bands on each axis is enough, and more bands invite false precision.

| | Low criticality | Medium criticality | High criticality | |---|---|---|---| | High confidence | Auto-apply, logged | Auto-apply with rollback armed | Single approver | | Medium confidence | Auto-apply with rollback armed | Single approver | Escalate | | Low confidence | Single approver | Escalate | Escalate, never auto |

Three things make this matrix honest rather than decorative.

First, the high-criticality column never auto-applies, at any confidence. Payments, authentication and authorization, regulated data, and irreversible operations always cost a human authorization. Confidence can make that authorization a single fast approval instead of a change-advisory ritual, but it cannot remove it. This is the line that keeps the system governed.

Second, auto-apply is always paired with an armed rollback and a recorded verdict. Auto-apply does not mean fire-and-forget. It means the agent applies, verifies in production behavior, and holds a one-action reversal ready. The absence of a human in the path raises the bar on evidence and reversibility; it does not lower it.

Third, the agent never sets its own cell. Criticality is derived from the graph and policy. Confidence is computed from evidence. The agent cannot relabel a high-criticality change as medium to skip the gate, because it does not own either axis. This is the difference between governance and theater.

The matrix introduces its own ways to fail. Name them so your implementation defends against each.

Where remediation runs inside a customer boundary or a regulated enclave, the evidence requirement is stricter. Edge Runners execute as signed capsules inside the boundary and emit audit-ready evidence outward, so the matrix decision and its supporting artifacts survive a compliance review instead of living in a CI log someone can edit. All of this is where Governance lives as first-class configuration rather than tribal knowledge, and where Remediation Fleets execute against the policy rather than around it.

You do not need to deploy autonomous remediation to start building the decision surface. Build the matrix before you arm the agent.

Each step moves human attention off the fixes that never needed it and concentrates it on the genuinely dangerous few.