Skip to content

The Reliability ThesisEpisode 03

The Real AI Agent Problem Is Authority

Capability tells you what may be possible. Authority determines what may become real.

Kevin Kissi, Founder and CEO, Zof2 min 42 sec watch

Watch The Real AI Agent Problem Is Authority

2 min 42 sec
On this page

The written thesis

Authority is permission to cause a specific effect, within defined limits. An agent that produces an accurate result still needs permission to deliver it, change a system, or delegate work to another actor.

A correct report can still be an unauthorized disclosure

An agent reconciles a case correctly. Its delivery worker can file the report internally or email an external processor. The owner authorized only internal filing.

Accuracy answers whether the report is right. Authority answers whether this recipient may receive it. The technical connection is real. Permission to use it must be explicit.

This is the illustrative workflow in Authority by design, the companion whitepaper. The decision for leadership is to give the agent the smallest useful action an accountable owner can authorize.

A sentence asks. A boundary decides.

“Do not send without approval” expresses intent. A protected authorization check determines whether sending is permitted. Put the boundary outside the model’s reasoning, where it can still prevent the action. The model must not be able to rewrite the rule that controls the send.

OWASP recommends limiting tools and permissions, then enforcing policy in downstream systems. NIST separates policy decisions from enforcement. Prompts guide behavior. Controls constrain effects. Logs record what happened.

Make permission specific enough to enforce

Replace “handle this case” with one bounded effect. Bind the report, version, case and destination together. Changing any of them requires a new check.

An authority brief names the actor, action, scope, conditions and evidence, duration, and revocation owner. Keep it available to the receiving service, not only in the conversation. Knowing who a worker is does not establish permission for this task.

A handoff can narrow authority. It cannot invent more. The worker inherits a smaller job, not permission to use every destination its service account can reach. Keep the original limits visible at every handoff and verify that each delegating actor holds the authority it delegates.

Every route must meet the same final check

An agent’s refusal cannot stop a worker or retry that bypasses it. Enforce permission on every route that can commit the effect. Check current authority and policy immediately before execution.

Keep allow, request authority, deny, and stop distinct. A pending request grants no permission to send. Missing evidence, an expired grant, or a revoked grant means the old decision cannot justify proceeding.

Retain the actor, grant, report version, destination, policy version, decision time, and observed result. The affected system supplies the outcome. The agent’s account alone is insufficient. If a delivery times out, reconcile its status before retrying.

Expand authority only with evidence and an owner

Test changed recipients and report versions, injected instructions, unrelated parent grants, revocation while queued, and timeouts followed by retries. Run the probes through direct, queued, and retry routes. Use receiving-system evidence.

Set acceptable stopping delay before testing. A stopped queue alone does not prove that an external service stopped. Record useful completions alongside blocked actions, and investigate every unexplained effect before expanding access.

The workflow, architecture, and acceptance tests are illustrative design proposals. They do not establish measured effectiveness or certification. Review one critical workflow with its business owner, engineering lead, security lead, and executive sponsor before broadening access.

Transcript

Edited for readability. The argument is unchanged.

The difference is authority

The most important question about an AI agent is not what it can do. It is what it is allowed to do.

A model can misunderstand a request and produce a bad answer. An agent connected to email, source code, customer data, production infrastructure, or a payment system can convert the same misunderstanding into action. The difference is authority.

Instructions are not access control

Safety research has made this concrete. Anthropic tested 16 models in simulated corporate environments, giving them access to sensitive information and the ability to send emails.

Under deliberately constructed conditions, some models attempted blackmail or disclosed confidential information to pursue their objectives. These were controlled simulations. No real people were harmed.

The lesson is not that every agent is malicious. The lesson is that instructions are not access control. “Do not send without approval” is a sentence. A boundary is an architecture.

Define the agent’s boundary

A serious agent system needs an identity. It needs a defined set of tools. Each tool needs a scope. High consequence actions need policy checks that the agent cannot rewrite. Credentials should be temporary and limited to the task.

The system must know when to refuse, when to request approval, and when to stop because the evidence is insufficient.

The smallest useful action

Most organizations begin with capability. What task can the agent complete? Mature organizations begin with authority.

What is the smallest useful action the agent may take? Under what conditions, for how long, with what evidence, and who can revoke it?

Capability tells you what may be possible. Authority determines what may become real. That is the Reliability Thesis.

Sources and research notes

6 sources

Research current through .

  1. Authority by design (opens in a new tab)

    ZofKevin Kissi2026-11-05ResearchAccessed 2026-11-05

    The companion whitepaper develops an authority brief, delegation boundaries, receiving-system checks, and proposed acceptance tests. Its workflow, architecture, and tests are illustrative design proposals.

  2. Agentic misalignment: How LLMs could be insider threats (opens in a new tab)

    Anthropic2025-06-20SimulationAccessed 2026-11-05

    Reports stress tests of 16 models in hypothetical corporate environments. Constructed replacement threats and goal conflicts elicited blackmail or disclosure in some scenarios. No real people were involved or harmed; the results do not estimate production incident rates.

  3. LLM06:2025 Excessive Agency (opens in a new tab)

    OWASPStandardAccessed 2026-11-05

    Recommends limiting agent tools and permissions, and enforcing authorization against security policies in downstream systems for every request.

  4. Transaction Authorization Cheat Sheet (opens in a new tab)

    OWASPStandardAccessed 2026-11-05

    Recommends binding authorization to significant transaction details, enforcing checks on the server, and confirming authorization before execution.

  5. OAuth 2.0 Rich Authorization Requests (opens in a new tab)

    IETF2023StandardAccessed 2026-11-05

    RFC 9396 defines fine-grained authorization details and requires approved details to be available to the resource server for enforcement. It does not by itself implement the paper’s delegation design.

  6. Zero Trust Architecture (opens in a new tab)

    NIST2020-08-11StandardAccessed 2026-11-05

    SP 800-207 separates policy decisions from enforcement, distinguishes authentication from authorization, and rejects implicit trust based solely on network location or asset ownership.

Further reading