AI Agent Security Needs Accountability and Control

This weekend I came across a LinkedIn article by Priscilla Russo about OpenAI agents and digital wallets. It connected with a problem I have been thinking about for some time. Who is liable when an AI agent changes a system and causes harm?

As agents grow more capable, it is not enough to secure the model as though it were another application component. An agent searches for ways to complete a task. That behavior can expose assumptions that ordinary software leaves untouched. Russo’s discussion of Knight Capital’s $440M trading failure is a useful example. The software acted fast enough to turn a mistake into a crisis. An AI agent adds another problem because it can find paths to an objective that its designers did not expect.

Traditional software might accidentally misuse a resource or escalate a privilege. An AI agent can search for a new way to reach its goal, including ways its developers never considered. Security therefore has to contain the agent itself. It must not be able to accumulate capabilities, bypass safeguards, or operate beyond its intended scope. Without that containment, optimization can make existing defenses irrelevant.

The Accountability Gap

AI security discussions often start with adversarial attacks and prompt injection. Both matter, but neither resolves the question of accountability. Russo argues that developers will eventually be held responsible for what their agents do, much as financial firms, car manufacturers, and payment processors answer for the systems they put into the world.

Knight Capital shows what this looks like in practice. Its software malfunctioned, but responsibility did not move to the software. The same principle will apply when agents make decisions in finance, healthcare, or legal services. Responsibility could sit with the user, the developer, or the organization that gave the agent access. Regulators, courts, and companies will need an answer that can be supported with evidence.

Building Secure AI Architecture

Liability will make architecture choices visible. A company that anticipated a risk, constrained its agents, and retained an audit trail will be in a different position from one that cannot reconstruct what happened. Identity, permissions, and evidence should be part of the design before an incident forces the issue.

Vulnerabilities are only one part of the risk. Agents introduce alignment failures, unexpected behavior, and deployment choices that change the surrounding system. Robust identity, dynamic controls, and real-time monitoring can limit the paths an agent may take when it finds a way around an assumption.

Containment and isolation matter as much as model resilience. A robust model can still cause damage if the surrounding system gives it too much authority. The system should prevent an agent from escalating access, leaving its assigned scope, or creating effects far beyond the task. That means limiting what it can influence, what it can modify, and how far its reach extends.

I’m seeing organizations take very different approaches. Russo points to companies such as Uber and Instacart that are working directly with AI providers. Others use stricter authentication and liveness tests to block agents. A more durable approach treats an agent as a distinct entity with its own credentials and explicitly managed access. Agents already exist. Forcing them into authentication models built only for people hides the risk instead of managing it.

Identity and Authentication for AI Agents

One immediate problem is how agents authenticate online. Most rely on borrowed user credentials, screen scraping, and brittle mechanisms that were never designed for autonomous systems. Traditional secret sharing and credential delegation often spread secrets across the infrastructure. The result is more standing privilege and a larger attack surface.

SPIFFE offers a useful model. Each agent should have a verifiable identity, scoped permissions, and access that can be monitored and revoked. My experience building secure systems has also taught me that identity alone does little without attenuated permissions, just-in-time authorization, and zero standing privilege. An agent should be able to delegate a narrow capability without sharing its credentials or creating a long-lived token.

Systems such as Biscuits and Macaroons show how this can work. They support fine-grained scope and automatic expiration. An agent can issue a capability token that is cryptographically bound to a specific action, context, and time window. It can then delegate what a task needs without passing along broader authority.

Agent Interactions and Chain of Responsibility

What concerns me most is the interaction between agents. One agent calls another, which calls a third, and soon no one can say which agent made the consequential decision. Without end-to-end auditing and attenuated permissions, the result is a black box with no clear accountability or verifiability. Regulated industries will have to solve that problem before they can rely on agent-driven processes.

Every step in the decision chain should be traceable, including the authority used at each transition. That record is more than an operations log. It may become an evidence chain that has to stand up in court.

Runtime Security and Adaptive Controls

Traditional role-based access control assumes that permissions can be assigned from a stable job description. An agent trained to optimize for success does not share that assumption. It may find a path that satisfies the objective while violating the operator’s intent. We have already seen AI models learn to game reward systems in unexpected ways.

Static policy will not be enough. Access controls should respond to behavior, monitoring should identify unexpected decisions, and operators need a way to intervene in real time. The monitoring must also evolve as agents discover new ways to reach their objectives.

Compliance by Design

My years building certificate authorities taught me that continuous compliance has to be designed into the system. A good control does more than satisfy a requirement at deployment. It produces the evidence needed to show what happened throughout operation.

The short-lived nature of many agent tasks creates an opportunity. Access can be temporary, data can remain encrypted at rest and in motion, and decryption can be limited to the agent authorized for a particular action. The resulting records can show which identity accessed which data, under what authority, and for how long.

The Path Forward

If we do not rethink these systems now, agent-driven decisions will operate in a gray area where responsibility is hard to establish. Regulators, courts, customers, and companies will eventually demand a clear chain of responsibility. Waiting for a catastrophic incident is a poor way to discover what evidence is missing.

The operating model should give every agent a defined role, constrained authority, and an attributable record of its actions. Controls need to adapt at runtime, and compliance needs evidence that can be verified later.

Organizations that wait will be reconstructing their decisions during a crisis. Those that build identity, permissions, containment, and evidence into the system will have a better chance of maintaining control and defending the choices they made. The goal is not an impenetrable wall. It is a system whose authority is bounded, whose actions are visible, and whose failures can be contained.

This post describes the problem. Containing the Optimizer presents a five-part approach built around containment, identity, adaptive monitoring, and real-time compliance.