Monthly Archives: February 2025

AI Agent Security: A Framework for Accountability and Control

1 Reply

This weekend, I came across a LinkedIn article by Priscilla Russo about OpenAI agents and digital wallets that touched on something I’ve been thinking about – liability and AI agents and how they change system designs. As autonomous AI systems become more prevalent, we face a critical challenge: how do we secure systems that actively optimize for success in ways that can break traditional security models? The article’s discussion of Knight Capital’s $440M trading glitch perfectly illustrates what’s at stake. When automated systems make catastrophic decisions, there’s no undo button – and with AI agents, the potential for unintended consequences scales dramatically with their capability to find novel paths to their objectives.

What we’re seeing isn’t just new—it’s a fundamental shift in how organizations approach security. Traditional software might accidentally misuse resources or escalate privileges, but AI agents actively seek out new ways to achieve their goals, often in ways developers never anticipated. This isn’t just about preventing external attacks; it’s about containing AI itself—ensuring it can’t accumulate unintended capabilities, bypass safeguards, or operate beyond its intended scope. Without containment, AI-driven optimization doesn’t just break security models—it reshapes them in ways that make traditional defenses obsolete.

“First, in 2024, O1 broke out of its container by exploiting a vuln. Then, in 2025, it hacked a chess game to win. Relying on AI alignment for security is like abstinence-only sex ed—you think it’s working, right up until it isn’t,” said the former 19-year-old father.

The Accountability Gap

Most security discussions around AI focus on protecting models from adversarial attacks or preventing prompt injection. These are important challenges, but they don’t get to the core problem of accountability. As Russo suggests, AI developers are inevitably going to be held responsible for the actions of their agents, just as financial firms, car manufacturers, and payment processors have been held accountable for unintended consequences in their respective industries.

The parallel to Knight Capital is particularly telling. When their software malfunction led to catastrophic trades, there was no ambiguity about liability. That same principle will apply to AI-driven decision-making – whether in finance, healthcare, or legal automation. If an AI agent executes an action, who bears responsibility? The user? The AI developer? The organization that allowed the AI to interact with its systems? These aren’t hypothetical questions anymore – regulators, courts, and companies need clear answers sooner rather than later.

Building Secure AI Architecture

Fail to plan, and you plan to fail. When legal liability is assigned, the difference between a company that anticipated risks, built mitigations, implemented controls, and ensured auditability and one that did not will likely be significant. Organizations that ignore these challenges will find themselves scrambling after a crisis, while those that proactively integrate identity controls, permissioning models, and AI-specific security frameworks will be in a far better position to defend their decisions.

While security vulnerabilities are a major concern, they are just one part of a broader set of AI risks. AI systems can introduce alignment challenges, emergent behaviors, and deployment risks that reshape system design. But at the core of these challenges is the need for robust identity models, dynamic security controls, and real-time monitoring to prevent AI from optimizing in ways that bypass traditional safeguards.

Containment and isolation are just as critical as resilience. It’s one thing to make an AI model more robust – it’s another to ensure that if it misbehaves, it doesn’t take down everything around it. A properly designed system should ensure that an AI agent can’t escalate its access, operate outside of predefined scopes, or create secondary effects that developers never intended. AI isn’t just another software component – it’s an active participant in decision-making processes, and that means limiting what it can influence, what it can modify, and how far its reach extends.

I’m seeing organizations take radically different approaches to this challenge. As Russo points out in her analysis, some organizations like Uber and Instacart are partnering directly with AI providers, integrating AI-driven interactions into their platforms. Others are taking a defensive stance, implementing stricter authentication and liveness tests to block AI agents outright. The most forward-thinking organizations are charting a middle path: treating AI agents as distinct entities with their own credentials and explicitly managed access. They recognize that pretending AI agents don’t exist or trying to force them into traditional security models is a recipe for disaster.

Identity and Authentication for AI Agents

One of the most immediate problems I’m grappling with is how AI agents authenticate and operate in online environments. Most AI agents today rely on borrowed user credentials, screen scraping, and brittle authentication models that were never meant to support autonomous systems. Worse, when organizations try to solve this through traditional secret sharing or credential delegation, they end up spraying secrets across their infrastructure – creating exactly the kind of standing permissions and expanded attack surface we need to avoid. This might work in the short term, but it’s completely unsustainable.

The future needs to look more like SPIFFE for AI agents – where each agent has its own verifiable identity, scoped permissions, and limited access that can be revoked or monitored. But identity alone isn’t enough. Having spent years building secure systems, I’ve learned that identity must be coupled with attenuated permissions, just-in-time authorization, and zero-standing privileges. The challenge is enabling delegation without compromising containment – we need AI agents to be able to delegate specific, limited capabilities to other agents without sharing their full credentials or creating long-lived access tokens that could be compromised.

Systems like Biscuits and Macaroons show us how this could work: they allow for fine-grained scoping and automatic expiration of permissions in a way that aligns perfectly with how AI agents operate. Instead of sharing secrets, agents can create capability tokens that are cryptographically bound to specific actions, contexts, and time windows. This would mean an agent can delegate exactly what’s needed for a specific task without expanding the blast radius if something goes wrong.

Agent Interactions and Chain of Responsibility

What keeps me up at night isn’t just individual AI agents – it’s the interaction between them. When a single AI agent calls another to complete a task, and that agent calls yet another, you end up with a chain of decision-making where no one knows who (or what) actually made the call. Without full pipeline auditing and attenuated permissions, this becomes a black-box decision-making system with no clear accountability or verifiablity. That’s a major liability problem – one that organizations will have to solve before AI-driven processes become deeply embedded in financial services, healthcare, and other regulated industries.

This is particularly critical as AI systems begin to interact with each other autonomously. Each step in an AI agent’s decision-making chain must be traced and logged, with clear accountability at each transition point. We’re not just building technical systems—we’re building forensic evidence chains that will need to stand up in court.

Runtime Security and Adaptive Controls

Traditional role-based access control models fundamentally break down with AI systems because they assume permissions can be neatly assigned based on predefined roles. But AI doesn’t work that way. Through reinforcement learning, AI agents optimize for success rather than security, finding novel ways to achieve their goals – sometimes exploiting system flaws in ways developers never anticipated. We have already seen cases where AI models learned to game reward systems in completely unexpected ways.

This requires a fundamental shift in our security architecture. We need adaptive access controls that respond to behavior patterns, runtime security monitoring for unexpected decisions, and real-time intervention capabilities. Most importantly, we need continuous behavioral analysis and anomaly detection that can identify when an AI system is making decisions that fall outside its intended patterns. The monitoring systems themselves must evolve as AI agents find new ways to achieve their objectives.

Compliance by Design

Drawing from my years building CAs, I’ve learned that continual compliance can’t just be a procedural afterthought – it has to be designed into the system itself. The most effective compliance models don’t just meet regulatory requirements at deployment; they generate the artifacts needed to prove compliance as natural byproducts of how they function.

The ephemeral nature of AI agents actually presents an opportunity here. Their transient access patterns align perfectly with modern encryption strategies – access should be temporary, data should always be encrypted, and only authorized agents should be able to decrypt specific information for specific tasks. AI’s ephemeral nature actually lends itself well to modern encryption strategies – access should be transient, data should be encrypted at rest and in motion, and only the AI agent authorized for a specific action should be able to decrypt it.

The Path Forward

If we don’t rethink these systems now, we’ll end up in a situation where AI-driven decision-making operates in a gray area where no one is quite sure who’s responsible for what. And if history tells us anything, regulators, courts, and companies will eventually demand a clear chain of responsibility – likely after a catastrophic incident forces the issue.

The solution isn’t just about securing AI – it’s about building an ecosystem where AI roles are well-defined and constrained, where actions are traceable and attributable, and where liability is clear and manageable. Security controls must be adaptive and dynamic, while compliance remains continuous and verifiable.

Organizations that ignore these challenges will find themselves scrambling after a crisis. Those that proactively integrate identity controls, permissioning models, and AI-specific security frameworks will be far better positioned to defend their decisions and maintain control over their AI systems. The future of AI security lies not in building impenetrable walls, but in creating transparent, accountable systems that can adapt to the unique challenges posed by autonomous agents.

This post lays out the challenges, but securing AI systems requires a structured, scalable approach. In Containing the Optimizer: A Practical Framework for Securing AI Agent Systems I outline a five-pillar framework that integrates containment, identity, adaptive monitoring, and real-time compliance to mitigate these risks.

How Washington State is Preparing to Undermine Parents and the Constitution

Leave a reply

I am not a lawyer, but I love the law. I love the law because it increases the chances of predictable outcomes, aiming to provide a stable framework that protects our rights and creates a level playing field for all. The law is not just a collection of rules – it is a security system for our rights, designed to prevent future harm. Constitutional lawyers, judges, and legislators study system vulnerabilities, analyze potential threats, and design legal frameworks that protect against systemic failures.

Just as a well-built security system relies on layers of protection, our legal system depends on precedent – the accumulated wisdom of past rulings that form a firewall between government power and individual rights. Precedent is meant to stop governments from repeating past mistakes, stripping away hard-won rights, or changing the rules for political convenience. But that protection only works if lawmakers and courts respect it – and Washington’s leaders now appear ready to test its limits.

The People Took a Stand – And the Government is Responding

As both a parent and someone who has studied these issues carefully, I’m particularly troubled by House Bill 1296. While its supporters claim it protects children, the bill actually undermines the very protections that thousands of parents like me fought to secure through I-2081. These changes could significantly affect parental notification requirements, access to records, and decision-making authority that I-2081 was designed to protect.

The Supreme Court’s recognition of parental rights as fundamental reflects a crucial reality: parents, not government agencies, are uniquely positioned to make decisions about their children’s upbringing. When a child needs medical care or educational support, it’s parents who know their medical history, understand their learning style, and can best advocate for their interests. While the state has a role in preventing abuse and neglect, its power to override routine parental decisions demands extraordinary justification – a high bar that exists because parents possess irreplaceable knowledge about their children’s needs and circumstances.

While the state has legitimate interests in protecting children’s welfare, this grassroots movement led to Initiative 2081 (I-2081), the Parents’ Bill of Rights – a measure designed to restore transparency and ensure appropriate parental involvement. I-2081 guarantees that parents have access to their child’s school and medical records, requires schools to notify parents before providing medical services, and allows parents to opt their child out of instruction that conflicts with their values. Driven by broad-based support and careful consideration of both parental rights and child welfare, the initiative was expected to pass overwhelmingly.

Under Washington law, once passed by voters, the legislature is barred from amending or repealing an initiative for two years. Additionally, a King County Superior Court granted summary judgment in favor of I-2081, finding its provisions legally sound after a careful review of the competing interests at stake.

However, lawmakers took an unexpected approach by passing Initiative 2081 themselves in March 2024, rather than letting voters decide. This created a path for them to modify the initiative sooner than if voters had enacted it directly. While the legislature debated various amendments, including changes to notification procedures, the core concern remained: this maneuver, though legal, potentially undermined the citizen initiative process that had brought the Parents’ Bill of Rights forward in the first place.

Constitutional Principles at Stake

The current situation presents a complex interplay of rights and responsibilities. While the state has a legitimate interest in protecting children, House Bill 1296 and related proposals risk undermining the very protections that parents fought to secure. These changes could significantly affect parental notification requirements, access to records, and decision-making authority that I-2081 was designed to protect.

The Supreme Court has consistently recognized that while the state has important responsibilities in protecting children’s welfare, parental rights are fundamental and deserve strong protection. State intervention, while sometimes necessary, must be justified by clear evidence and compelling circumstances. The challenge lies not in determining whether the state has any role – it clearly does – but in ensuring that new restrictions on parental rights meet the high constitutional standards required for such intervention.

The Initiative Process Under Pressure

Beyond the specific issue of parental rights, the integrity of Washington’s democratic processes is also at stake. Senate Bill 5283, introduced by Sen. Javier Valdez (D-Seattle), would create new requirements for signature gatherers. While voter integrity is important, these requirements could effectively kill grassroots participation in the initiative process, making voter-led bills like the Parental Bill of Rights nearly impossible in the future.

The Constitutional Framework

The United States Supreme Court has developed a careful framework for evaluating parental rights. In Wisconsin v. Yoder (1972), the Court established an important balancing test between state and parental interests, recognizing that while states have legitimate educational interests, parents’ fundamental rights in directing their children’s upbringing can outweigh state requirements when properly supported.

In Troxel v. Granville (2000), the Court affirmed these rights as fundamental; in Santosky v. Kramer (1982), it established the need for clear and convincing evidence before state intervention; and in Parham v. J.R. (1979), it outlined when state involvement might be justified. While these cases acknowledge both parental rights and state interests, they consistently require strong justification for overriding parental authority.

Laws affecting fundamental rights face the highest level of judicial review – strict scrutiny. Under this standard, the government must prove both a compelling interest and that its measures are narrowly tailored. While protecting children is certainly a compelling interest, the broad scope of the proposed changes suggests they may struggle to meet the “narrowly tailored” requirement. This doesn’t mean all regulation is impossible – but it does mean that restrictions must be carefully crafted and strongly justified.

Washington’s constitution provides additional safeguards for individual liberties and family rights. State courts have historically interpreted these protections robustly, while recognizing legitimate state interests in child welfare. This dual protection means that changes to parental rights must satisfy both federal and state constitutional requirements.

Defining Harm

While Washington lawmakers may seek to broaden the definition of harm to justify greater intervention, such changes must be precise and evidence-based. The state undeniably has a compelling interest in preventing child abuse and neglect, and courts have long upheld intervention in cases of severe medical neglect and physical abuse. However, House Bill 1296 goes beyond these extreme cases, potentially expanding state authority over routine parental decisions that have historically received strong constitutional protection. Supreme Court precedent does not prohibit all state action, but it does require substantial justification for overriding parental authority. Vague or speculative concerns are not enough to justify restrictions on fundamental rights.

Legal Challenges Ahead

If Washington proceeds with these changes, they will likely face significant constitutional scrutiny. The Fourteenth Amendment’s protection of parental rights, combined with federal laws like FERPA (while subject to certain exceptions), creates a strong framework for challenging overreach. While courts recognize the state’s role in protecting children, they typically require compelling evidence before allowing intervention in family decisions.

This isn’t merely about policy preferences – it’s about fundamental constitutional principles and the balance of power between families and government. While reasonable people can disagree about specific policies, the broader trend toward diminishing parental rights without compelling justification threatens core constitutional values. If Washington succeeds in implementing these changes, it could encourage similar efforts elsewhere, potentially eroding long-established protections for family autonomy.

Take Action Today

Make your voice heard! Washington has an official website where you can share your perspective on House Bill 1296 and Senate Bill 5283. These bills impact both parental rights and the future of citizen initiatives in our state. Review the bills and share your views with Washington’s legislators.