Monthly Archives: July 2026

The Status Quo Outlived Its Status

Leave a reply

In security we like to say that the problems live in the gaps between systems. Each system, on its own, is usually coherent. It has a threat model, invariants, and someone who owns it. The seam between two systems is owned by nobody, and each side quietly assumes the other is handling the thing that neither is. The load balancer assumes the backend validates. The parser assumes the canonicalizer normalized. The audit covers the software but not the network it runs on. Attackers don’t have to beat either component. They just have to find the assumption neither side wrote down. The attacker gets to pick the threat model, and they pick the one that lives in the seam.

Once you see this pattern, you see it everywhere, and not just in security.

Quality Lives in the Gaps of Ownership

In software quality, the same topology produces a different failure class. Security gaps produce exploits. Ownership gaps produce jank.

A user’s journey through a product is inherently horizontal. They sign up, configure, use, get billed, get help. But ownership is vertical, carved along team boundaries. So the experience degrades precisely at the handoffs. The onboarding flow owned by one team dumps you into a product owned by another, with terminology that doesn’t match, settings that don’t carry over, and an error message that references a concept from a third team’s domain model. Every screen passed its own review. The journey never got one, because the journey has no owner.

This is the same root cause as the security version. Contracts between components are written in terms of what each side provides, not what the whole must feel like or withstand. Functionality composes. Quality attributes don’t. Not security, not usability, not performance, not consistency. Those are emergent properties of the composition, and emergent properties are exactly what per-team accountability structures can’t see. A dashboard that takes eight seconds to load is usually five services each meeting their SLO.

You Ship the Org Chart in N Dimensions

Conway’s law is usually quoted as a statement about architecture, that organizations design systems which mirror their communication structures. But architecture is just the most legible projection. The org chart also manifests in the security posture, where trust boundaries land wherever the reporting lines do. It shows up in the latency profile, where every org boundary becomes a network hop plus a queue plus a retry policy. It shows up in the data model, where the same “customer” is defined four ways because four VPs own four systems. It shows up in the compliance scope, where audits map to cost centers rather than to where the risk actually lives. It even shows up in the documentation, where each team documents its interior and nobody documents the crossings.

You don’t ship your org chart once. You ship it in every dimension at the same time.

And it’s stickier than the chart on the wall, because the formal org chart is only the visible part. The real partitioning is cultural. It is who trusts whom, which teams have history, where the scar tissue from the last reorg sits, and who won the last budget fight. Systems calcify around those boundaries. That’s why reorgs so rarely fix seam problems. You can redraw the chart in a day, but the shipped artifact embodies the org chart as it existed at every point in the system’s history. Legacy code is really legacy org structure. You’re maintaining the fossil record of decade-old turf wars, and the team that could explain a given seam disbanded three reorgs ago.

Bureaucracy Is the Fixative

Here’s where it hardens. Process is how organizations serialize distrust between units. Every approval gate, every ticket queue, every review board is a treaty boundary between fiefdoms, and treaties optimize for non-aggression, not for the emergent properties of the whole.

The bureaucratic instinct when a seam fails is to add process at the seam. A checklist, a sign-off, a form. This papers over the gap without giving it an owner, and now the seam has a compliance artifact defending its existence.

Which brings us to the uncomfortable part. Bureaucracy defends the status quo long past the point where the status quo lost its status. Not out of malice, and usually not even out of preference. The mechanism is simpler and more forgivable than that. Process is memory without comprehension.

Every rule is a compressed lesson. Some incident happened, someone got burned, a control was born. But the compression is lossy. The rule survives while the context doesn’t. The organization keeps executing the answer long after everyone who understood the question is gone. It’s Chesterton’s fence, except nobody can find the fence’s author, the field it enclosed is now a parking lot, and there’s a Fence Compliance team whose headcount depends on the fence.

That’s the inversion point. Controls that began as instruments become constituencies. A process accretes staff, tooling, budget, an annual review cycle. It stops being a means and becomes a stakeholder. And stakeholders defend themselves.

Asymmetric Bookkeeping

The genius of bureaucratic self-defense is that it never has to argue the status quo is good. It only has to make change expensive.

Every proposal to remove a control gets evaluated by asking what risk removal creates. Nobody asks what risk retention creates. The cost of the existing process is denominated in currencies the review process can’t count, things like velocity, morale, and opportunities that quietly went elsewhere, while the cost of change is denominated in the one currency it’s built to count. With bookkeeping that asymmetric, the ratchet only turns one way.

There’s a reliable tell for when status is lost but the defense continues. The justifications go circular. Ask why we do this and the answer stops referencing a threat or an outcome and starts referencing the process itself. It’s required for the audit. It’s policy. That’s the template. When a control’s referent is another control, you are no longer managing risk. You are maintaining a liturgy. And liturgies are stable. That’s what they’re for.

The people defending the liturgy usually aren’t cynics, either. Institutions promote the people who thrived under the current rules, which means the people with the authority to change the system are precisely the ones whose careers validate it. They don’t defend the status quo because they’ve weighed it and found it good. They defend it because it’s the ladder they climbed, and it’s genuinely hard to see your own ladder as arbitrary. The system doesn’t need guards. It manufactures believers. That’s why correction so often comes from outside, from a competitor, a collapse, or a technology that routes around the institution entirely, rather than from reform. Reform requires the institution to metabolize the idea that its own selection function is the problem, which is roughly asking the liturgy to audit itself.

Deletion Has No Constituency

So what do you do about it? Two things, and both are harder than they sound.

First, admit that the seams are the system, and staff them accordingly. You can’t fix an n-dimensional Conway problem with a one-dimensional intervention. A design system fixes the UX projection. A service mesh fixes the network projection. A GRC tool fixes the audit projection. But the generator is the accountability topology itself, so the pathology just re-expresses in whatever dimension you didn’t treat. The only durable moves are changing the human topology, which is rare, painful, and temporary, or forcing the interfaces to be explicit, adversarially specified, and owned as products. That’s the real lesson of the Amazon API mandate. It wasn’t about services. It was about giving the gaps owners.

Second, build a decay function. Controls get created by incidents, which are vivid and have advocates. Control removal has no incident, no advocate, no ceremony. The beneficiary of deletion is diffuse while the loser is a specific person in the room. So organizations accumulate process the way arteries accumulate plaque, one reasonable deposit at a time. Sunset clauses, zero-based process reviews, and deletion treated as a first-class ritual with the same ceremony as launch are ideas everyone nods at and almost nobody funds. The organizations that stay fast are the ones that treat removing a rule as an achievement, not an admission.

Because the status quo isn’t defended because it won an argument. It’s defended because it’s the null hypothesis, and the burden of proof only ever runs one direction. Every so often, you have to flip the burden and make the process re-justify itself in terms of an outcome rather than another process. If it can’t, it isn’t protecting you anymore.

It’s just protecting itself.

Why FIPS 140 Means Running Old Code

2 Replies

You need to use FIPS 140 because of compliance, but have you ever asked what that requirement is actually for? What security properties are the authors of these policies trying to achieve?

In high-assurance deployments, the practical goal is usually to establish a meaningful security boundary around cryptographic keys. Organizations want explicit controls over who and what can use a key, and they do not want the answer to be every application or administrator with access to the host. They are also worried about key theft and abuse. For important signing and decryption keys, keeping the key out of the hands of the application and host OS is often the simplest way to force reasonable key-protection practices.

These are real problems. When the threat group Storm-0558 acquired a highly sensitive Microsoft MSA signing key, operational failures allowed key material to escape the isolated signing environment and become accessible from a compromised engineering environment. That single extraction let the attackers forge tokens and compromise customer email accounts at scale. It is much harder to see that happening when a key is non-exportable and managed inside a hardware boundary. HSMs are not the only way to get these properties, but they are the one tool that forces you to think hard about how you operationalize a key, and that discipline has value.

The trade-off is that you end up running old software you cannot patch for upstream security vulnerabilities. In the best case, you are years behind.

Worse, this is usually non-memory-safe code that is entirely opaque to you. The firmware, the middleware, and the technical documentation are kept strictly behind lock and key by the vendor. You cannot inspect the code to see if it is vulnerable, and independent review is virtually impossible. The HSM vendor may have proactively patched those vulnerabilities, but more likely they have not.

The cryptography in these modules almost never breaks. What breaks is the plumbing. For example, when an HSM takes in a payload like an administrative command or an authentication token of some sort, it has to parse it. This may mean relying on complex parsers or business logic, often written in aging C code. Because that plumbing is not memory safe, a single malformed input can lead to a buffer overflow or remote code execution right past the validated boundary, or even worse, a long-forgotten feature combined with new code could bypass policy controls around accessing the module altogether.

Take the U-Boot forks as another example. When you look at how these embedded systems are actually built, they lean heavily on bootloaders, embedded operating systems, and vendor firmware that sit outside the cryptographic boundary but inside your trust story. A vendor might fork U-Boot, validate their module, and then that code is essentially frozen.

Think about what boot verification actually involves. Something has to parse the firmware image, figure out which bytes are covered by the signature, compute the digest, and check the signature. The math for the signature might be FIPS validated, but the parsing, the offset arithmetic, and the handling of attacker-controlled structures is all just non-memory-safe C code.

When researchers find vulnerabilities in X.509 parsing or U-Boot image processing, the flaws are not in the cryptography. They are in the plumbing. And because vendor version strings in this opaque, locked-down firmware often have no meaningful relationship to upstream release numbers, that fork may be a decade old.

The real danger is that two curves are moving in opposite directions. The attack surface is being examined continuously. AI and automated variant analysis are getting exponentially better at finding previously unknown exploitable memory-safety bugs in old C code. Meanwhile, the validated base firmware remains essentially fixed.

We do not have to guess about this. I recently published a data project analyzing recent FIPS 140-3 validations. The data shows how frozen these systems actually are. Out of 415 validated modules we looked at, 324 of them, 78 percent, showed no recorded public update after their initial validation. The knowledge of security issues compounding every week is colliding with code that barely moves.

FIPS 140 compliance buys you necessary domain separation and forces good operational habits. But the way the market achieves that separation often leaves you depending on a static, aging, and opaque codebase. The certificate tells you that a particular version met the requirements when it was evaluated. It does not tell you that the same code remains secure years later. At some point, the certificate becomes evidence not only of what was validated, but of how long the underlying system has stood still.

The Certification Ends Where the Code Begins

Leave a reply

Disclosure: I am an advisor to Binarly.

I recently built the FIPS 140-3 Corpus, a dataset that pulls together the public record of FIPS validations. It combines CMVP certificate records, Security Policies, implementation details, operational environments, firmware versions, algorithm claims, and lifecycle data into something you can actually query and analyze rather than read one certificate at a time.

I built it because I have spent enough years around certification programs to know that the interesting information is rarely in any single document. It emerges when you look at the record as a system. Once you do, a pattern shows up that I think deserves more attention than it gets. The public evidence tells you a great deal about what was evaluated and almost nothing about whether the code that shipped actually behaves the way the evaluation assumed.

What the paper trail shows

When you read Security Policies in bulk, you start seeing the same dependencies over and over. Validated modules lean on bootloaders, embedded operating systems, update agents, and vendor firmware that sit outside the cryptographic boundary but inside the trust story. The certificate covers the module. The security property depends on everything around it.

U-Boot is a good example. Several modules in the corpus disclose it as part of their firmware or boot environment, and in some of those cases it participates directly in verifying firmware integrity before execution. Think about what that verification actually involves. Something has to parse the firmware image, figure out which bytes are covered by the signature, compute the digest, check the signature, and then decide what to run. RSA and SHA-256 handle two steps in that sequence. The parsing, the offset arithmetic, the decision about which fields are authenticated and which are attacker controlled, all of that is ordinary C code, and it is exactly where things tend to go wrong.

Binarly’s researchers recently published Unfit to Boot, which found previously unknown vulnerabilities in U-Boot’s FIT image processing and signature verification path. The flaws were not in the cryptography. They were in the handling of attacker controlled structures before and around the verification operation. The math was fine. The plumbing was not.

This is the oldest lesson in applied cryptography and we keep relearning it. The primitive is almost never the weakest link. The code that feeds the primitive is.

Where the evidence runs out

Here is where it gets uncomfortable. A Security Policy might identify its bootloader with a string like CNN35XX-UBOOT-4.03-03. That tells you a vendor U-Boot derivative is present. It tells you almost nothing else. Which upstream revision was it forked from? What did the vendor change? Which FIT features were compiled in? Were the fixes for known parsing flaws ever backported? Can externally supplied firmware even reach those code paths in this product?

None of that is answerable from the certification record. Vendor version strings in embedded firmware often have no meaningful relationship to upstream release numbers. The fork may be a decade old. The fixes may have been applied selectively, or renamed, or lost in a rebase nobody documented.

Conventional software composition tools do not close this gap either. They work by matching. Filenames, manifests, version strings, hashes, YARA rules, CVE mappings. That approach answers a useful question, namely whether a binary appears to contain a component already known to be vulnerable. Firmware defeats it routinely. Dependencies get statically linked into larger executables, symbols get stripped, and vendor forks drift far enough from upstream that the signatures stop matching anything.

And matching cannot help with flaws nobody has found yet. Before the Unfit to Boot research existed, there was no CVE to map, no affected version range, no signature to match. Someone had to go look at the implementation first. Credit to the Binarly team for doing that work, and disclosure noted, but the point stands independent of any vendor. Until somebody examines what actually shipped, every downstream tool, database, and compliance process is working from an empty record.

Why I care about this for HSMs and BMCs

The corpus is full of devices that sit in unusually trusted positions. HSMs hold the keys for certificate authorities, payment systems, and governments. BMCs sit beneath the host operating system with control over firmware updates, recovery, and remote administration. I have spent much of my career depending on the first category and being quietly worried about both.

These devices are exactly where the paper trail is weakest. They accumulate long lived vendor forks, inherited open source components, proprietary parsers, and hardware specific code written over many years, most of it statically linked and distributed only as compiled firmware. HSM firmware makes the visibility problem even worse. It is almost never publicly accessible, and on the rare occasion you do get an image, it is often encrypted or obfuscated, frequently with a key shared across the product line. Whatever that design accomplishes, it means customers and independent researchers see less of the code than a motivated attacker willing to recover the key. So we end up trusting these devices on the strength of certifications that, as the U-Boot example shows, stop well short of the code paths where real failures happen.

That does not make the certifications worthless. It makes them a starting point. A validation record that discloses a U-Boot derivative in the boot chain has handed you a concrete question to ask your vendor. What evidence supports the claim that your product is unaffected by this class of flaw? Has anyone analyzed the released binary, or is the answer derived from a spreadsheet of version strings? Which fields of an incoming update are actually authenticated before any code touches them? Vendors who can answer those questions with evidence are telling you something important. So are vendors who cannot.

Documentation, inference, evidence

The way I think about it, assurance comes in layers and each layer answers a different question. The certificate tells you what was evaluated and under what assumptions. The corpus connects those artifacts across the whole ecosystem and exposes the shared dependencies and recurring architectures the individual documents obscure. The final layer is evidence about what actually shipped, and it can come from several places. Vendors tracking their forks against upstream and documenting backports. Independent analysis of released binaries. Researchers publishing the failure modes of mechanisms everyone assumed were sound.

Most of the industry stops at the first layer. Procurement checks for the certificate, the checkbox gets ticked, and the boot chain full of forked bootloader code goes unexamined until someone publishes research like Unfit to Boot and everyone scrambles to figure out whether they are affected.

I built the corpus to make the second layer easier, so that the public record can generate the right questions instead of just decorating RFP responses. The questions are the point. A validation record that names a bootloader fork should end in a conversation with the vendor about what is in it, not in a filed PDF. Trust in labels got us the last twenty years of firmware security. Trust supported by evidence is going to have to get us the next twenty.

Steve Jobs, AI, and the Problem of Analysis Without Ownership

Leave a reply

There is an old Steve Jobs clip from a 1992 MIT Sloan talk that feels newly relevant in the age of AI. In the talk, available here as Steve Jobs MIT 1992 Lecture, Jobs is asked about consultants. His answer is not that consultants are unintelligent or useless. His criticism is more subtle. He says consultants often get to see a lot, analyze a lot, and recommend a lot, but they do not stay with the work long enough to own the consequences.

They do not spend years living with the product, the team, the tradeoffs, the mistakes, the customers, the budgets, the bugs, or the recovery. They may see the fruit, as Jobs put it, but they “never really taste it.”

That distinction matters.

There is a kind of knowledge that comes from observation, and there is a different kind of knowledge that comes from ownership. Observation can make you articulate. Ownership makes you careful. Observation helps you describe what should happen. Ownership teaches you what actually happens when a recommendation meets constraints, incentives, politics, timelines, systems, and human behavior.

There is also a kind of knowledge that only comes from time.

Some problems cannot be understood in a single sitting. You need to carry them around for a while. You read, step away, come back, notice what still bothers you, test a different framing, sleep on it, and then see the thing that was hiding in plain sight. That kind of soaking is not inefficiency. It is often how judgment forms.

That is the parallel to AI.

AI is making analysis abundant. It can read more than we can read, summarize faster than we can summarize, find patterns across larger datasets, generate plausible options, and produce recommendations that sound polished and confident. That is useful. But it is not the same as judgment.

Used poorly, AI becomes consulting at machine scale. It is fast, articulate, and superficially impressive, but disconnected from whether its recommendations actually survive contact with reality.

It can say what should be done without knowing what happened after someone tried to do it. It can identify risks without understanding which ones mattered. It can produce a roadmap without living through the missed dependency, the customer objection, the policy constraint, the budget cut, the migration failure, or the second-order effect six months later.

It can also make us confuse speed of response with depth of understanding. That may be the deeper risk. AI can collapse the slow work of thinking into the first plausible answer. It can make a problem feel resolved before we have really spent time with it. It can produce fluency before we have earned conviction.

That does not make AI useless. It makes the design problem clearer.

The lazy version of the AI story is the self-driving car analogy, namely once the machine becomes safer, faster, or more consistent, the human gets pushed out of the loop. There will be domains where that is true. But much of knowledge work is different. The goal is not only to execute a task correctly. The goal is to understand the problem well enough to make better decisions the next time.

Execution tools can displace. Reasoning tools should compound.

That is why the most interesting promise of AI is not simply that it becomes a better consultant or even a better operator. It is that AI can help humans become better operators.

Used well, AI becomes a way to think with the material. It helps us understand datasets that are too large to hold in our heads. It lets us explore problem spaces from more angles. It helps test assumptions, compare interpretations, surface edge cases, and ask better questions. It can show us patterns we would have missed, but the value is not just the pattern. The value is that, through the process of interrogation, we understand the problem more deeply ourselves.

AI should not shorten our attention so much as deepen what our attention can hold.

A good AI system should help us return to a problem with more context than we had the last time. It should preserve the questions we asked, the assumptions we tested, the contradictions we found, the evidence that mattered, and the places where our understanding changed. It should make it easier to spend real time with the problem, not merely produce an answer faster.

In that sense, the best use of AI is not instant certainty. It is structured patience.

It lets us soak in a problem more effectively. By that I mean it enables us to hold more evidence in view, revisiting prior interpretations, comparing today’s answer to yesterday’s uncertainty, and gradually turning analysis into understanding.

The goal should not be to outsource judgment to AI. The goal should be to use AI to improve the conditions under which judgment is formed.

A good AI system should not merely say, “Here is the answer.” It should help us see why the answer might be true, where it might be fragile, what evidence supports it, what alternatives exist, and what would change our mind. It should help us move from diagnosis to action, from action to feedback, and from feedback to learning.

That is the line between AI as consultant and AI as learning partner.

This is also where many AI products will disappoint. Dashboards full of findings, risks, summaries, and recommendations can look impressive while still leaving the actual burden on the human team. They create the appearance of progress without necessarily improving understanding. The human still has to decide what matters, translate the finding into action, make the change, verify the outcome, and remember the lesson later.

The point is not that findings and recommendations are useless; they are necessary. Findings are the beginning of the loop, not the end of it. Systems that stop there are not doing judgment automation. They are doing analysis transfer.

The more interesting systems will close the loop. They will connect analysis to execution, execution to verification, and verification to institutional memory. Not because humans should be removed from the process, but because humans should be able to reason from a better substrate.

This is where Jobs’ point lands today. The scarce thing is not access to analysis. AI will make analysis abundant. The scarce thing is accumulated judgment, something that only comes from acting, observing, correcting, and learning over time.

Observation gives you language. Ownership gives you consequence. Time gives you depth. Feedback gives you judgment.

Jobs’ critique of consulting was not just a warning about consultants. It was a warning about any tool, process, or person that gets rewarded for sounding right without having to live with whether they were right.

AI will be most valuable not when it becomes the smartest consultant in the room, but when it helps teams build judgment faster, seeing more, acting sooner, sitting with the problem longer, verifying outcomes, and remembering what reality taught them.

The future of AI in knowledge work should not be analysis without ownership. It should be ownership made smarter.