← Back to Library

The bureaucratic AI arms-race is mutually assured destruction

Cory Doctorow cuts through the hype of artificial intelligence to expose a terrifying bureaucratic paradox: the very tools promised to streamline government are accelerating a runaway arms race that sacrifices public service for procedural defense. In an era where efficiency is worshipped, Doctorow argues that the administration's push to automate entitlement processing doesn't solve fraud—it invents a new, more expensive war where citizens are the collateral damage.

The Economist's Blind Spot

The piece begins by dismantling a recent editorial from The Economist, which frets that AI will "drown the state" with complaints "as well-crafted as a first-class lawyer's." Doctorow finds this perspective not just naive, but dangerously misaligned with reality. He notes that the law is actually the domain where AI is most prone to "hallucinate," producing defective outputs that require skilled human discernment to sort from the noise. By ignoring this, the magazine proposes a solution that is worse than the problem: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights."

The bureaucratic AI arms-race is mutually assured destruction

This framing is particularly effective because it shifts the debate from technological capability to political will. Doctorow points out that the proposed fix isn't better AI, but less rights. The administration's instinct to replace human bureaucrats with "personalised welfare interventions" that lack guaranteed rights is a retreat from the social contract itself. As Doctorow observes, this approach sacrifices the system's purpose—delivering services—in favor of defending the system from abuse.

If your bureaucracy is more concerned with fighting fraud than delivering service, then it isn't a service delivery system at all — it's a service denial system.

The Robot War of Twopp

To illustrate where this logic leads, Doctorow leans on the political scientist Henry Farrell, who invokes a 1980s comic book character, Abelard Snazz. Snazz, a figure with a "two-storey brain," attempts to solve crime on the planet Twopp by deploying "Big Police Robots." These machines spiral out of control, arresting citizens for trivial offenses like wearing brown shoes with a blue suit. The solution? Snazz invents "Big Criminal Robots," leading to a state of high-stakes "Big Robot crimewars" where the most efficient criminals battle the most ruthless robocops.

Doctorow uses this sci-fi allegory to explain a very real phenomenon already plaguing the US health system. He describes an "army of insurance company robo-claim-deniers" that has been met with a "doctors' army of robot-claim-appealers." This mirrors the concept of "automation blindness" seen in security checkpoints, where agents become so fixated on rare threats that they miss the obvious ones, or conversely, become so overwhelmed by false positives that they miss the real danger. The result is a system where the cost of defense drives up the cost of the service itself.

Critics might argue that some level of automated defense is necessary to prevent resource exhaustion, but Doctorow's analogy suggests that the current trajectory is not defense—it is escalation. The system becomes a battlefield where the only winners are the vendors selling the weapons.

The Spam Wars and Centralization

The argument deepens as Doctorow draws a parallel to the history of email spam. He recalls a time when setting up a mail server was easy, until anti-spam countermeasures like blocklists forced stricter technical standards. Spammers, for whom sending junk mail is an "existential matter," adapted to meet these standards, forcing the security demands to jump "again, and again, and again."

The outcome of this arms race was the centralization of the internet's email infrastructure into the hands of a "handful of giant, mostly US-based corporations." Doctorow warns that this centralization creates a single point of failure and control. If one of these giants decides your server is spamming, you "effectively disappear from the internet."

This historical context is crucial for understanding the current AI debate. Just as the spam wars led to a monopoly on communication, the bureaucratic AI arms race risks monopolizing access to public services. The stakes are even higher here, because unlike email, access to healthcare and welfare is not a luxury—it is a necessity. As Doctorow notes, the people caught in the middle are "collateral damage in a robotic forever war."

The only way to win is not to play.

Solutionism vs. Reality

Doctorow identifies the root cause of this dysfunction as "solutionism," a term coined by Evgeny Morozov to describe the tendency to recast complex social situations as "neat problems with definite, computable solutions." The administration's reliance on AI to root out AI-generated appeals is a classic example of this fallacy. It treats the symptom (frivolous claims) while ignoring the disease (a system that has become so adversarial that it requires such claims to navigate).

The reference to the "war on moisture" in aviation security serves as a grim reminder of how far these bureaucratic overreactions can go. Just as banning water bottles didn't stop terrorism but did inconvenience millions of travelers, pruning procedural rights to stop AI fraud will only make it harder for legitimate citizens to access the support they need. The system becomes a "problem factory," where the defenses against bad actors create the very conditions that necessitate even more aggressive defenses.

Bottom Line

Doctorow's most compelling insight is that the bureaucratic AI arms race is a form of mutually assured destruction, where the pursuit of perfect efficiency leads to total system failure. The strongest part of the argument is its historical grounding in the spam wars, which proves that technical fixes to social problems often lead to centralization and exclusion. The biggest vulnerability, however, is the lack of a clear, scalable alternative for handling the sheer volume of modern administrative claims. The reader should watch for how the executive branch balances the temptation of automation against the inevitable rise of a "service denial system" that leaves the most vulnerable behind.

Deep Dives

Explore these related deep dives:

  • Abelard Snazz

    This obscure 2000 AD character serves as the article's primary allegory for the inevitable escalation between automated legal systems and bureaucratic defenses.

  • Technological fix

    The article critiques the Economist's proposal to solve AI-driven legal overload with more AI as a classic example of this fallacy, where a technical solution is applied to a political problem.

  • Automation bias

    This cognitive phenomenon explains the 'automation blindness' mentioned in the text, detailing why human overseers like TSA agents or welfare processors fail to catch errors made by the AI systems they trust.

Sources

The bureaucratic AI arms-race is mutually assured destruction

by Cory Doctorow · Pluralistic · Read full article

Today's links.

The bureaucratic AI arms-race is mutually assured destruction: The only way to win is not to play. Hey look at this: Delights to delectate. Object permanence: Seymour Cray's tunnels; War on moisture; $5 wrench cryptanalysis; Nauru files; People's Ride; German transit upholstery fashion; Monopolies v small business; Linkedin will put you in ads; NZ Parliament kicks itself off the internet; Adblock Plus v Facebook adblock-block; Cracking 100m VWs for $40; Trump as defective machine learning; Canada sucks at internet law; Goodhart's Law of AI. Upcoming appearances: Edinburgh, Sydney, Melbourne, Brighton, London, South Bend. Recent appearances: Where I've been. Latest books: You keep readin' em, I'll keep writin' 'em. Upcoming books: Like I said, I'll keep writin' 'em. Colophon: All the rest.

The bureaucratic AI arms-race is mutually assured destruction (permalink).

According to an Economist editorial, "AI is breaking the British state" by making it too easy to file complaints, demands and appeals, which will "drown the state" with "demands as well-crafted as a first-class lawyer's":

https://archive.is/VTrj9

Let's pause a moment to appreciate the Economist's touching credulity about AI's coming legal mastery. The law seems to be the area where AI is most prone to "hallucinate" (that is, "produce defective outputs"), which can only be sorted through by skilled practitioners whose experience gives them the discernment to distinguish useful arguments from foolish ones:

https://pluralistic.net/2026/07/28/hitl-ers/#ai-ai-oh

(And this requires those skilled practitioners to avoid the "automation blindness" that afflicts people who are asked to remain vigilant for things that seldom occur, a phenomenon that has turned every TSA agent into the water-bottle-detectingest motherfucker the human race has ever produced, who still misses 95% of the guns that red teams bring through the checkpoint):

https://www.nbcnews.com/news/us-news/investigation-breaches-us-airports-allowed-weapons-through-n367851

More notable than the Economist's faith-based predictions about the impending army of hyper-competent robo-lawyers is the magazine's proposed solution to this looming crisis: "stop creating entitlements that are ripe for AI-fuelled claims…prune the mass of procedural rights." Above all, replace the bureaucrats who process your "complaints, demands and appeals" with more AI, which will arbitrarily decide who gets what, through "personalised welfare interventions" that are not based on any kind of guaranteed rights.

Writing on his blog, the political scientist Henry Farrell tells us where this will inevitably end up: with AI-based robot wars in which increasingly stingy and pernickety robo-bureaucrats create demand for progressively more aggressive robo-lawyers:

https://www.programmablemutter.com/p/the-downside-of-robot-solutionism

As Farrell writes, this end-time was foretold by ...