← Back to Library

Do AI risks require extraordinary government intervention?

Most policy debates treat AI as a unique anomaly demanding emergency powers, but Arvind Narayanan & Sayash Kapoor argue that this panic ignores a simpler, more effective path: building societal resilience. They challenge the prevailing narrative that we must restrict innovation to survive, suggesting instead that the real danger lies in granting the executive branch unchecked authority over research and speech.

The Trap of "Abnormal" Technology

The piece opens by dismantling the idea that AI requires a special regulatory regime. While some observers point to emerging cyber and bio-risks as proof that AI is fundamentally different from past technologies, Narayanan & Sayash Kapoor contend that this distinction is often overblown. They write, "Thompson argues that AI seems particularly 'abnormal' because of risks that are emergent and unknown even to AI developers. And this justifies plans by the government to treat AI as an abnormal technology." This framing is compelling because it exposes a logical leap: just because a risk is hard to predict does not mean the only solution is to halt development. The authors suggest that treating AI as an exception rather than a continuation of technological progress risks creating a permanent state of emergency governance.

"Extraordinary interventions impose restrictions on the liberty of actors who are not directly responsible for the harms in question."

The authors define "extraordinary intervention" with surgical precision, noting that such measures often restrict the freedom of builders to punish potential bad actors. This is a crucial distinction. By targeting the companies that create tools rather than the individuals who misuse them, the government risks cutting off beneficial access for the public. Critics might argue that in the face of existential threats, the cost to liberty is a necessary trade-off, but Narayanan & Sayash Kapoor push back, reminding us that "the burden falls not on the malicious actors who cause harm, but on companies that build tools that could, in principle, be misused." This approach mirrors historical struggles where the cure was worse than the disease.

Do AI risks require extraordinary government intervention?

The Illusion of Nonproliferation

A central pillar of their argument is the futility of trying to replicate nuclear-style nonproliferation in the digital realm. Unlike enriched uranium, which requires physical bottlenecks, the core techniques for building AI are well known and easily replicated. Narayanan & Sayash Kapoor write, "AI is different from nuclear weapons. For one, there is no equivalent 'physical' bottleneck. The core techniques for building AI systems are well known." This is a devastating critique of the current policy obsession with export controls and licensing. They point out that even with strict controls, the gap between frontier models and public access is shrinking to mere months, not years.

The authors draw a sharp parallel to the history of cryptography. In the 1990s, the government attempted to restrict encryption software, arguing it would help criminals evade law enforcement. They even investigated a programmer under the Arms Export Control Act for releasing code. Yet, as Narayanan & Sayash Kapoor note, "These restrictions were eventually rolled back... Encryption became the foundation of digital security, enabling e-commerce, online banking, and many other applications." This historical context is vital; it shows that attempts to suppress dual-use technology often backfire, stifling the very defenses needed to secure society. The lesson is clear: trying to wall off technology rarely works, but making society robust enough to withstand it is a proven strategy.

"Resilience distributes defenses across society."

Instead of a single chokepoint, the authors advocate for a distributed defense. They argue that we have successfully managed similar transitions before, such as the rise of automated vulnerability detection tools. These tools, once feared for empowering attackers, became the backbone of modern cybersecurity because defenders had access to them too. "Since defenders had access to the same tools, they became core defensive tools, largely funded by the cyberdefense ecosystem," they explain. This suggests that the offense-defense balance can be managed through investment and adaptation rather than prohibition. A counterargument worth considering is that AI's speed of iteration might outpace our ability to build resilience, but the authors maintain that betting on a brittle nonproliferation regime is a far riskier gamble.

The Slippery Slope of Executive Power

Perhaps the most urgent warning in the piece concerns the long-term impact on democratic governance. If we accept the premise that AI is an "abnormal" threat, we open the door to permanent expansions of state power. Narayanan & Sayash Kapoor ask, "Would they support restrictions on open-weight models? What about requiring approvals for each new model release, or restrictions on the movement of researchers who build frontier AI across countries?" They warn that without clear limits, the demands for government control will escalate as capabilities advance. This is not just about AI; it is about the precedent it sets for all future technologies.

The authors highlight the danger of bypassing normal governance processes. "Extraordinary interventions bypass normal processes of governance, and instead rely on unilateral authority such as emergency declarations or executive orders," they write. This bypasses the democratic accountability that exists to protect liberty. The piece forces us to confront a difficult choice: invest in the slow, unglamorous work of improving resilience, or accept a future where the government decides what research is safe to publish. As they put it, "Nonproliferation is brittle because it relies on a single chokepoint. Resilience distributes defenses across society."

Bottom Line

Narayanan & Sayash Kapoor make a powerful case that the rush to regulate AI as an existential emergency is a strategic error that threatens both innovation and liberty. Their strongest argument is the historical evidence that suppression of dual-use technology fails, while resilience succeeds. The biggest vulnerability in their position is the assumption that society can adapt fast enough to AI's rapid evolution without some form of pre-emptive restriction, but their call for a shift from control to preparation is a necessary corrective to the current panic. Watch for how policymakers balance these competing pressures in the coming months.

Deep Dives

Explore these related deep dives:

  • The Age of Surveillance Capitalism Amazon · Better World Books by Shoshana Zuboff

    How tech companies turned human experience into raw material for prediction and control.

  • Export of cryptography from the United States

    This historical precedent illustrates the practical failure of 'extraordinary' nonproliferation regimes, showing how export controls on dual-use technology often drive innovation underground rather than stopping it, directly supporting the article's argument that resilience is superior to brittle chokepoints.

  • Dual-use technology

    This concept explains the specific tension the article identifies where the same AI capabilities that drive economic growth also lower the barrier for cyber and bio-attacks, clarifying why the author argues that restricting release is a flawed strategy compared to building societal resilience.

  • Regulatory capture

    The article suggests that the ease of targeting AI companies rather than society stems from bureaucratic incentives; this topic provides the mechanism for how agencies might become overly dependent on the very industries they are supposed to regulate, leading to the 'sclerosis' and ineffective extraordinary interventions the authors warn against.

Sources

Do AI risks require extraordinary government intervention?

by Arvind Narayanan & Sayash Kapoor · AI Snake Oil · Read full article

In a recent essay, Derek Thompson engages with AI as Normal Technology (AINT). He agrees with our thesis about AI’s slow labor market impacts, relying on the fact that GDP growth has so far been average, unemployment is below 5%, and even jobs that seemed vulnerable to automation show rising employment and wages. He concludes that so far, the macroeconomic picture is consistent with what we would expect from a “normal” general-purpose technology.

But when it comes to AI risks, he is far more bearish. He points to examples of cyber- and bio-risks and expresses pessimism about AI quickly becoming dangerous across many new domains. He argues that AI’s emergent capabilities make it fundamentally different from previous technologies, and that this difference justifies “extraordinary” government responses including restrictions on what companies can release.1

In this essay, we lay out the downsides of extraordinary government intervention in response to new technology. We discuss proposals for improving resilience that do not require such intervention. We also discuss why governments have so far been reluctant to invest in resilience. In short, resilience requires us to get better at the *normal* process of policymaking. But sclerosis in the federal government and the ease of justifying interventions on AI companies rather than society at large make extraordinary intervention seem appealing, despite its limitations.

What happens next? Policymakers can either get their act together and invest heavily in improving resilience, or be forced to take extraordinary actions (such as on AI nonproliferation) that are more onerous and less effective.

A Recap of AINT’s Arguments and Thompson’s Areas of Agreement and Disagreement.

Many people, including Thompson, have found AI as Normal Technology to be a useful framework for thinking about AI’s economic impacts while being unconvinced by our views on safety. In the AINT essay, we make different arguments about slow labor market impacts and resilience to misuse risks.

The labor market argument rests on the speed of diffusion: there are many speed limits between a new AI capability and its economic impact, including the need to build products, change organizational workflows, and navigate regulation. This has proven helpful both to understand why claims of rapid and widespread job displacement (such as Amodei’s claims about an imminent white-collar bloodbath) are unlikely to materialize, as well as to identify bottlenecks to diffusion that could hinder the beneficial adoption of AI.

But our argument about AI’s misuse risks depends ...