← Back to Library

How predictable are laws?

Most policy debates assume that if you write a law clearly, it will work as intended. Brian Potter dismantles that assumption with a startling dataset: nearly one in ten federal laws diverges so wildly from its original purpose that it creates a new reality the authors never imagined.

The Accidental Architects

Potter begins by exposing the chaotic origins of the National Environmental Policy Act (NEPA), a law that now dictates the timeline of almost every major infrastructure project in the US. The provision requiring environmental impact statements was not a carefully crafted strategy; it was a last-minute add-on to an aspirational declaration. Potter notes that the clause received “neither debate, nor opposition, nor affirmative endorsement” in Congress. It was passed because it was Christmas, and legislators wanted to go home.

How predictable are laws?

This historical context is crucial. It suggests that the massive bureaucratic machinery we rely on today often stems from legislative negligence rather than strategic design. Potter writes, “None of these effects of NEPA, however, were envisioned when the law was written.” The law was intended to be a statement of policy, yet the addition of the impact statement provision turned it into a potent legal weapon for environmental groups to halt projects for years.

The pattern repeats with the 401(k). What is now the backbone of American retirement savings began as a minor, overlooked provision in the 1978 Revenue Act. Potter highlights the absurdity of this outcome: “There was absolutely no discussion in ’78 that if you do this, the world is going to change.” The provision was estimated to have a “negligible effect upon budget receipts,” yet it evolved into the fifth-biggest tax break for individuals, costing the government billions annually.

“If anyone had known how important 401(k) would become, the Joint Committee on Taxation never would have let him, a 28-year-old junior lawyer, write it.”

This framing forces a reevaluation of legislative competence. It is not that lawmakers are malicious or incompetent; it is that the complexity of modern systems makes prediction nearly impossible. A counterargument worth considering is that some of these “accidents” are actually features of a flexible system that allows society to adapt laws to new realities without constant re-legislation. However, Potter’s data suggests the unpredictability is often a bug, not a feature, leading to massive inefficiencies.

The Methodology of Uncertainty

To quantify this unpredictability, Potter turned to artificial intelligence, a choice that is as pragmatic as it is controversial. He tasked an AI model to analyze 239 random federal laws passed between 1976 and 2023, scoring them on a divergence scale from -10 to 10. The goal was to compare the expected effect, based on contemporaneous sources, against the actual effect decades later.

The results form a bell curve with a slight positive skew, but the outliers are where the story lies. Potter found that while 40% of laws behaved exactly as expected, a substantial fraction—around 11%—diverged significantly. Some laws had effects far larger than anticipated, while others failed completely.

Potter illustrates this with the Trade Facilitation and Trade Enforcement Act of 2015. Billed as a customs modernization bill, it contained a provision raising the “de minimis exemption” from $200 to $800. Potter explains that this small change is “credited as a driver of the explosion of Chinese imports from companies like Shein/Temu over the next several years.” The law was designed to streamline trade, but it inadvertently fueled a specific, massive shift in global retail dynamics that the drafters likely never foresaw.

Conversely, he points to the Enhanced Partnership with Pakistan Act of 2009, which aimed to foster goodwill through billions in aid. Instead, the relationship deteriorated. Potter notes that by 2012, “74% of Pakistanis viewed the US as an enemy,” a stark reversal of the law’s intent. This failure wasn't just a lack of results; it was an active worsening of the situation, driven by events like the raid on Osama bin Laden’s compound in 2011—a historical footnote that Potter uses to show how external shocks can render well-intentioned legislation irrelevant.

The methodology has limitations, which Potter admits. Determining the “expected” effect of a law is fraught with ambiguity, especially when laws contain aspirational goals. For instance, the Indoor Radon Abatement Act of 1988 set a goal for indoor air to be as clean as outdoor air—a standard Potter suggests the authors knew was unlikely to be met. The AI scored this as a minor shortfall, but the line between “failure” and “aspirational goal” is often blurry.

“Most federal laws, it seems, do more or less what they’re expected to do. But a substantial fraction of them — around 11% — diverge significantly.”

Critics might argue that using AI to interpret historical legislative intent introduces a layer of algorithmic bias or hallucination. Potter acknowledges this, calling the results “preliminary” and noting the difficulty in getting the AI to research esoteric laws thoroughly. Yet, even with these caveats, the sheer volume of data—239 laws analyzed systematically—provides a compelling snapshot of legislative unpredictability.

The Cost of Unintended Consequences

The implications of these findings extend beyond academic curiosity. In the realm of housing, Potter highlights a series of California laws designed to supercharge construction. One law was supposed to flip defunct strip malls into apartments; another aimed to turn church parking lots into affordable housing. Despite the clear intent, a report by YIMBY Law found these measures had “limited to no impact on the state’s housing supply.”

This is a critical insight for anyone involved in infrastructure or policy. The gap between legislative text and real-world outcome is not always a matter of poor execution; sometimes, the system itself resists the intended change. Potter’s analysis of the Alaska Natural Gas Transportation Act of 1976 further illustrates this. The act was meant to build a massive pipeline, the “largest privately financed energy project ever undertaken.” It failed not because of a lack of will, but because subsequent laws created a gas supply glut that made the project economically unviable.

The divergence scores reveal a troubling trend: laws often fail not because they are ignored, but because they interact with a complex, shifting environment in ways that cannot be predicted at the drafting stage. Whether it is the deregulation of airline cargo that birthed the FedEx empire, or the failure of water compacts between Alabama and Georgia, the data shows that the executive branch and Congress are often flying blind.

Bottom Line

Potter’s most compelling argument is that the unpredictability of law is a structural constant, not a temporary glitch. The strongest part of this piece is the rigorous attempt to quantify a phenomenon usually dismissed as anecdotal, using a diverse sample of laws to show that 11% of legislation produces radical, unanticipated outcomes. The biggest vulnerability lies in the reliance on AI to interpret historical intent, a method that, while innovative, risks oversimplifying the nuanced political compromises behind every statute. As policymakers face increasingly complex global challenges, the lesson is clear: writing a law is only the beginning; surviving the unintended consequences is the real test.

Sources

How predictable are laws?

An enormous amount of ink has been spilled (some of it by me) on the trials and tribulations of complying with the National Environmental Policy Act, better known as NEPA. NEPA is what requires projects to perform years-long, thousand-plus page environmental impact studies before construction can begin, and suing a project for an insufficiently detailed environmental study is one of the chief ways environmental groups are able to slow down or stop projects they don’t like. And NEPA’s influence goes beyond federally funded projects: NEPA also influenced the creation of many similar laws, both at the state level (such as California’s CEQA) and in countries around the world.

None of these effects of NEPA, however, were envisioned when the law was written. NEPA was seen primarily as an (aspirational) statement of US environmental policy, which was to “encourage productive and enjoyable harmony between man and his environment, to promote efforts which will prevent or eliminate damage to the environment and biosphere and stimulate the health and welfare of man; [and] to enrich the understanding of the ecological systems and natural resources important to the Nation.” The provision that requires environmental impact statements was added last minute as a way to try to give some teeth to these high-minded but somewhat abstract ideals, and received virtually no attention at the time. As Alec Stapp and I noted:

[The provision] was not covered in any major media publication. In Congress, it received “neither debate, nor opposition, nor affirmative endorsement.” Caldwell would later state that “most [members] had never really understood the bill and only agreed to it because it was from Jackson; it was about the environment which was a very ‘hot’ issue at the time; and it was almost Christmas and they wanted to get home.”

Not until several months after NEPA was passed did environmental groups realize what a potent weapon they’d been handed.

It’s not hard to find other examples of laws whose effect was far different than what the authors anticipated. The 401(k) retirement account, now used by tens of millions of Americans as the primary vehicle for retirement savings, was originally considered an insignificant provision of the 1978 Revenue Act. Per a Bloomberg piece about the Act:

…The initial provision was estimated to have a “negligible effect upon budget receipts.” Now, defined contribution plans are the fifth-biggest tax break for individuals, with an estimated revenue loss to ...