AI safety
Based on Wikipedia: AI safety
In 2023, as generative AI systems exploded into public consciousness, a quiet crisis unfolded in research labs worldwide. The same technology that wowed users with its ability to generate human-sounding text and stunning artwork also raised uncomfortable questions: What happens when these systems become powerful enough to reshape industries—or destabilize societies? That year, the United Kingdom hosted a summit at Bletchley Park—not the first choice for modern AI discussions, but an appropriate location for a field obsessed with historical analogies. The event drew researchers, policymakers, and technologists grappling with a simple but profound challenge: how to ensure artificial intelligence remains safe, beneficial, and under human control.
AI safety, as a formal discipline, sits at the intersection of technical research, policy advocacy, and ethical philosophy. It encompasses preventing accidents—where an AI system behaves unexpectedly in ways that harm people—and combating misuse, where actors deploy AI for deliberately destructive purposes. The field gained significant traction during 2023, driven by rapid advances in generative models and public alarm voiced by researchers and executives about potential dangers. Yet the concerns themselves are older than most observers realize.
The roots of modern AI safety thinking stretch back to the earliest days of computing. In a 1988 book published years before neural networks entered mainstream discourse, Blay Whitby already outlined the need for developing artificial intelligence along ethical and socially responsible lines—arguing that every degree of independence given to machines represented a potential degree of defiance against human intentions. From those early warnings emerged decades of academic engagement.
The turning point came in 2014, when philosopher Nick Bostrom published Superintelligence: Paths, Dangers, Strategies. The book argued that the rise of artificial general intelligence—AI capable of performing any intellectual task a human can—had the potential to create various societal issues ranging from workforce displacement to manipulation of political and military structures, even including the possibility of human extinction. Bostrom's argument prompted figures like Elon Musk, Bill Gates, and Stephen Hawking to voice similar concerns publicly.
Yet AI researchers have never reached consensus on these risks. In 2015, Andrew Ng compared existential anxieties about AGI to "worrying about overpopulation on Mars when we have not even set foot on the planet yet"—arguing that current AI systems remain far from such threats. Conversely, Stuart Russell has urged caution, stating that "it is better to anticipate human ingenuity than to underestimate it." The debate remains unresolved, but two surveys offer insight into expert perspectives: in separate polls of AI researchers, the median respondent placed a 5% probability on an "extremely bad" outcome—such as human extinction—from advanced AI. In a 2022 survey of the natural language processing community, 37% agreed or weakly agreed that it was plausible that AI decisions could lead to a catastrophe at least as severe as an all-out nuclear war.
The technical heart of AI safety involves three interconnected problems: robustness, monitoring, and alignment. Alignment ensures systems behave as intended—rather than optimizing for goals that look correct but contain hidden dangers. Monitoring involves building tools to detect when AI systems malfunction or behave in unexpected ways. Robustness means engineering systems that remain stable against adversarial manipulation.
The concept of adversarial examples illustrates these challenges with particular clarity. In 2013, researchers discovered that adding specific perturbations—imperceptible changes—to images could cause neural networks to make confident but incorrect classifications. A picture of a schoolbus could be misclassified as a cat. These vulnerabilities persist across modern architectures, and while recent work shows the perturbations are often noticeable rather than imperceptible, the underlying problem remains unsolved.
Concrete milestones mark the field's evolution. In 2015, dozens of leading AI experts signed an open letter calling for research on societal impacts—an appeal that attracted over 8,000 signatories including Yann LeCun, Shane Legg, Yoshua Bengio, and Stuart Russell. The same year, Berkeley's Center for Human-Compatible Intelligence emerged alongside the Future of Life Institute awarding $6.5 million in grants aimed at ensuring AI remains safe, ethical, and beneficial.
By 2016, the White House Office of Science and Technology Policy partnered with Carnegie Mellon University to host a public workshop on safety and control—part of four consecutive workshops investigating advantages and drawbacks of artificial intelligence. Concrete Problems in AI Safety, one of the first influential technical agendas in the field, was published that same year.
The 2017 Asilomar Conference on Beneficial AI, sponsored by the Future of Life Institute, gathered over 100 thought leaders to formulate principles including "Race Avoidance: Teams developing AI systems should actively cooperate to avoid corner-cutting on safety standards." By 2018, DeepMind's Safety team outlined problems in specification, robustness, and assurance—their work catalyzed a workshop at ICLR focused entirely on these issues. The following years saw Unsolved Problems in ML Safety published in 2021, outlining research directions spanning robustness, monitoring, alignment, and systemic safety.
In November 2023, the AI Safety Summit convened with an explicit focus on risks of misuse and loss of control associated with frontier AI models—the same models now deployed across major industries. During the summit, intentions emerged to create an International Scientific Report on the Safety of Advanced AI.
The following year brought formal collaboration. On April 1, 2024, U.S. Commerce Secretary Gina Raimondo and UK Technology Secretary Michelle Donelan signed a memorandum of understanding to jointly develop advanced AI model testing—a partnership born from commitments announced at that Bletchley Park summit. The agreement established national AI Safety Institutes in both countries as operational entities.
By 2025, an international team of 96 experts chaired by Yoshua Bengio published the first International AI Safety Report—commissioned by 30 nations and the United Nations. The report represented the first global scientific review of potential risks associated with advanced artificial intelligence: detailing threats stemming from misuse, malfunction, and societal disruption, aiming to inform policy through evidence-based findings without specifying recommendations.
Today, researchers discuss both current and emerging risks. Present dangers include critical systems failures—AI making consequential decisions in medicine, finance, or infrastructure—and algorithmic bias embedded in surveillance technologies that disproportionately target marginalized groups. Emerging risks encompass technological unemployment at scale, digital manipulation of political discourse, weaponization of AI systems, cyberattacks enhanced by artificial intelligence, and bioterrorism enabled by AI's biological knowledge.
Speculative risks include losing control over future AGI agents—systems far more capable than anything currently deployed—or AI enabling perpetually stable dictatorships that resist removal. Yet critics like Andrew Ng remain unmoved: his Mars analogy from 2015 suggests concerns about artificial general intelligence may be premature given we haven't even landed on the planet yet.
The field now operates at multiple levels—from academic conferences to government regulation—and encompasses technical research, norm development, and policy advocacy. In 2023, Prime Minister Rishi Sunak stated ambitions for the United Kingdom to become "the geographical home of global AI safety regulation"—a goal that reflects how deeply this discipline has entered mainstream discourse.
The fundamental question remains: as capabilities accelerate faster than governance frameworks, can humanity maintain meaningful control over systems increasingly embedded in daily life? The answer will likely determine whether artificial intelligence becomes history's greatest amplification of human potential—or its most consequential miscalculation.