← Back to Library

Xi-the administration to talk AI safety, huh?

This piece cuts through the diplomatic fog surrounding the upcoming US-China summit to reveal a startling truth: the sudden urgency for AI safety talks isn't born of altruism, but of a terrifying realization that dangerous capabilities are already here. Jordan Schneider and his guests dissect how a single breakthrough in artificial intelligence has forced both Washington and Beijing to abandon their posturing and confront the reality that vulnerability and capability are inextricably linked. For the busy strategist, this is not just a recap of a meeting; it is a warning that the window for theoretical debate has closed.

The Mythos Moment

The core of the argument rests on a dramatic shift in the US executive branch's posture, driven not by policy evolution but by a specific technological shock. Julian Gewirtz, a former National Security Council senior director, notes that while the previous administration had to fight to get AI safety on the agenda, the current leadership initially dismissed the entire concept. "The Trump administration initially showed little concern about AI safety. JD Vance and other senior officials openly mocked AI safety as a construct, making US-China AI safety dialogue a non-starter," Gewirtz explains. This dismissal has now evaporated.

Xi-the administration to talk AI safety, huh?

What changed? The emergence of "Mythos," a frontier AI capability that proved dangerous potential is not a distant conjecture. "What's changed recently in the Trump administration appears directly tied to the Anthropic Mythos moment," Gewirtz argues. The administration has realized that "extraordinary and potentially dangerous AI capabilities aren't theoretical conjectures for years down the line but exist in the real world right now." This is a crucial pivot. It suggests that the drive for safety is less about moral leadership and more about immediate national security survival. The administration is now backgrounding expectations that AI will be a primary topic, driven by the fear that "advances in capability cannot be separated from increases in vulnerability."

One lesson from Mythos appears to be that for both the United States and China, advances in capability cannot be separated from increases in vulnerability.

This framing is effective because it strips away the ideological posturing that usually dominates US-China relations. However, it also highlights a dangerous fragility in the current approach: if the motivation for safety is purely reactive to a specific model's capabilities, what happens when the next breakthrough renders the current safety protocols obsolete? The reliance on a "shock" to drive policy is a risky strategy for long-term governance.

Beijing's Calculated Shift

On the Chinese side, the evolution is equally significant but follows a different trajectory. Matt Sheehan, a fellow at the Carnegie Endowment, observes that Beijing has moved from a "cold shoulder" to actively placing AI safety on the table, though their internal consensus remains fluid. "They have essentially put it on the table as a topic that they need to think through, but they haven't made up their mind on what they think of it," Sheehan notes. This ambiguity is reflected in their policy documents. The "AI Safety and Governance Framework 2.0" shows a marked shift from the 2024 version, particularly regarding the economic impact of automation.

In the earlier document, the impact on labor was "handwaving," but the updated version acknowledges that AI "will lead to a devaluation of labor relative to capital and social disruptions related." This is a profound admission from a system that has historically prioritized stability above all else. It signals that the Chinese leadership is beginning to grapple with the domestic fallout of the very technology they are racing to dominate.

The mechanism for this shift is a "control, harness, govern" playbook that mirrors their handling of the internet era. Just as the state first cracked down on speech (control), then promoted "Internet Plus" to boost the economy (harness), and finally implemented strict data laws (govern), they are now applying this same logic to AI. "For AI, they've resuscitated the 'plus' formulation with 'AI+.' For those unfamiliar, AI+ means AI+ manufacturing, AI+ healthcare," Sheehan explains. This historical parallel is vital context; it suggests that the current focus on safety is not a sudden moral awakening but a calculated move to manage the "knock-on effects" before they threaten the regime's stability.

Critics might note that assuming Beijing will follow the same predictable regulatory path as the internet era ignores the unique speed and opacity of AI development. The "govern" phase in the internet era took years; AI could compress that timeline into months, leaving little room for the state to react before systemic risks materialize.

The effort should go into trying to establish some working level, more technical conversations, specifically on testing and evaluation for safety risks.

Sheehan's assessment of what is achievable is refreshingly grounded. He expresses "very low expectations" for a grand bargain, arguing instead for technical cooperation on testing. This is a pragmatic stance that acknowledges the deep mistrust between the two powers. "If you take Chinese capabilities relatively seriously... their capabilities matter," he says, urging the US to help bolster China's internal testing regimes. This is a counter-intuitive but necessary argument: a safer China is a safer world, even if the two nations remain geopolitical rivals.

The Inevitable Crackdown

The conversation concludes with a sobering prediction: the era of "hands-off" AI development in China is ending. Julian Gewirtz draws a direct line from the current AI landscape to the crackdown on internet giants like Alibaba. "Initially, it was, as long as you do censorship, you're okay... But then they began to realize that even with that set of technologies, there were systemic risks," Gewirtz recalls. He argues that the same regulatory storm is coming for AI. "That other shoe has to drop. I don't see a way around it."

The catalyst will likely be a combination of cyber vulnerabilities and labor disruption. "When things hit the fan in China from a domestic perspective, you have to think they're going to start doing more testing than just checking if you're saying anti-party stuff," Jordan Schneider adds. This suggests that the Chinese state's primary motivation for AI safety is not global altruism, but the preservation of its own control against the very tools it seeks to deploy.

The US government... was able to spend a year and a half dismissing it because it wasn't really all that pressing. But everyone's consensus view now is that... at some point in the not-too-distant future, there will be Chinese labs able to create extremely cheap, extremely potent cyberweapons from a domestically trained model.

This prediction carries immense weight. It implies that the current window for dialogue is narrow. If the US and China fail to establish even basic technical standards now, they risk facing a future where both nations are armed with AI-driven cyberweapons that neither fully understands or controls. The historical precedent of the internet crackdown offers a roadmap, but the stakes are exponentially higher.

Bottom Line

Schneider's analysis succeeds in reframing the AI safety dialogue from a diplomatic courtesy to a strategic imperative driven by immediate, tangible threats. The strongest part of the argument is the identification of the "Mythos" moment as the catalyst that forced both superpowers to drop their guard and acknowledge shared vulnerability. The biggest vulnerability, however, lies in the assumption that technical cooperation can survive the intense geopolitical friction between the two nations; without a broader political thaw, these working-level talks may remain fragile. Readers should watch for the specific content of the "AI Safety and Governance Framework 2.0" implementation, as that will be the first real test of whether Beijing is prepared to govern the risks it has helped create.

Deep Dives

Explore these related deep dives:

  • Cyberspace Administration of China

    This regulatory body is the specific Chinese entity tasked with implementing the AI safety protocols and export control responses discussed in the dialogue.

  • Internet Plus

    This state strategy provides the historical context for China's initial focus on rapid AI deployment over safety, explaining their early reluctance to engage in US-led safety dialogues.

Sources

Xi-the administration to talk AI safety, huh?

by Jordan Schneider · ChinaTalk · Read full article

ChinaTalk is in SF! RSVP for an impomptu meetup tonight.

Today, the second half of our conversation previewing the summit that just kicked off. With Mythos scrambling everyone’s priors on frontier capabilities, AI safety is suddenly back on the bilateral agenda. Julian Gewirtz (former NSC senior director for China) and Matt Sheehan (Carnegie) join to map how Beijing is processing the shift and what’s actually achievable in renewed US-China dialogue.

Check us out on YouTube or your favorite podcast app!

Good Job Alert: Coefficient Giving (formerly Open Philanthropy) is hiring senior generalists and grantmakers with a China background. Coefficient created CSET (who won ChinaTalk’s inaugural ‘think tank of the year’ award) and are set to fund more in the US-China AI space. Check out the job posting here and do consider applying..

The AI Safety Angle.

Julian Gewirtz: Both sides have been signaling that AI will feature prominently in upcoming discussions. During the Biden administration, we pushed hard to get AI safety on the agenda when President Biden met with President Xi. Beijing initially gave us the cold shoulder, but gradually realized there was no major downside to including it on the agenda.

The Trump administration initially showed little concern about AI safety. JD Vance and other senior officials openly mocked AI safety as a construct, making US-China AI safety dialogue a non-starter — the United States didn’t even want it.

What’s changed recently in the Trump administration appears directly tied to the Anthropic Mythos moment. The realization that extraordinary and potentially dangerous AI capabilities aren’t theoretical conjectures for years down the line but exist in the real world right now has made the administration take this issue more seriously.

Both the Chinese and Americans are now backgrounding expectations that AI will come up in discussions, with potential AI safety-related deliverables. During the Biden administration, we pushed hard to get this topic on the leaders’ agenda. China’s initial response was essentially a cold shoulder — they weren’t interested in having the conversation. They felt it was happening in an environment of heating AI competition and were unhappy with export controls and other steps we were taking.

Whether we wore them down or won them over, the topic eventually came up between the leaders. Jake Sullivan also discussed it with Wang Yi. Beijing shifted its approach after realizing this was an area where the world was looking to the two most ...