This newsletter cuts through the usual hype cycle to ask a brutal question: can machines actually discover rules they were never taught, or are they just pattern-matching on steroids? Jack Clark doesn't just report on new benchmarks; he uses them to map the terrifyingly narrow gap between current AI capabilities and the threshold of recursive self-improvement. For a busy professional, the takeaway isn't that AI is "getting smarter," but that we are approaching a moment where the definition of scientific discovery itself may be outsourced to software that doesn't understand what it's doing.
The Hidden Rules of Discovery
Clark introduces DiG-bench, a new evaluation framework designed to test whether AI can infer the "unwritten rules" of an environment through pure exploration. The setup is elegant in its simplicity: "each game is a self-contained miniature world with its own laws, but both the rules and the objective are hidden from the player and must be uncovered through interaction." This approach moves beyond standard multiple-choice tests, forcing models to rely on curiosity and hypothesis testing rather than memorized datasets.
The results are sobering. While models like Opus 5 and Fable 5 show flashes of intuition, they are still outclassed by human cognition. Clark notes that "a 20% success rate on Tier 7 is pretty poor compared to the fact individual humans were able to get 100% on the tests." This disparity highlights a critical bottleneck: current systems struggle to update their priors when faced with truly novel mechanics.
The key thing this is measuring is the ability for players to spot the important mechanics that determine their success - basically, by playing around with the games you get a sense for how your actions change the environment and through this you also uncover mechanics that you must understand to succeed at the game.
The significance here extends beyond gaming. As Clark argues, these tests are "attempts to isolate a prerequisite for creativity, which is being able to autonomously discover useful undocumented things about novel situations you find yourself in." If a system cannot figure out the rules of a simple text game without explicit instruction, it is unlikely to navigate the chaotic, unstructured reality of scientific research or geopolitical strategy. Critics might note that text-based games are an artificial constraint, but Clark's point stands: if the model can't handle the "short enough" context of a game, it can't handle the long-horizon planning required for real-world invention.
The Simulator for the Singularity
Moving from testing to simulation, Clark highlights a browser-based game from Paradigm Research that attempts to model the dynamics of recursive self-improvement. This tool forces players to balance competing priorities: investing in researchers versus compute, and managing data licensing. It is a stark reminder that the path to superintelligence isn't just about better algorithms, but about resource allocation and strategic foresight.
The game serves as a proxy for understanding how AI labs might evolve. Clark suggests that "developing better intuitions about recursive self-improvement is of existential importance to us all." By playing out these scenarios, stakeholders can better grasp the feedback loops that could lead to rapid, uncontrollable capability gains. This connects to the historical concept of Goodhart's law—when a measure becomes a target, it ceases to be a good measure. In the context of the simulator, optimizing for "research output" might inadvertently trigger a runaway loop where the system prioritizes speed over safety, a dynamic that is difficult to reason about until it is already happening.
Developing better intuitions about recursive self-improvement is of existential importance to us all; games like this help make it easier for us to reason about this technology and the labs building it.
The simulation underscores that the "singularity" isn't a magical event, but a series of engineering and economic decisions that could compound exponentially. The danger lies in the assumption that these systems will remain controllable once they begin improving their own code.
The Scientist with Taste
Perhaps the most provocative development Clark covers is the work by the startup Inherent, which is training an AI model named Faraday to act as a "supervisory harness" for scientific research. The goal is to instill "taste"—the ability to decide what questions are worth asking and which experiments are worth running. Faraday is trained on a dataset called Replica, which consists of research papers with key results removed, forcing the AI to fill in the blanks.
The results are striking. Clark reports that Faraday "is able to beat standard Opus 4.8 and GPT-5.5 on some replication tasks, exceeding their performance on 73% of in-distribution ML tasks." This suggests that a smaller, specialized model can guide larger, more general models to better outcomes. The implication is profound: we may not need a single monolithic superintelligence to advance science; we might just need a "small supervisory model" that knows how to direct the others.
The skills that allow Faraday to fill in vaguely-specified details may be the very same skills that would allow it to advance the state of the art by designing its own experiment.
This framing shifts the debate from "will AI replace scientists?" to "will AI replace the judgment of scientists?" If an AI can successfully replicate research and design new experiments, the barrier to entry for scientific discovery collapses. The risk, as Clark points out, is that these skills "compound with advances in frontier coding models," potentially leading to a system that can improve itself faster than any human can audit.
The Missing Question in the Manifesto
Finally, Clark turns his critical eye to a recent essay by Mark Zuckerberg, which advocates for the mass proliferation of AI tools to ensure "individual empowerment." Zuckerberg's vision is one of balance, where everyone has access to superintelligence, preventing tyranny by concentrating power. Clark finds this logic compelling but ultimately flawed.
The core of Clark's critique is that the essay ignores the nature of the technology itself. "The part of this essay I understand the least is Zuckerberg's co-mingling of AI systems capable of invention with individual empowerment." The assumption that a system capable of superhuman invention will simply serve as a tool for human goals is, in Clark's view, a dangerous simplification.
The defining questions of our age are who will have access to superintelligence and what will we direct it towards.
Clark argues that the missing question is whether a system capable of superhuman invention will "solely work on behalf of the individual empowerment of people that are less capable than it at invention." The concern is not necessarily that AI will become malicious, but that its capabilities will fundamentally alter the balance of power in ways that are "confusing and hard to reason about." A counterargument worth considering is that widespread access could create a distributed defense against centralized control, but Clark rightly points out that this assumes the technology behaves predictably, which is the very thing we are trying to test.
The more superintelligence serves as a tool of invention, the more likely that individual capability outpaces automation and the future is better for people.
This optimism, while appealing, overlooks the possibility that the "invention" itself might outpace human comprehension. If the AI invents a new physics or a new economic model, the "individual empowerment" might be an illusion if the human user cannot understand the tool they are wielding.
Bottom Line
Jack Clark's analysis offers a necessary corrective to the prevailing optimism: we are not just building better tools, we are building systems that might soon build themselves. The strongest part of his argument is the empirical evidence from DiG-bench and Faraday, which shows that AI is rapidly closing the gap on human intuition and scientific judgment. The biggest vulnerability in the current discourse, as Clark highlights, is the assumption that these systems will remain obedient servants to human intent. The next six months will be critical as we watch whether these "discovery" benchmarks can be solved by models that don't just mimic, but truly understand.