← Back to Library

Are open models catching up?

Dylan Patel delivers a counterintuitive verdict that should unsettle the entire artificial intelligence industry: the gap between proprietary and open-source models isn't just closing; it is collapsing at an accelerating pace. Rather than relying on hype or marketing fluff, Patel constructs a rigorous, era-by-era analysis showing that open models now take half as long to catch up to frontier leaders with every new cycle of innovation. For investors and engineers alike, this suggests the "moat" of closed models is evaporating faster than anyone predicted.

The Accelerating Cycle of Convergence

Patel's central thesis reframes the history of large language models not as a linear race, but as a series of distinct eras where the lead changes hands. He argues that "naively, you might pick a single set of benchmarks to measure all historical models, but this is a mistake." Instead, he divides the timeline into three phases: early scaling, reasoning, and agentic work. This structural approach allows him to isolate exactly when and how open-source communities replicate the breakthroughs of their closed competitors.

Are open models catching up?

The data reveals a disturbing trend for frontier labs. "At the start of each era, a frontier lab completes some promising research, trains an impressive model, deploys it at scale to their users, and jumps ahead," Patel writes. "Then, other labs identify the key advances, reverse-engineer what the frontier lab is doing, replicate them in their own models, and close the gap." The speed of this replication is the story. In the first era, it took roughly a year for open models to approach the capabilities of the best closed systems. By the third era, that window had shrunk to mere months.

"Nothing stays secret forever—especially when you factor in distillation. It's just a question of how long it takes."

This observation is particularly stark when looking at the "reasoning" era. Patel notes that while the release of advanced reasoning models like o1 initially seemed to reset the board, the open-source response was immediate. He points to DeepSeek R1, which narrowed the capability gap to just 12.1 points, a stark contrast to the 35.8-point gap seen in the previous era. The market's initial panic over this development was short-lived, replaced by the realization that "good models being open source is good for AI infrastructure."

Critics might argue that benchmark scores do not fully capture the nuance of real-world deployment, where proprietary safety layers and specific tool integrations matter. However, Patel anticipates this, acknowledging that while benchmarks are imperfect, the trend of halving catch-up times is "remarkably consistent" across three distinct technological shifts.

The Agentic Era and the Commoditization Risk

The most significant shift Patel identifies is the move toward "agentic" capabilities, where models don't just answer questions but execute complex tasks like coding and web navigation. Here, the economic stakes are highest. Patel highlights that Anthropic's focus on this area generated over $65 billion in annual recurring revenue, yet the open-source catch-up was faster than ever. "Kimi K2.6 surpassed Opus 4.5 with a score of 56.3 in 4.8 months, and GLM-5.2 cleared GPT-5.2 with a score of 72.4 in 6 months," he notes.

This rapid convergence raises a terrifying question for the business models of the executive branch's primary AI partners and the private labs they fund. "If open models stay capable enough relative to the closed frontier at a fraction of the cost, won't the model layer become commoditized?" Patel asks. The implication is that the massive capital expenditure required to train frontier models may yield diminishing returns if the resulting intelligence can be replicated by smaller, more efficient open-source efforts.

"Competition is heating up, usage resets are being doled out, and the battle for your tokens now extends beyond the OpenAI-Anthropic duopoly."

Patel's analysis of the agentic era also exposes a shift in what constitutes a "win." It is no longer just about the model weights; it is about the "model + harness" product. While OpenAI and Anthropic raced to release models every 51 days on average, open-source competitors like Kimi and GLM were closing the gap by focusing on specific, high-value tasks. This suggests that the barrier to entry is no longer just compute power, but the ability to integrate models into useful workflows—a task where open-source communities are proving surprisingly agile.

The Limits of Benchmarking and the Future

Despite the compelling data, Patel remains grounded in the limitations of his own methodology. He admits that "benchmarks don't tell the full story" and that public evaluations can be gamed through "hill climbing"—where models are specifically tuned to pass tests without gaining general intelligence. He even concedes that while a model like Kimi K3 might score higher on a composite chart, his team still prefers using proprietary models for daily work due to better productization.

"Benchmarks aren't a perfect proxy for real work. This is especially true for public benchmarks, which model makers can easily hill climb by simply creating a bunch of RL environments that closely mimic the benchmark tasks."

This caveat is crucial. It prevents the piece from becoming a triumphalist manifesto for open source and instead frames it as a warning about the sustainability of the current closed-model economic model. If the primary differentiator is the speed of release and the quality of the surrounding ecosystem, rather than raw model intelligence, the "commoditization" fear becomes a reality.

Bottom Line

Patel's most valuable contribution is the empirical evidence that the "secret sauce" of frontier labs is leaking faster than ever, turning a decade-long advantage into a matter of months. The argument's greatest vulnerability lies in its reliance on benchmarks that may not fully reflect the safety and reliability requirements of enterprise deployment. Readers should watch not just for the next model release, but for how the industry pivots from selling raw intelligence to selling the proprietary tools that make that intelligence usable.

"Nothing stays secret forever—especially when you factor in distillation. It's just a question of how long it takes.""

Deep Dives

Explore these related deep dives:

  • Knowledge distillation

    The article identifies distillation as the primary mechanism allowing open-source labs to reverse-engineer frontier capabilities, making this technical process the engine behind the predicted halving of catch-up times.

  • Mixture of experts

    Understanding this specific neural network architecture is essential to grasp how models like Kimi and GLM achieve high performance at a fraction of the cost, directly addressing the article's concern about the commoditization of the model layer.

  • Humanity's Last Exam

    The article argues that historical evaluation metrics become useless once models master them, a phenomenon best explained by this concept which illustrates why the 'reasoning' and 'agentic' eras require entirely new measurement standards.

Sources

Are open models catching up?

by Dylan Patel · SemiAnalysis · Read full article

The past two months have been a breakout period for open source AI. Yes, there was the “DeepSeek moment” back in January 2025, but no one actually used R1 to do any economically valuable work. In contrast, models like GLM 5.3 and Kimi K3 are genuinely capable of many of the same coding and agentic tasks that rocketed Anthropic to $65B+ ARR. Unlike others who inflated ARR, our figures were much closer to reality.

It is an exciting time to be a token consumer. Competition is heating up, usage resets are being doled out, and the battle for your tokens now extends beyond the OpenAI-Anthropic duopoly. Fireworks alone is processing over 40T tokens per day—2x the OpenAI API’s volume at the end of March.

However, major FUD has also emerged as a result of open model success: if open models stay capable enough relative to the closed frontier at a fraction of the cost, won't the model layer become commoditized? This outcome would obviously be disastrous for frontier lab margins. For full details on Anthropic and OpenAI’s financials, see our Tokenomics Model.

To project how the open vs closed capability gap will progress in the future, we first need to measure the past. Naively, you might pick a single set of benchmarks to measure all historical models, but this is a mistake. Every benchmark is a product of a particular era. When someone creates a new benchmark, their goal is to discern differences in model capabilities at the time. If they’re successful, the model makers will climb said benchmark until it becomes saturated. Once that happens, everyone stops caring about the benchmark, and the cycle repeats.

There have been three eras thus far in the history of LLMs: early scaling, reasoning, and agentic. Each era represented a step-function increase in model utility, and rather than trying to plot a single continuous trend, we believe it’s better to evaluate the models and benchmarks from each era individually.

When viewed this way, it becomes clear that the open vs. closed gap moves in cycles. At the start of each era, a frontier lab completes some promising research, trains an impressive model, deploys it at scale to their users, and jumps ahead. Then, other labs identify the key advances, reverse-engineer what the frontier lab is doing, replicate them in their own models, and close the gap. Nothing stays secret forever—especially when you factor ...