Sholto Douglas and Trenton Bricken predict that by late 2025, AI agents will complete a full day's worth of software engineering work—marking a decisive shift from mere chatbot assistants to genuine digital workers. The claim isn't speculative: it's based on concrete evidence from competitive programming and mathematics domains where reinforcement learning has achieved expert-level performance.
The New Reality
Sholto Douglas and Trenton Bricken are both researchers at Anthropic, focused respectively on scaling reinforcement learning and mechanistic interpretability. They sat down with host Dwarkesh Patel to discuss what's changed since their 2024 conversation—and the answer is striking: RL and language models have finally worked.
The proof points are concrete. In competitive programming and mathematics, these systems now demonstrate reliable performance when given clean feedback signals. Unit tests pass. Code compiles. Math problems resolve to correct answers. The axis of intellectual complexity has been scaled.
What hasn't emerged yet is long-running agentic performance—autonomous systems that can operate across extended time horizons without constant human intervention. But the early steps are visible, and real software engineering agents are beginning to demonstrate competence.
Why Software Engineering Leads
The domain where these models excel most immediately is software engineering—not by accident but by design. The field offers what researchers call "verifiable rewards": does the code pass a test? Does it compile? Does it run?
Writing essays lacks equivalent verifiability. Grading an essay requires subjective taste, and humans are notoriously poor judges of quality—laced with length biases and preferences that don't correlate with actual excellence.
The researchers noted an illuminating comparison: a Nobel Prize-winning discovery may be more achievable than a Pulitzer-worthy novel. Scientific work layers verifiable experiments, measurable outcomes, and iterative testing. Literary creativity remains stubbornly resistant to automated verification.
This reveals something fundamental about the current generation of AI systems: when given a clear feedback loop, they perform impressively. Without one, they struggle.
The Reliability Question
The core bottleneck isn't merely "extra nines" of reliability—ten times more reliable, or ninety times more reliable—as previously assumed. Instead, it's the absence of context and the ability to handle complex, multi-file changes across larger scopes of work.
When a task requires discovery and iteration within an environment, these systems still falter. They handle well-scoped problems with high intellectual complexity effectively but struggle when tasks become amorphous or require sustained interaction with the world.
The researchers expect agents to perform close to a junior engineer's worth of work by the end of 2025—perhaps saving an entire day of boilerplate coding or delivering several hours of competent, independent software engineering.
The Capabilities Debate
A persistent critique questions whether RL is actually creating new capabilities or simply revealing what was already present in pre-training. A paper from Stanford demonstrated that giving a base model enough tries allows it to answer questions as well as reasoning models—essentially narrowing down the probability space of possible answers rather than generating new ones.
Douglas counters that the compute investment in reinforcement learning represents genuine capability addition, not mere revelation. The amount of training data matters: when RL signal is sufficiently clean, these systems learn knowledge exceeding human-level performance—from chess to Go—and do so from scratch.
The economics are telling. Labs currently spend hundreds of millions on base model pre-training but only a few million on reinforcement learning. This isn't compute-limited yet—but will become increasingly significant as RL scales.
Counterpoints
Critics might note that the distinction between "new capabilities" and "revealed capabilities" matters significantly for understanding what's actually happening with these systems. If reasoning models merely narrow exploration rather than learn new patterns, the apparent progress in mathematics and programming could be an artifact of evaluation design rather than fundamental improvement.
Additionally, predicting software engineering agents performing a full day's work by 2025 carries inherent uncertainty—the field has seen premature declarations before.
Bottom Line
Douglas and Bricken present compelling evidence that RL plus language models has crossed a threshold in verifiable domains like competitive programming and mathematics. Their prediction for end-of-2025 agent capability represents the most substantive shift from chatbot assistants to autonomous workers we've yet seen. The fundamental insight is clear: when feedback loops are clean, these systems perform remarkably; without them, they falter. The next frontier isn't more powerful base models—it's constructing environments where AI can reliably learn from genuine success and failure.