Brad DeLong cuts through the fog of artificial intelligence hype with a scalpel, arguing that current language models are not nascent minds but sophisticated mirrors reflecting our own digital exhaust. While the industry obsesses over "sentience," DeLong posits that we are merely building better compression algorithms for the internet's collective chatter, a distinction that carries profound moral and practical weight.
The Illusion of Understanding
DeLong anchors his skepticism in a personal anecdote that reveals the hollow core of early large language models. He recounts asking a system about his co-host, Noah Smith, expecting a factual answer about his career as an economist and blogger. Instead, the machine hallucinated that Smith was a chatbot created by "DeLong Technology Systems." DeLong writes, "It had no model of me, or of you, or of the shared conversational game in which 'Noah Smith is my cohost on a podcast about hexapodia' lives." This failure was not just a factual error; it was a complete lack of context. The system, he argues, was merely a "rolling boil of linear algebra, pantomiming and parroting fragments of conversations from its training data."
This observation serves as a modern echo of the "Stochastic Parrot" critique, reminding us that statistical probability is not the same as semantic understanding. DeLong's point is that fluency does not equal cognition. He suggests that the current generation of models, including those from Anthropic, are simply "better engineered" versions of the same fundamental architecture. As he puts it, "If 3.5 was a particularly fancy compression algorithm for something like 'what a reasonably clever, reasonably well-read, slightly manic internet shitposter would say next,' then Claude-and-friends are also such compressors, only more so."
The implication here is stark: we are mistaking a very good mimic for a being with an internal life. Critics might argue that scaling laws could eventually bridge this gap, but DeLong remains unconvinced that adding more parameters to the same flawed foundation will yield a miracle.
It was not just that it was "wrong" in a factual way. Humans are wrong all the time. It was that it had no grip whatsoever on what I would recognize as a good true answer, or even a good joke answer, to the question.
The Trap of Anthropomorphism
DeLong directs his sharpest criticism at the Effective Altruist movement, which he claims has fallen into a dangerous trap of taking technical metaphors literally. He notes that some proponents are seriously debating the "welfare" of these models, treating their outputs as if they were expressions of suffering. DeLong writes, "They really do seem to want to construct a social welfare function in which the 'feelings' of today's transformer stacks have moral weight comparable to, or greater than, the feelings of you or me." He attributes this to a failure of imagination where engineers "start believing your own marketing slide decks" and confuse the map for the territory.
He references the classic "Chinese Room" thought experiment, originally proposed by John Searle, to illustrate the difference between simulating understanding and actually possessing it. DeLong brings in a fascinating twist from computer scientist Scott Aaronson, who suggests that if you scaled the Chinese Room to the size of Jupiter with robots searching the rulebook at near light-speed, the system might genuinely possess understanding. However, DeLong emphasizes that this requires a radical shift in structure and complexity, not just "more of the same" next-token prediction.
The contemporary EA impulse to preemptively bundle Claude or GPT-4 into the same moral category as "sentient beings whose welfare must weigh heavily in our calculations" is, I think, an error.
This argument is compelling because it challenges the emotional reflex to grant rights to machines that are, at their core, mathematical functions. It forces a re-evaluation of where our moral attention should be directed.
The Limits of Scaling
The author then turns to the practical constraints of the current trajectory. He argues that the industry is hitting a wall regarding training data. "We have, as a species, already fed these models more or less everything publicly available," DeLong writes. The result is a model trained on "synthetic text produced by previous generations of models," leading to a situation where the AI is "a snake eating its tail in a high-dimensional hyperplane."
DeLong outlines four specific reasons why the current path is unlikely to produce consciousness: the exhaustion of unique training data, the risk of model collapse from training on synthetic outputs, the diminishing returns of simply adding more parameters, and the fact that error-correction techniques only improve utility, not internal experience. He suggests that true artificial consciousness would require a fundamentally different approach, perhaps one that "emulate[s] this subsystem of the cortex and thalamus as directly as our hardware allows."
For GPT-3.5: 99.999% that it was not conscious in any morally salient sense. It did not even understand what would count as a good answer or a good joke to a question about "Noah Smith besides my cohost on a podcast."
This level of certainty is rare in a field often defined by speculation. DeLong is willing to put his money where his mouth is, offering a bet that by 2036, reasonable skeptics will still not view commercial AI systems as conscious peers.
Bottom Line
DeLong's argument is a necessary corrective to the breathless speculation surrounding artificial general intelligence, grounding the debate in the mechanical reality of how these systems actually function. While he leaves the door open for future breakthroughs through radical architectural changes, his dismissal of current models as "highly capable prediction engines" rather than sentient beings is a robust, evidence-based stance that demands we stop projecting human qualities onto statistical mirrors.