In an era where artificial intelligence is increasingly framed as a mirror to human cognition, Jacqueline Nesi offers a necessary corrective: the metaphor that equates our minds with machines is not just imprecise, it is actively dangerous. While the tech industry leans into the idea that consciousness can be engineered through better code, Nesi argues that this belief stems from a fundamental misunderstanding of how human brains evolved to handle abstraction. This is not a Luddite rant, but a cognitive critique suggesting that our reliance on the "brain as computer" analogy blinds us to what truly makes us human.
The Trap of the Rock
Nesi opens with a charming anecdote about her two-year-old son misinterpreting a Father's Day rock painted with the phrase "My dad rocks." The child, lacking the capacity for abstract thought, takes the statement literally. Nesi uses this to illustrate a profound biological divide. "At age two, children generally have not yet developed abstract thought," she writes, noting that this ability to see one thing as a symbol for another is a recent evolutionary development. This framing is effective because it grounds a high-level philosophical debate in a relatable, everyday failure of logic. It reminds the listener that the ability to understand metaphor is not a default setting, but a hard-won cognitive achievement.
She connects this to the brain's architecture, specifically the Default Mode Network, a system of neural connections that scientists believe evolved roughly 100,000 years ago to facilitate this very capacity for symbolic thought. "Abstract thinking is such an advanced skill," Nesi argues, "that it is one of the very things that makes us human." The piece suggests that when we strip away the biological reality of the brain and replace it with the clean, modular logic of a computer, we lose the context of how we actually experience the world.
"We are increasingly interconnected with our technologies... and this interconnectedness is only accelerating. As we enter this new era of intelligent machines... it's worth revisiting our metaphors."
The Metaphor That Became Reality
The core of Nesi's argument is that metaphors are not merely decorative language; they are the lens through which we perceive reality. She points out that we have long used metaphors to understand the intangible, from DNA as an "instruction manual" to time as "money." However, with the rise of AI, the metaphor has shifted from a tool of understanding to a source of confusion. "Our most fundamental metaphor for AI is baked right into the name: intelligence," Nesi writes. "We imagine systems made up of neural networks... and when things go wrong, hallucinate."
This is where the argument gains its critical edge. Nesi suggests that by using human-centric terms like "hallucinate" or "think," we are projecting human qualities onto systems that do not possess them. "The problem is when we start to see those abstractions as reality," she warns. "When we start to believe our machines are just like us because, after all, they are intelligent..." This framing forces the reader to question the language used in Silicon Valley daily. It challenges the assumption that because a machine can mimic human output, it shares the human internal state.
Nesi brings in science writer Michael Pollan to bolster this claim, citing his rejection of "computational functionalism." Pollan, as quoted by Nesi, notes that the consensus in Silicon Valley is that consciousness will emerge spontaneously in machines, but he calls this a "faulty metaphor." The distinction Nesi highlights is crucial: computers separate hardware and software, allowing code to run on different processors. Brains do not. "The brain's hardware is its software," she explains. "Our brains are shaped by our experiences, and we cannot simply extract one brain's software... and run it on another processor." This is a powerful, tangible distinction that cuts through the hype of digital immortality.
Critics might note that this view risks underestimating the complexity of future neural architectures, which may eventually mimic the plasticity of biological brains in ways we cannot yet predict. However, Nesi's point remains valid for the present: we are currently projecting human agency onto statistical models, and that projection has real-world consequences for how we regulate and trust these systems.
The Cost of Confusion
The piece concludes by returning to the danger of these blind spots. Because we cannot see the inner workings of an AI model, we fill the gap with our own metaphors, convincing ourselves we understand more than we do. "They can, by some miraculous evolution of our cognitive architecture, tell us what something is like," Nesi writes. "But they cannot tell us what that something, whether human or machine, really is." This is a sobering reminder that as we integrate AI into our lives, we must remain vigilant about the stories we tell ourselves about what these tools are.
The argument also subtly critiques the current rush to automate human expression. Nesi mentions the recent introduction of AI detection tools by platforms like Substack, noting the irony of using AI to detect AI. She declares her refusal to edit her writing to "sound more human" if a machine flags her work, stating, "then the machines really have won." This stance reinforces the article's central thesis: human expression, with all its quirks and stubbornness, is not a bug to be optimized, but the very feature that distinguishes us from the machine.
"The belief that AI can be conscious is based on an idea called computational functionalism that is essentially saying that the brain is a kind of computer... but it's a faulty metaphor and it's pretty easy to show that."
Bottom Line
Jacqueline Nesi's strongest contribution is her dismantling of the "brain as computer" metaphor, exposing it as a historical artifact that no longer serves us in the age of generative AI. The argument's biggest vulnerability is its reliance on current biological consensus, which may shift as neuroscience advances, but for now, it provides a vital guardrail against the anthropomorphizing of algorithms. Readers should watch for how this distinction influences future policy debates on AI safety and consciousness, where the definition of "mind" will determine the rights and risks we assign to our creations.