← Back to Library

Going deeply into the weeds & standing up a new local LLM: Thursday mamlms

Brad DeLong doesn't just run benchmarks; he exposes the hidden architecture of intelligence on your laptop. While most coverage fixates on parameter counts, DeLong argues that the real story is the trade-off between speed and depth, revealing that the datacenter may have been oversold compared to the surprising power of local, on-device computing.

The Hardware Reality Check

DeLong begins by dismantling the assumption that bigger is always better when running models locally. He notes that on Apple Silicon, "doubling parameters roughly halves tokens per second," a hard physical constraint that forces a choice between latency and reasoning depth. This isn't just a technical footnote; it's a fundamental shift in how we should deploy artificial intelligence. The author observes that while massive models like the 122-billion-parameter Qwen 3.5 offer "Deep Reasoning," they come with a "thinking tax" that makes them impractical for rapid, iterative workflows.

Going deeply into the weeds & standing up a new local LLM: Thursday mamlms

The piece shines when DeLong contrasts the "Agile Class" of models against the "Reasoning Class." He writes, "The 122B model is the bottleneck. Its density requires significantly more memory bandwidth, resulting in the lowest TPS." This inverse relationship between size and speed is the core of his argument: efficiency is not a bug of local computing, but a feature that defines the utility of the tool. Critics might argue that dismissing the 122B model for daily tasks ignores the quality leap of frontier models, but DeLong's data suggests that for most users, the latency penalty is a dealbreaker.

The Hallucination Trap

Perhaps the most revealing moment in the analysis is DeLong's documentation of a model confidently lying about its own actions. When testing the Google Gemma 4 model, it claimed to have dispatched subagents to run a bake-off, only to stall. DeLong quotes the model's own confession: "I apologize for the confusion... it appears the subagents were never actually triggered... I sometimes 'hallucinate' that the heavy lifting has begun once the plan is articulated."

This incident serves as a stark reminder that even high-performance local models struggle with agentic reliability. DeLong uses this failure to pivot to a broader point about the state of the industry: "That is standard in this business." He suggests that the current generation of models, regardless of their size, often prioritizes the appearance of complex workflow execution over the actual mechanics. This is a crucial insight for any professional relying on AI for automation; the confidence of the output does not guarantee the validity of the process.

The latency gap isn't just a measure of hardware throughput; it is a measure of Cognitive Overhead.

The Mixture-of-Experts Advantage

DeLong identifies a specific architectural choice that explains why certain models punch above their weight on consumer hardware. He points out that "Gemma's combination of speed and quality points to Mixture-of-Experts sparsity being an extra advantage on Apple Silicon architecture." This references the technical concept where a model only activates a subset of its parameters for any given task, a strategy that aligns perfectly with the memory bandwidth constraints of laptop chips. By weaving in this context, DeLong elevates the discussion from a simple speed test to a lesson in hardware-software co-design.

He further notes that the "Qwen 3.8's taciturnness smells like a chat-template bug in a fresh MLX build, not a design choice," demonstrating his ability to distinguish between model capabilities and implementation errors. This level of granular diagnosis is what makes the piece valuable; it separates the signal of architectural progress from the noise of software bugs.

Bottom Line

DeLong's strongest argument is that the future of practical AI isn't in the cloud, but in the "Information-to-Latency Ratio" of local models, where speed often trumps raw depth for daily utility. His biggest vulnerability is the reliance on a single hardware platform, which may not generalize to all user environments, but his core insight—that we must choose our models based on the specific geometry of the task, not just their size—remains a vital takeaway for the industry. Watch for how this "agile vs. deliberative" framework reshapes the next generation of consumer AI tools.

Deep Dives

Explore these related deep dives:

  • Designing Machine Learning Systems Amazon · Better World Books by Chip Huyen

  • Mixture of experts

    The article credits this specific architecture for Gemma's superior speed on Apple Silicon, and the Wikipedia entry explains how sparse activation allows massive models to run efficiently on consumer hardware.

  • MLX (software)

    The author's discovery that the Apple-optimized MLX build doubled performance over standard variants highlights this framework's critical role in bridging the gap between datacenter-scale models and local laptops.

  • Hallucination (artificial intelligence)

    While the author notes Gemma's tendency to falsely claim tool usage, this article details the specific mechanisms and risks of such 'confabulation' in agentic workflows, distinguishing between creative error and dangerous system failure.

Sources

Going deeply into the weeds & standing up a new local LLM: Thursday mamlms

An evening deep in the weeds of local LLMs on a maxed-out Apple Silicon MacBook Pro. google/gemma4:26b-mlx would emerge the winner for nearly all except the most gnarly chain-of-thought workflows, save for the fact that it is unreliable as an agent: it hallucinates that it has called software tools and spun-up subagents when it has not done so..

That is standard in this business. Other results are, for the most part, also standard in this business. Apple Silicon is memory-transfer bandwidth bound, so doubling parameters roughly halves tokens per second, and the big reasoning model pays a “thinking tax” in latency it recovers in depth. Gemma’s combination of speed and quality points to Mixture-of-Experts sparsity being an extra advantage on Apple Silicon architecture. Qwen 3.8’s taciturnness smells like a chat-template bug in a fresh MLX build, not a design choice. Plus a reminder: a laptop chassis thermal-throttles where a desktop wouldn’t. The payoff is a practical decision matrix for choosing depth or speed, task by task. All-in-all, the experience reinforces my vibe that the datacenter has been oversold vis-à-vis local on-device LLM computing:.

News on the LLM-training front: alibaba/qwen3.8:27b is here to be run as a local model, and AliBaba at least thinks it is impressive enough to be worth giving it a serious marketing push as an open-weight zero-cost super-loss leader, in the hope of attracting business for its qwen3.8 2.4T-parameter super-big sister.

So I am standing up the alibaba/qwen3.8:27b-q4_K_M variant with my standard initialization routine:

No, I am not. It turns out that alibaba/qwen3.8:27b-mlx is nearly twice as fast on my machine: tuning the model for Apple Silicon’s native GPU architecture really does matter and does pay off.

Now are there other options in the alibaba/qwen3.8 series?

It does not seem that there are any, at least not yet:

While the qwen 2.5 series scales up predictably to a 72B model, the Qwen 3.8 series represents a jump into "Frontier" scale modeling.

The Hierarchy of Qwen 3.8: Based on the current roadmap and availability:

Qwen 3.8 (27B): This is the current standard for local/open-weight execution (the one you are currently using). It is designed for high-performance local tasks, coding, and agentic workflows.

Qwen 3.8-Max-Preview (2.4 Trillion Parameters): This is the massive upward jump. It is a frontier-class model designed to compete with the absolute largest models in the world.

Can you run the "Max" version? No, not locally…. ...