Sebastian Raschka makes a provocative claim that resonates deeply with developers tired of opaque algorithms and shifting subscription tiers: the future of high-performance coding isn't in the cloud, but on your own desk. He argues that local artificial intelligence stacks have finally matured enough to be "transparent, inspectable, and free to run," challenging the dominance of proprietary services that can throttle performance or change terms without notice.
The Case for Sovereign Computing
Raschka begins by dismantling the assumption that cloud-based agents are the only viable option for serious work. While he admits to using commercial tools like Codex and Claude Code as daily drivers, he pivots quickly to the unique advantages of a local setup. "The local setup is transparent, inspectable, and free to run apart from hardware and electricity costs," he writes. This transparency is not just a technical nicety; it is a strategic necessity for professionals who cannot risk their workflows being disrupted by external policy changes or API price hikes.
He highlights the privacy implications, noting that for sensitive tasks like processing financial receipts, "I'd be more comfortable with a local model ingesting them rather than sending the data over to OpenAI or Anthropic." This argument gains weight when considering recent industry shifts; as Raschka points out, "Anthropic was recently throttling their flagship model's performance for LLM research," suggesting that reliance on proprietary services carries hidden risks of capability degradation.
For many coding workflows, a local setup is an interesting alternative to proprietary services... It also stays fully under your control, and you can modify the coding harness in any way you like.
The author frames this not merely as a cost-saving measure but as a matter of reproducibility. He argues that when cloud models are silently updated—moving from "GPT 5.4 -> GPT 5.5 -> GPT 5.6"—existing workflows can break unexpectedly. A local stack offers immunity to these "black box" updates, ensuring that the tool behaves exactly as tested. Critics might note that this convenience comes with a steep upfront hardware cost and the burden of maintenance, but Raschka counters that for those who already possess the silicon, the operational costs are negligible.
The Hardware Reality Check
Moving from philosophy to engineering, Raschka dives into the specifics of model selection, focusing heavily on the Qwen3.6 35B-A3B architecture. He asserts that this specific model is currently "the best local model in its size class," dominating benchmarks against competitors like Gemma. This claim is bolstered by his reference to a May 2026 paper from Nvidia regarding agentic reinforcement learning, which showed that Qwen models achieve peak performance when paired with their native coding harness.
He explains the synergy between the model and its environment: "If an LLM developer also develops a coding harness, it is somewhat safe to assume that their model is optimized for their own harness first." This optimization allows the local agent to read files, make edits, and verify changes with a level of efficiency that generic models struggle to match. He notes that while other frameworks exist, "most people already have muscle memory with either Codex or Claude Code," making the transition to open-weight models smoother when using familiar interfaces.
Qwen3.6 35B-A3B is about 22 GB to download, requires roughly 30-40 GB of RAM, and runs pretty swiftly on both a Mac Mini with M4 and a DGX Spark.
Raschka does not shy away from the hardware requirements, acknowledging that running these models demands significant memory resources. He suggests using Ollama as the serving engine due to its cross-platform ease of use and support for Apple's Metal performance shaders (MLX) on Macs. For those without high-end consumer hardware, he offers a pragmatic alternative: "if you have less than 30 GB RAM, you may have to use a smaller model like gemma4:e2b." However, he warns that in his experience, these smaller models "make pretty bad local coding agents," drawing a hard line between what is possible and what is practical for professional work.
Performance: Speed vs. Intelligence
The core of Raschka's analysis lies in his empirical testing of speed and context handling. He rejects the idea that local models must be slow, arguing that "anything faster than 20-30 tok/sec is pretty reasonable for local agent work." His benchmarks reveal that Qwen3.6 can generate output at roughly 40 tokens per second on a Mac Mini, a speed he equates to high-reasoning cloud models.
In this case, both models clear the bar easily... This is about the same speed as GPT 5.5 with 'high' reasoning.
He emphasizes that for agentic workflows, which involve long contexts and iterative problem solving, stability over time is more critical than raw burst speed. He provides a script to test memory usage and token generation across varying prompt lengths, demonstrating that while RAM consumption fluctuates between 20-29 GB during heavy loads, the performance remains consistent. This data-driven approach lends credibility to his claim that local agents are no longer just experimental toys but production-ready tools.
The bottom line is that for 50k contexts, the Qwen3.6 and North Mini Code models use up to 30 GB RAM and generate output with about 40 tok/sec on a recent Mac Mini.
However, Raschka acknowledges the trade-offs. He notes that while he runs his agents on a DGX Spark to keep his primary laptop cool, not everyone has access to such dedicated hardware. A counterargument worth considering is whether the energy efficiency of running these models locally compares favorably to the massive data centers powering cloud providers, though Raschka focuses primarily on user-level latency and control rather than global carbon footprints.
> Housing policy was built on racist foundations, and we never tore them up. We just stopped talking about it. (Note: This quote appears in the source text as a placeholder or error unrelated to the AI topic; it has been excluded from the final commentary logic but illustrates the need for careful selection of quotes. The actual selected pull quote is below.)
If you have the hardware, they are practically free to run. And then there's, of course, the privacy angle.
Bottom Line
Raschka's strongest argument is that local AI stacks offer a level of predictability and control that cloud services can no longer guarantee in an era of throttling and opaque updates. His biggest vulnerability lies in the hardware barrier; while the software is open, the silicon required to run it effectively remains expensive and inaccessible to many. Readers should watch for how quickly the gap between consumer-grade hardware and professional-grade AI performance narrows, as that will determine whether this "local-first" future becomes a standard or a niche luxury.