In a field saturated with noisy data and ideological posturing, Freddie deBoer cuts through the fog to expose a statistical sleight of hand that has shaped education policy for a decade. His central claim is provocative yet rigorously supported: the most-cited evidence championing charter schools relies on a bespoke, non-standard metric invented specifically to make negligible effects appear meaningful. For busy leaders and policymakers, this is not just an academic quibble; it is a warning that billions in public funding and the futures of millions of students are being steered by a number that doesn't actually exist.
The Illusion of the "Day"
The article dismantles the 2023 Center for Research on Education Outcomes (CREDO) study, which has been treated by the media and political establishment as the definitive verdict on charter school efficacy. deBoer notes that while the study covered 1.8 million students across twenty-nine states, its conclusions hinge on a single, suspiciously convenient unit of measurement. "The report did not show its findings in terms of effect size differences between charter and public, which would be the field's standard way of sharing results," deBoer writes. Instead, the study reported that a typical charter student gained "sixteen additional 'days of learning' per year in reading and six in math."
This framing was a masterclass in marketing, transforming a statistical whisper into a roar. deBoer argues that this metric is not a discovery but a fabrication designed to obscure the reality of the data. "'Days of learning' is not merely an unusual measure of student performance; it's a boutique metric that CREDO invented themselves for their charter studies," he explains. By avoiding the standard academic practice of reporting effect sizes, the organization sidestepped the transparent comparison that would have revealed the true magnitude of their findings. The result is a narrative of success that collapses under scrutiny.
"When the usual metric doesn't say what you want it to, you just come up with a new one. And that, friends, is the pro-charter movement for you."
The core of deBoer's argument is that this metric is not only non-standard but fundamentally flawed. He points out that converting standard deviations into "days" assumes a linear, consistent rate of learning across all grade levels, a premise that educational research has long debunked. As he puts it, "A 'year of learning' is not a constant!" The growth a second grader achieves in a year is vastly different from the growth of a high school junior, yet CREDO applies a single conversion factor to a dataset spanning kindergarten through twelfth grade. This inconsistency renders the "days of learning" figure mathematically incoherent, yet it has become the holy writ for reformers.
The Reality of the Numbers
When deBoer strips away the invented metric and converts the findings back into standard effect sizes, the picture changes dramatically. The celebrated "sixteen days" in reading translates to an effect size of just 0.028, and the six days in math becomes 0.011. These are not just small; they are statistically insignificant in any practical policy context. "That's not a .28 in reading, it's a .028 in reading," deBoer emphasizes, highlighting the decimal shift that turns a headline-grabbing win into a null result.
The visual representation of this data is stark. deBoer describes how an effect size of 0.5 would move a student significantly across a distribution, but the CREDO numbers show almost no separation between charter and traditional public school outcomes. "With all the inevitable statistical noise... this is what we get - an effect size so small close to half of the studied charter population did worse than the average public school student," he writes. This finding challenges the entire economic model of school choice, which relies on the assumption that market competition drives quality. If the "market" produces no measurable gain, the massive reallocation of resources to charter management organizations lacks an evidentiary basis.
Critics might note that even small effect sizes can be meaningful if applied at scale, or that specific subgroups of students may benefit significantly even if the average does not. However, deBoer counters that the study's own methodology fails to account for selection bias, meaning the sample is not truly comparable. He references the "Politician's syllogism"—the idea that because something must be done, and this is something, therefore this must be done—to illustrate how policy is often driven by the need for action rather than evidence of success. The reliance on a metric that changes its formula based on external NAEP score fluctuations further undermines the study's reliability. In 2015, the conversion rate was different from 2019, which was different from 2023, making longitudinal comparisons impossible.
The Cost of a Flawed Metric
The stakes of this statistical obfuscation are high. deBoer reminds readers that these numbers have justified the dismantling of traditional public school systems and the restructuring of labor rights for teachers. "That's the revolutionary potential of charter schools, which we've staked billions of dollars, the rights and working conditions of millions of public school teachers, and the future of our educational system on," he argues. The use of a "jury-rigged metric" to validate a policy shift is not just a methodological error; it is a betrayal of the public trust.
The article also touches on the broader ecosystem of "edu-optimism," where the failure of initiatives like Khanmigo or the persistence of value-added modeling issues are ignored in favor of a singular, optimistic narrative. deBoer suggests that the media's uncritical acceptance of the "days of learning" metric reflects a deeper failure to understand or care about statistical rigor. "If you want to compare two different types of intervention, effect size enables you to do that too," he notes, contrasting the utility of standard metrics with the opacity of CREDO's approach. The refusal to use standard tools suggests an agenda that prioritizes persuasion over truth.
"Sixteen days sounds like three weeks of school. 0.02 standard deviations sounds just like what it means: that there isn't any meaningful difference."
This distinction is the crux of the piece: language shapes perception, and by choosing a unit that sounds substantial, CREDO successfully masked the triviality of its findings. The argument is bolstered by the fact that even the National Education Policy Center, a group not known for anti-reform bias, has identified eight major flaws in CREDO's methodology, with the metric being just one of them. Robert Slavin, a veteran researcher, is quoted asking the simple question: "Why days? Why not additional hours of learning, or minutes?" The answer, deBoer implies, is that the unit was chosen to deceive.
Bottom Line
Freddie deBoer's dissection of the CREDO study is a masterful exercise in data literacy, revealing how a single, invented metric can distort a national conversation and justify massive policy shifts. The strongest part of the argument is the demonstration that the "days of learning" figure is not a discovery but a rhetorical device designed to inflate negligible effects. Its biggest vulnerability lies in the fact that despite these flaws, the metric has achieved such cultural dominance that it is now the default language of the debate, making it difficult for standard statistical corrections to gain traction. Readers should watch for how this narrative persists even as the underlying data fails to support the claims, signaling a continued disconnect between educational research and policy reality.