Fuzzy Math Word Problems

Numbers never lie. Or do they? American humorist Evan Esar defined statistics as “the only science that enables different experts using the same figures to draw different conclusions.”

The last few months have provided ample opportunity for the education “experts” to duke it out over data. This summer, the National Center for Education Statistics (NCES) released two reports comparing student performance in a variety of schools, sparking both controversy and confusion.

The first study, released by NCES on July 14, compared the performance of public and private school students on the 2003 National Assessment of Educational Progress (NAEP). The study “statistically controlled” for both student characteristics (gender, race/ethnicity, disability status, and English language learner) and school factors (size, location, composition of the student body), and then adjusted test scores accordingly. Based on the adjusted “formula,” public school students performed as well or better than their private school peers.

Reg Weaver, head of the nation’s top teachers’ union, lauded the report as proof that traditional government schools were “doing an outstanding job.” But are public schools really on top of their game? Harvard professor Paul Peterson used a more accurate method of measuring student characteristics and found a different story. According to Dr. Peterson’s analysis, NAEP data showed a “consistent, statistically significant private school advantage.”

Got that? Just wait … there’s more. On August 22, NCES released a second study, this time comparing the performance of students in traditional public schools with their peers in public charter schools. Again, NCES researchers adjusted 2003 NAEP scores for student demographic characteristics. According to adjusted data, students in traditional public schools performed better than those in public charter schools.

Charter school opponents pounced on the results, suggesting the numbers raised a red flag for wide-scale charter school expansion.
Charter school supporters cited concerns with the report’s methodology (especially with regard to how poverty characteristics were measured) and the fact that the study failed to track student progress over time. Still others, like the Alliance for School Choice, highlighted the obvious variations among charter schools, pointing out that “some charter schools are five-star French restaurants, and others are ramshackle dives.”

Whose numbers are telling the truth? It’s hard to say. The fact is, across the country, good schools and bad schools abound, whatever “system” researchers choose to study. Fortunately, parents are smart enough to know the difference.

Even with conflicting data, here’s one thing we know for sure: when it comes to choosing a school, families – not researchers – are the real experts. That won’t ever change, no matter who’s running the numbers.

Lindalyn Kakadelis is the director of the North Carolina Education Alliance.