I know Bill is on vacation, so I will do my level best on this latest from the Globe, ‘though it appears the Herald jumped on this one, too. Cross-posted on who-cester, with a Worcester connection
The new study put out by the Boston Foundation gives a superb example of a principle from Statistics 101: correlation is not causation.
In other words, just because THIS happens and THAT happens, THIS did not necessary cause THAT.
Just because charter schools students on average spend more time in school and charter school students get better grades on tests (‘though they don’t, come to think of it), the time spent in school does not necessarily cause the better grades.
In order to prove that, you would have to control all other variables. You’d have to have exactly the same sort of kids–same grades, study habits, family involvement, neighborhoods, ethnicity, disabilities. You’d have to change ONLY the time spent in school: not, say, the amount of family involvement you have to have in a “choice” program.
This would, one would hope, be screamingly obvious to reporter at a metropolitan daily, not to mention the backer of charter school research, but, alas, it appears their educations did not extend quite so far.
<
p>The actual ‘principle’ at stake is more often stated thusly:
<
p>Correlation does not imply causation.
<
p>By the statement you make above, “correlation is not causation“, you’ve admitted your bias, however unconsciously, and have fallen into the apposite fallacy that correlation must therefore rule out causation.
<
p>
Tracy’s paraphrase of the principle should pass your test. She wrote “just because THIS happens and THAT happens, THIS did not necessar[ily] cause THAT,” which does NOT imply that correlation must rule out causation.
<
p>The only quibble I have with Tracy’s post is calling Scot Lehigh a “reporter,” when he’s an opinion writer, appearing exclusively on the oped page. His immovable opinions are that public schools have failed to educate poor urban kids, charters are superior, teachers unions are the major obstacle to improving educational outcomes. Scot seizes on two studies funded by the charter-promoting Boston Foundation, conducted by researchers at Harvard and MIT (so they couldn’t possibly have any flaws, right?) and disregards any and all other evidence of charter problems.
<
p>Tracy’s larger point is important: what these studies show is something much more limited and narrow and inconclusive than Scot says (over and over and over). Scot takes the results of two small studies and inflates their significance beyond all reason. He pronounces the debate over. It has been proved beyond all doubt that charters are superior and longer school days create better outcomes, therefore recalcitrant teachers unions should agree to work longer school days without compensation for the sake of the students.
<
p>He doesn’t even take the time to acknowledge and dismiss all the other evidence that, as a New York Times front page article put it, “for all their support and cultural cachet, the majority of the 5,000 or so charter schools nationwide appear to be no better, and in many cases worse, than local public schools when measured by achievement on standardized tests, according to experts citing years of research.”
<
p>xkcd
appears to match the downward spiral in circulation. Too many columnists, too few reporters with real analytic skills.
<
p>This particular column isn’t even the worst example. Last year they ran a “newspinion” piece claiming the Air France crash off Brazil was caused by using composite materials, and we are all in danger from these materials. This just days after the crash, when only a tiny fraction of debris had even been found, let alone analyzed.
<
p>The only evidence: speculation by anonymous sources. No questioning of the “sources”, no data, not even bad correlation/causation analysis. Nothing.
<
p>Sadly, it’s the state of media today. Either simply repeating what they’ve been told (or googled); screaming at each other on “talk” shows, or unfounded speculation presented as fact.
<
p>There was a time when the Globe, and others, actually had science and technology reporters, even good ones. People who could actually analyze numbers. Not anymore. Just functional innumerates.
<
p>This sums it up pretty clearly:
<
p>
<
p>At the risk of making my own bad correlation/causation argument, it appears to me that journalism departments are failing to imbue their graduates with the necessary technical, mathematical, scientific, and analytic skills to adequately report on life in the 21st Century.
I don’t know if this merits its own entry, but the Globe is being characteristically dishonest with today’s “follow-up” on the case in Central Falls, an opinion piece oddly couched in today’s Metro News section. The teachers in this corrupt curio of a city were publicly fired en masse because their students’ test scores did not meet a certain standard to much acclaim from the Globama faction on “reform”. The Globe reporter, Brian MacQuarrie, wrote a story on the recent settlement that led to a mass rehiring. To scan the article, this settlement was apparently a blow toward teachers and contract integrity, and a giant win for reformers who like to fire teachers. As he writes, the “teachers in this city conceded nearly everything”.
<
p>Two problems.
<
p>One is that, as so often happens, children interviewed about their education tend to disagree with the Obama approach to “reform”. Small wonder interviews with students are included on such a miserly basis — here is another Globe article where students at a school don’t think what the paper’s editors want them to think.
<
p>Two is that the article’s carefully crafted mood is a lie. The “fire everybody” approach was used when teachers refused to raise their workload by over 10% without any compensation. The Superintendent had no interest in paying for the additional work, so fired all the complainers. The conflict wasn’t even about test scores — it was about paying people for the work they do.
<
p>So reading the headline and several opening paragraphs of the article, you’d be forgiven for thinking that the teachers received a proper consequence for their unreasonable demands, and would be working more while making less. However, MacQuarrie stalls for eight paragraphs before admitting the truth. According to this disastrous settlement…teachers will be paid extra for extra work.
<
p>So the teachers got what they wanted. But because they were briefly fired (to Arne Duncan’s delight…imagine if our Sect’y of Health was so anti-doctor) before winning the stand-off, the teachers lost.
From the NY Times
<
p>”Central Falls High School is one of the state’s lowest-performing schools, with a graduation rate of 48 percent. Only 7 percent of students are proficient in mathematics by the 11th grade.”
<
p>http://www.nytimes.com/2010/05…
<
p>The school spends $15,005.00 per student and has 13 teacher per student. The resources are there for a good school. But the school is broken and should be shut down. Why are the kids even going to school there?
<
p>
Show me a school that services a comparable population and does better in real terms — not just standardized scores.
<
p>Do we declare a police station in a high-crime area to be “broken”, and shut it down so we can feel good about ourselves?
You can debate about the use of standardized scores at higher performing schools. But these kids just need basic skills and the school is failing at providing them. I’ve had staff who have attended these schools – they think the schools are a joke and drop out.
<
p>
<
p> A police station that doesn’t make any difference in the crime rate – should be shut down. Or a restaurant that doesn’t serve customers or an automotive manufacturing firm or bank that doesn’t make money.
<
p>
about the use of standardized test scores anywhere. Despite the claims, they don’t do a particularly good job of evaluating basic skills, either. You’re much better off having a child read aloud to evaluate reading, for example.
As an employer I can tell the difference, between a student from a failing school and a top school. The top high schools in Massachusetts are teaching at a college level, I’m not sure the failing schools are teaching anything at all.
<
p>In Central Falls “Only 7 percent of students are proficient in mathematics by the 11th grade.” What are these students going to do the rest of their lives, this is sad and I agree when their school board that something dramatic has to be done.
…because you’re doing a “on the ground” test. That isn’t what the MCAS (or the other tests) test.
Of course charters look better in these advocates’ studies. And the simple reason is that they get a self-selected student/family population to start with, and then can throw out imperfect students as they go along. They all have huge attrition rates. If students were randomly assigned among public and charter schools, and had to be kept all the way, we’d see immediately that the charter nostrums (young, under-qualified, low-paid teachers, de-unionization, merit pay, longer days, “no excuses,” uniforms, military discipline, packaged curricula, private funds, principals’ firing freedom, etc.) have nothing to do with their “success” (when they even show success). That’s why this idea is immediately rejected by charter advocates, who prefer to trot out these unscientific studies.
<
p>Yesterday, my Congressman, Mike Capuano held another “telephone community meeting”.
<
p>I asked his opinion of charter schools, which have now become a policy priority for government at all levels. He has a very pragmatic view, as usual. He said he’s open to alternative public educational approaches, BUT they all have to teach the same population of kids — and charters don’t. They can cherry-pick their students, and they teach far fewer special ed, English language and other extra-needs students. So if charters want to be a legitimate part of the public school community, they have to take on the same responsibilities — no screening, no dumping. He said that as they work now, the charters will lead to a two-tier educational system.
<
p>And that is the core problem.
<
p>If that became the focus of the charter debate, we could get at the real issue: what the charter movement bodes not just for providing education but for building a society. Public education in America has been a vehicle for equalization of opportunity, and for creating a body of citizens capable of peopling a democratic society.
<
p>Capuano is one of the few political officials who have not fallen for the charter fad, and understand what “public” really means — it’s not just about where the money comes from, it’s about what it does for the people as a whole. This takes a lot of courage these days, as both progressive and conservative wings embrace charters, each for their own reasons preferring an escape from, rather than the improvement of, the public realm. Capuano is not an ideologue: protecting the public realm is a very practical way of achieving social stability, with all its attendant economic and other benefits.
<
p>Deval Patrick’s recent legislation supporting more charters acknowledges the charters’ selectiveness, and requires efforts to admit and retain more at-risk students. But charter admissions will still depend on parental initiative to research schools, submit applications, sign parent-commitment agreements, etc. And the legislation still leaves a big loophole for charters to keep throwing out the kids that would tarnish their glowing statistics: “No part of this student retention plan requirement shall be construed as requiring a school to lower its academic or behavioral standards in order to retain students.”
<
p>Ultimately, if we want to compare schools, we’ll have to require the same assignment and retention policies. Only with comparable student populations can we truly test the difference in effectiveness of the different educational methods.
good in these studies is because these studies are–excuse the vulgarity–shit.
<
p>The bullshit process goes like this: 1) commission a no-peer-reviewed report to the policy-making meme of choice. 2) Do a press release press and get an article/interview 3)During the interview, spin the report down to your meme of choice 4) Count on the reporter not to read your report.
<
p>James Vaznis’s article doesn’t reflect the study, which doesn’t seem to say anything about charter schools being better because of increased instructional time.
<
p>Here’s Vaznis: “Students at Boston charter schools appear to have an academic edge over their peers at the city’s traditional schools because of the additional time they spend in school each year, according to a report being released today.”
<
p>Here’s the report: While the high-achieving charter schools have a longer school day… traditional schools may be able to creatively design their schedules and utilize staff so that the existing school schedule can incorporate more time for supporting struggling students and for teacher collaboration (p. 21 of the Executive summary, I think).
<
p>As research, this paper is a joke. The methodologies are survey and case study. No regression analysis of time on task and achievement. I have questions about the sampling and groups of samples, ex. is it a good idea to combine high school and elementary school principals into the same sample?
<
p>There’s not a single recommendation–increase school time, allow for flexible school staffing and structures, look for opportunities to engage teachers in decision-making–that hasn’t been written about extensively, though the jury may still be out. Actually, others have written more intelligently on these issues, but alas, the report really doesn’t review previous literature to put their work in context.
<
p>I have to go a select board meeting. I could go on.
right there, Mark, would have been an article worth reading!
when you have a chance.