I have written a number of times about the NAEP exam, both here and on other blogs – specifically about how the NAEP is considered the “Gold Standard” of education achievement testing for a number of reasons.
The NAEP is also known as the Nation’s Report Card on Education reform. It is a key piece of evidence in the health and success of America’s public schools.
A couple of important points about the NAEP:
1. Over the last five years the NAEP has shown the Massachusetts traditional public schools are the highest performing schools in the United States – hands down. If you think of the NAEP as the olympics of public education, then in the last ten “events”, Massachusetts has competed against fifty other states and walked away with ten “gold” and “silver” medals – meaning we have come in either first or second (or tied) in every event including math and reading at the 4th and 8th grade level. Let’s hear it for Massachusetts public schools!!
2. Analysis of NAEP data shows that traditional public schools are out performing charter schools pretty much every where and in every category across the country – a statistical fact the Bush administration tried to cover up for nearly three years.
In fact the data from the NAEP was so bad for charter schools that the Massachusetts DOE sponsored their own “study” (guffaw!) comparing MCAS scores between charters and traditional public schools in Mass. Of course the state sponsored study claims to show that the charters are outperforming traditional publics – which is absolutely laughable given the way the MCAS is administered.
The NAEP exam is considered the gold standard because, unlike the MCAS, it is administered according to rigorous scientific standards of measuring and statistical sampling.
For example:
The MCAS exam routinely includes questions from previous years exams which allows teachers to ‘drill and kill’ the material in order to prepare kids and increase scores. Nothing unethical about that – in fact when kids diplomas are on the line it is imperative. But it is hardly scientific.
On the other hand, with the NAEP exam the questions are not available to the general public or even the teachers or the schools who are chosen to be part of the rigorous statistical sample.
Here is how rigorous the NAEP is. On Monday, the students in my class will take the NAEP. I am allowed in the room, but:
1. I am not allowed to proctor the exam. An outside proctor is brought in – and as the classroom teacher I am only there to assist the NAEP representative.
2. I am not allowed to look at the questions (at least that is what I have been told so far), never mind look at past test questions and use them to prep my students, and;
3. I have to sign a release – presumambly stating that I will protect the confidentiality of the exam and the questions.
This is the way a real scientific analysis of student achievement should be conducted. Not in some overly politicized, tootsie-pop owl fashion that is designed to cook the numbers to support the agenda of those who don’t believe in free and equal education for all.
At any rate, here is to my kids – I hope they ‘represent’ on Monday. Also on Monday I will share what observations I am allowed about the test process.
Good info that people ordinarily wouldn’t find in the media.
Recommended.
Massachusetts public schools had the highest score in the NAEP in math and ELA last year, you say? An unprecedented achievement in the US? Furthermore, it would rank as the third highest-achieving nation that takes it, were its scores ranked against other countries’?
<
p>
Must be something wrong with the test.
<
p>
/snark
<
p>
(Good luck to your students Monday — looking forward to hearing how it went. And remember, you can’t spell “standardized test” without f-u-n…oh wait, you can…)
Trying. To. Not. Respond. And. Watch. Jet Li. Movie.
<
p>
Can’t…..
<
p>
1. First, a point of agreement: you’re right NAEP is a good test.
<
p>
(Although what you describe is “rigorous” test security, not rigor of test itself. But anyway.)
<
p>
Question: If I post 6 non-Googlable questions, 3 from NAEP Grade 8 math and 3 from MCAS Grade 8 math, can you distinguish which is which?
<
p>
We could bet a mooninite or something.
<
p>
2. In 2005, just 13% of Boston’s black and Hispanic 8th graders were “proficient” on NAEP math.
<
p>
Meanwhile, in 2006, 14% of Boston’s black and Hispanic 8th graders were “proficient” or above on MCAS math.
<
p>
Don’t the NAEP and MCAS tell pretty much the same story here?
<
p>
Yet you despise MCAS and embrace NAEP.
<
p>
3. If it were easy to “teach to the MCAS,” as you claim, then why did so few Boston minority kids do well on MCAS?
<
p>
4. You didn’t link to anything. Is there a new study? I haven’t seen new NAEP numbers since 2005, and that was national data, and that data had all sorts of problems.
<
p>
I’m aware of no NAEP data that compare MA charters to MA district schools, or Boston charters to Boston district schools. The only stuff like that is MCAS, which show charters overall doing fantastically well (with some doing okay, and some doing terribly). But you think the only test which gives us a comparison is cooked. Convenient for you!
<
p>
5. Big picture:
<
p>
I don’t think anyone disputes there are two Massachusetts situations
<
p>
*Good outcomes for white and Asian students, mostly though not entirely in the suburbs
<
p>
and
<
p>
*Dreadful outcomes for black and Hispanic students, mostly though not entirely in the cities.
<
p>
NAEP confirms both points.
<
p>
That’s why Clinton, Edwards, Kerry, Obama et al all support charters public schools.
<
p>
<
p>
Turns out, Jet Li kicked everyone’s butt. What a shocker!
GGW came on giving his usual contribution to a discussion of education, regardless of topic. What a shocker! I got love for you GGW, and don’t mean to take away from the work you do, but no matter what the question or issue is here about education, we get the same answer from you.
<
p>
1 — I can’t distinguish which test question is which. What I can do is look at the MCAS (one-third of which is practice questions that will be used on later exams) and teach those exact questions. Some districts drill kids in those questions and voila their scores are up.
<
p>
2 — It’s no matter of despising or embracing anything. The value of the NAEP is that it is comparative. The MCAS is designed to…well, I’m not sure what. Compare this year’s eighth graders with last year’s eighth graders, and blame the teacher for having different studetns. Or something.
<
p>
3 — If Boston is like most cities, the racial minority fares poorly on MCAS for a host of reasons. Unstable home life (if they have permanent home) kind of leaps out as a reason. Lack of educational support at home — whether interested family, or even access to the Internet. How are charter schools (disproportionately represented in the lowest rung of MCAS performance, by the way — nearly 50% of the bottom 20) going to improve that?
<
p>
4 — The latest DOE study (released August 20th or so) showed charters with a miniscule lead in test scores, but a general statistical wash. The NAEP data I’ve heard in several education courses and in-services. I don’t have a link, but I have several photocopies.
<
p>
5 — Only a fool would say that educational outcomes are good for those two racial minorities. In that sense, Massachusetts is like most Western countries (take a look at NAEP data on French students of African – mainly Algerian – descent if you want to see an achievement gap. Yowsa. Check out Natives in Canada, or Aboriginals in Australia.)
<
p>
GGW, do you have a policy to respond to any slightly positive news about education with an automatic reply scoping out the subgroup that is having the hardest time? Did anyone say “all our problems are solved”?
<
p>
Every country, every jurisdiction has a lower-performing subgroup that is almost always a racial minority. Shockingly, one with a lower median outcome that has similarly lower Quality of Life indicators across the board. As I’ve said to you repeatedly, it does nobody a service to analyze the educational system through only one lens. Once again, offer a system that has worked in eliminating racial and social discrepencies in education. Charters in the US haven’t worked from any data I’ve seen (link me to your study, if you want). Any system, any country — I’d love to see it.
<
p>
I realize you have a political agenda to divert public money to private corporations in education. Apparently the inspiring examples of Bechtel and Blackwater have convinced you. Or the mess that was Edison private schools. I don’t know.
<
p>
And just because Edwards or Obama or whomever thinks something doesn’t mean I will. Even my man Richardson is cool with charters. But have you ever noticed that Democrats say that, but don’t actually do much about it? Could their position be less a policy statement and more a middle ground to get corporate donations and also support from teacher’s unions? Could it?
<
p>
And in other news, the crew of the Battlestar Galactica had to destory the Olympia Express because it looked like Cylons took over (got the season 1 dvd yesterday).
Thank you for the responses.
<
p>
1. I’m not sure how my post is “regardless of topic.”
<
p>
The guy made an argument about NAEP. Then he attached it to charter public schools.
<
p>
I responded about NAEP. Then I responded about charter public schools. Is that off-topic?
<
p>
Can I get an impartial ruling here? Bueller?
<
p>
2. As you say, it’s possible that key Dems support charters b/c they think it helps them with “business interests” — and their hearts are really anti-charter.
<
p>
Consider the alternative: that the opposite is true. Dem leaders’ hearts are with charters and the voters — and they hedge b/c they fear the teachers unions, the single most powerful interest group in Dem politics.
<
p>
They say things like “I believe it’s time to stop viewing [charter schools] as anomalies or threats to traditional public schools and begin seeing them as part of the future of public education.”
<
p>
3. Here is the August 2006 study you described above “as a wash.” Let the reader decide.
<
p>
4. My agenda is not to “divert public money to private corporations.” My agenda is to strengthen public education for all students — and to find a decent condo in JP or Watertown.
<
p>
Hey, you think charters are the Cylons. I think of the Achievement Gap as the Cylons. Let’s blast away!
Goldstein, I will hold out that there is a possibility that each one of the Democratic candidates support charter schools (but do nothing about them) because they genuinely believe that they offer improved outcomes.. But I find it unlikely. About the only thing all four tier 1 Democratic candidates have in common is a need to fund their campaigns. Shockingly, despite widely differing philosophies, they agree about charters! So either charter support is so amazing that it unifies all voices in the Dem party — indeed in both parties, or they help bank accounts. I’ll take the simpler, more direct explanation.
<
p>
As for the study (found here — I prefer studies to press releases from David Driscoll), I think that we have an amazing example of the difference between mean and median. On average, charters performed better than comparative districts in a barely statistically significant way. The median charter, however, performed bang on with the CSD (second conclusion requires looking at the data, not reading the press release).
<
p>
I believe your agenda is to “improve public education” inasmuch as President Bush’s agenda is to “bring peace to the Middle East”. I don’t disagree on the stated goal, but rather the method.
<
p>
And to drive the BG analogies into the ground, I think charters are one of those Cylons that looks all nice and friendly up until BLAMMO and you’re too late. Friend don’t let friends be friends with Cylons…or charters.
I just despise the fact that the state is using the MCAS data to prop up charter schools because they know the more rigorous data shows that the charter schools are getting beat by the traditional publics on the NAEP.
<
p>
As for Boston students – yes, you are correct. The NAEP and the MCAS do show an achievement gap between white and non-white students.
<
p>
But here is a HUGE difference. The NAEP exam shows that students in Boston (who are the lowest achieving students in Massachusetts) are outperforming the AVERAGE student in ALL of California – which is shocking to say the least.
<
p>
But keep in mind that California is the sixth largest economy on the planet. So those who try to argue that our kids are not competitive economically, etc., have to deal with that.
<
p>
Again, I don’t despise the MCAS at all. As a matter of fact, since starting in Boston six years ago I have come to embrace it as a necessary tool to motivate underachieving kids who otherwise try to get through school without doing any work.
<
p>
I didn’t get a chance to read the rest of your post – I will try to before 8PM and respond. But I am not missing the first Police reunion in 20+ years!! So it might have to wait until tomorrow!
<
p>
Thanks for responding though.
Someone (I think Goldstein) asked why the MCAS shows an achievement gap if teaching to the test (using past test questions) is so easy.
<
p>
I am paraphrasing here (and probably horribly) so let me clarify my original point by telling a story and explaining the Orwellian nature of teaching in Boston.
<
p>
In my third year of teaching in Boston (when I was still teaching general middle school science), the 8th grade math curriculum did not include anything about Measurement (one of the state math strands covered by the DOE).
<
p>
My principal and I decided that I should spend part of my class time prepping the kids on measurement – easy enough to do with a science curriculum.
<
p>
From January to April, I used past MCAS test questions that dealt with measurement (mostly area and volume) as the “Preview/Do-Now” at the beginning of class. And I drilled it into my students in a very traditional manner that is highly frowned upon by the higher ups in the BPS.
<
p>
Moral of the story: I managed to single-handedly increase the entire schools math MCAS scores by 6-7 points. Don’t get me wrong here – this doesn’t make me teacher of the year. But it does show a gigantic weakness in using the MCAS as though it is “ratio-level data” (the term used for the highest quality statistical and scientific data).
<
p>
If a single teacher can have such a profound impact on the results of a single school simply by covering past years exam questions and using drill-and-kill methods then the MCAS can hardly be considered ‘ratio-level’ data when trying to measure student achievement between charters and traditionals.
<
p>
A couple of big questions that no one has asked about the DOE charter school study yet: 1)did the charter schools have any prior knowledge about the study, and; 2) what kind of MCAS test prep are they doing?
<
p>
So now the second part of your question – if teaching to the test is so easy then why doesn’t every teacher in Boston simply do the same thing I did?
<
p>
Well, the answer is tragically simple. Most of the time, the administrators won’t allow us to “teach to the test” – and this is the God’s honest truth – even though doing so would save many kids who are otherwise failing the MCAS exam.
<
p>
Forget what you see on the tv show Boston Public – the real BPS is incredibly strict about enforcing curriculum and pacing on teachers – especially for ELA and math. In math for instance, teachers must adhere to a curriculum that is planned out for them almost to the minute. No joke. When it comes to enforcing curriculum and pacing the BPS makes Frederick Taylor look like a Luddite.
<
p>
Here is my prediction, based on my own experience. If teachers in Boston were allowed to teach to the MCAS exam, our students scores would increase dramatically.
<
p>
Finally, Goldstein – I will say one thing about the problem with charters vis-a-vis our vision of society as Democrats and democrats. Charter schools are based on the philosophy of segregation. They are contributing to the resegration of the Boston Public Schools based on the criteria of ‘readiness to learn’, which is to say that those families who are more engaged are putting their kids into charters and other schools – thus dumping the rest into a smaller and smaller number of schools.
<
p>
In other words, the children who are not supposed to get “left behind” by No Child Left Behind are getting left behind. And the teachers like me, who break their backs every day trying to help these kids, are being blamed for it and our collective bargaining rights are being dismantled as a result.
<
p>
At some point, this whole readiness to learn strategy is going to break down along racial lines – actually in some places that has already happened. (See the Harvard Civil Rights Project website). Then the only thing that is going to make all of this okay is overturning Brown v. Board. Believe me when I tell you that as an urban educator, I fully understand why some parents don’t want their children exposed to the violence, drugs, and gangs etc., But as Democrats, we cannot embrace solutions to these problems that take us back to Plessy v. Fergusson. It is absolutely horrific that the Democrats embrace anything that may set us back two centuries on the issue of ‘separate but equal’.
<
p>
Again, thanks for responding. I like the give and take. Hope you enjoyed Jackie Chan. The Police were great. Hope they come to Beantown when they go on tour.