SAT Score Report: The Latest Annual Education Non-News

(Most important of all, he has a Ph.D. and was a professor of history and politics for many years at the Mass. Maritime Academy before he became a state senator. Without any doubt the most qualified candidate for the position. - promoted by Bob Neer)

The most recent release of SAT scores were released today.

Governor Patrick used the opportunity to point out that our scores than those of the rest of the country. (My local paper then picked up the story today).

Here’s an excerpt:.

MALDEN – Governor Deval Patrick today announced that Massachusetts public high school students made greater gains on the mathematics SAT exam than any other state over the past decade, and on the 2010 exam once again outscored all other states with high participation rates. Additionally, African-American students in Massachusetts now lead their peers nationwide

The Governor’s statement includes some actual statistics from the report. However, nowhere in the DESE’s press release is the College Board’s disclaimer:

Media and others often rank states, districts and schools on the basis of SAT® scores despite repeated warnings that such rankings are invalid. The SAT is a strong indicator of trends in the college-bound population, but it should never be used alone for such comparisons because demographics and other nonschool factors can have a strong effect on scores.

In other words, rankings of schools and states do not mean anything. Why? For one thing, kids who take the SAT are self-selected. For another, we know the results of kids who took the SAT, but we have no way of knowing how representative they are of their state or school.

Perhaps more interesting and NEVER considered in popular accounts of test results is the reliability of the test itself. The SAT is highly reliable for a test, but like the polls we like to analyze here at BMG, it comes with a margin of error. Harvard Professor Daniel Koretz states the issue neatly in his book Measuring Up:

the margin of error on the SAT is not negligible–to have even a 5 percent chances of scoring more than 66 points above or below the true score is not trivial.

In other words, the 600 your kid scored on her SAT math could really be as low as a 544 or a 666. How do you think that margin of error relates to aggregate scores for groups? Not at all?

If there is a moral to be taken from the College Board’s yearly release of SAT scores, it’s that as a country, we have yet to start talking seriously about what test scores mean and don’t mean.

This post was originally published with Soapblox and contains additional formatting and metadata.
View archived version of this post


7 Comments . Leave a comment below.
  1. not everything that can be counted, counts.... quote my favorite quote from my favorite quote-meister, Einstein.

    I can understand Gov. Patrick wanting to showcase these stats. I'm wondering, though, what the trend lines are for poor and minority students in MA actually going to college and graduating. Anyone know? Looking at college for my high school senior, I know the spiraling costs make college seem increasingly inaccessible for middle class families here and elsewhere.

    Excellent point, Mark, about the disclaimers that never see the light of day. There's also this one from the American Evaluation Association:

    "The American Evaluation Association opposes the use of tests as the sole or primary criterion for making decisions with serious negative consequences for students, educators, and schools."

    Is there any other way to describe our national obsession with misusing standardized tests to judge and punish students, teachers and schools? Is there any other way to describe the way the high-stakes MCAS, No Child Left Behind and now Race to the Top have everyone running in circles trying to get those scores up by any means necessary?

    I understand why we are all so comfortable with standardized tests, which we've all faced throughout our school years, but I don't know why it's so hard for politicians and other humans to understand the way education (and other important pursuits) get corrupted when we place too much emphasis on these imperfect measures.

    David Berliner, Deb Meier, Diane Ravitch, Richard Rothstein and many others have written about how high-stakes testing in education demonstrates Campbell's Law, which says that "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

    We've seen this corrupting influence at work when schools under pressure to raise test scores game the system in many ways, whether it is outright cheating, pushing low-scoring students out of school, or discouraging them from enrolling in the first place, all to improve the test score bottom line.

    Last week's episode of This American Life gives us a chilling example of Campbell's Law at work in policing. Anyone happen to hear it on NPR? The last segment tells the story of NYC cop Adrian Schoolcraft, who tried to blow the whistle on NY's quota system, developed to make crime stats look like they were always improving (remind you of anything?).

    In reality, the quotas had cops writing people up for imagined offenses like drinking in public. They also downgraded serious crimes like rape, so that one serial rapist was free to commit rape after rape because his crimes were recorded as minor offenses (it's about looking safer, not being safer). When Schoolcraft tried to blow the whistle on the whole corrupt process, he was handcuffed and committed to a mental hospital.

    Unfortunately for his superiors, he had everything on tape, including when they wrestled him from his bed and cuffed him, tape you can hear on this remarkable and important piece of journalism.

    You can listen to the segment online. (This American Life, by the way, has done some of the best education reporting around, including a balanced and thoughtful piece of NY's so-called rubber rooms.)

    Lisa Guisbond Citizens for Public Schools http://www.citizensforpublicsc...

  2. What percentage of high school students takes SATs?

    I'm not sure I personally know anyone who didn't take it at least once, and most of the people I know took it a second time and also took one or more SAT II subject tests.  In my world, at least, it was nearly univeral as far as I could tell.

    • according to the Globe...

      in an article yesterday, "In the class of 2010, 75 percent of Massachusetts public school students participated in the SAT. Only Maine, which requires all students to take the SAT, had a higher participation rate, according to The College Board. Overall, 45,670 public school students in the state took the SAT."

      • Sounds like we're doing something right!

        Of course ME used to be part of MA so I wonder where that state got its commitment to educational excellence:)

        • We are definitely doing some things

          right. It's time our policy-makers start taking responsibility for the metrics we use to say so.  

      • You that reminds me,

        Bowdoin College in Maine does not require the SAT for applicants.  Bowdoin lets the students decide if their SAT score accurately represents their achievement, and about 19% of the kids who go to Bowdoin never submitted SAT scores.

        And really, given that high school GPA is a better predictor of success in college than SAT scores, Bowdoin's clearly making the right decision.

        • Bowdoin AND about 830 other U.S. colleges and universities

          It's a trend, running counter to the trend in K-12 to put more and more emphasis on test scores uber alles.

          FairTest has a list of SAT and ACT optional colleges here.

          Another great school, Bates, has been SAT-optional for decades, has studied the impact of the policy and concluded that it has increased the quality and diversity of its student body.


« Blue Mass Group Front Page

Add Your Comments

You must be logged in to post a comment.

Wed 29 Mar 1:07 PM