As an achievement test, the purpose of MCAS is to assess students? skills and knowledge, which are reflected in the Massachusetts Curriculum Frameworks. Each question is aligned with at least one standard. Test questions are written by actual teachers, then vetted by psychometricians and the Bias Review Committee addressing issues of bias and diversity.
Remember the Sixties line: ?What if they threw a war a nobody came??? To a certain extent, the same is true for a standardized test. No one, let alone the standards-bearers, would administered a standardized test if everyone were to receive an A on it. As paradoxical as it sounds, most standardized tests, the SAT, AP tests, and MCAS tests, are norm-referenced. They are designed to guarantee people receive different scores and that these scores are distributed along a curve. Standardized tests spread scores along a curve. I don?t know the shape of the curve.. On the SAT test, the curve is a normal, or bell-shaped curve. Think about it: the SAT test, which is a requirement for admission to most colleges, is intended to sort students along a pre-determined curve
Although the DOE denies the existence of a curve on the MCAS test, it is a normed test in that it references back to a normative group. According to the DOE, ?The standard scoring system was developed in August of 1998 when 200 panelists were asked to assess the students’ tests and categorize the resultant scores into one of four categories: Failing, Needs Improvement, Proficient, or Advanced.? Every year the MCAS test is referenced back to this original scoring. If the scores on, say, the 2005 test vary from this norm, the cutoffs for failing, needs improvement, proficient, and advanced are adjusted. In spite of new questions every year, the MCAS test remains the same.
Of MCAS and curves, the DOE says,
The panelists [the graders of the original questions], who were mainly educators, were not given quotas of the number of tests to be placed in each category. This allowed for the achievement levels to be determined in the absence of any type of relative scale. The four categories are absolute and therefore allow for each student to score above the failing category. In other words, there is no bell curve, thereby allowing every student who meets the standard to pass the exam.
Aside from statistical considerations, there are certain question qualities that can affect a standardized test. Producing a spread or distribution of scores is a requirement for the MCAS test. For our purposes, test questions can be divided up into easy, medium, and hard difficulty. A high percentage of kids get easy questions right; a small percentage of kids get hard questions right; and about half the kids get medium questions right. If you don’t have a lot of medium difficulty questions, you’ll get a lot of kids at the bottom and a few at the top. In other words, you get a sort of pass-fail test. Test makers have to include a lot of questions of medium difficulty to make sure there are scores across the continuum. I know this is kind of complicated if you’re unfamiliar with statistics, but testing is a technical business.
Intelligence Bias. It’s more difficult to eradicate questions biased toward smarter kids. Psychometricians have access to demographic information, but not IQ scores so they can’t filter these questions out. What’s wrong with smarter kids getting more questions right? MCAS is supposed to be measure achievement, learned skills and knowledge, not intelligence. MCAS is supposed to be holding kids accountable for what they’ve learned, not whether or not they’re stupid, but my guess is that some questions that’s exactly what it’s doing.