The White House is pushing a change where states would “opt into” some common standards and assessments. This would require some MCAS updates no matter what.
So there’s a policy window to open up a few other questions.
My proposal is for DOE (oops: now DESE) to open up 3 different working groups of “People with ideas on MCAS.” The 3 groups:
a. Big Change.
One working group are people who want really big, politically significant changes to the test itself. Like Sab. It would be understood that the bar for change would be high, b/c it’s not like there’s a lot of political capital to make unpopular changes.
b. Graduation.
Another group would look at data on the MCAS graduation requirement. There are various “alternatives” and I don’t think we’ve seen good study on what really happens there.
c. Technical Stuff
The third group would be people like me who think, on balance, MCAS is quite useful at setting a low bar (ie, the pros outweigh the cons). So we’re not trying to blow up the MCAS.
BUT there are a bunch of TECHNICAL issues that would NOT be politically tough to fix. Fixing these issues would help teachers better RESPOND to the data. This is stuff where I can imagine some educators who are often at odds (on things like charters) could work together.
I remember a post where Lightiris described some technical issues with the reading section of the test. Stuff like that.
* * *
I’ll give you one example of a technical problem that needs fixing, and where I don’t see many political problems: the 3rd grade math MCAS.
MCAS is given in Grades 3 to 8, then again in 10. So Grade 3 is the first MCAS.
The key thing that this Grade 3 MCAS SHOULD identify is kids who can’t add, subtract, multiply, and divide. This is a (sadly) big chunk of kids in Springfield, Lowell, Lawrence, Boston, et al.
Instead, the current Grade 3 MCAS is way too ambitious. It over-complicates the job. For example:
The number of students at Park Hill School who were absent each day last week is shown in the tally chart below. (The tally chart has Roman numerals, showing Monday: 12, Tuesday: 5, etc). How many more students were absent on Friday than were absent on Wednesday?
So if a kid gets it wrong, what do you do as a teacher? What exactly did he misunderstand?
Maybe he didn’t understand the words in question.
Maybe he’s confused by tally charts.
Maybe he doesn’t know Roman numerals.
Maybe he can’t subtract (ie, if you simply showed him 15 minus 7, he’d get that wrong, too).
The data makes it hard for 4th grade teachers (and school support systems) to intervene effectively.
Look, I know the Grade 3 and 4 math standards and tests cold. Our charter school sends 25 tutors into 2 different Boston district schools 3x/week for 3 hours/pop. I think I could quote much of the standards document.
The poorly designed early grade MCAS makes tutoring so much harder. You’re working with 1 kid. She cannot add yet. Of course what you want to do is focus on addition til she gets it. But because MCAS covers more math topics than is necessary for the early grades, there’s acute pressure to “move on.”
Now imagine instead you’re the teacher with 20 kids in September, not a tutor with 1 kid. You have this Grade 3 MCAS data. You think “Wow, almost none of my kids got any questions right from last May.” But that data doesn’t really do much to help you understand specifically what kids need.
* * *
An analogy:
Imagine if you wanted to evaluate the basketball skills of 3rd graders. First, you want to know if the kid can run (likely, but he may be disabled). Second, can he dribble. Third, can he make a layup.
But the test was whether a kid could do all 3. Someone watched the kid run, try dribble the length of the floor, and try to make a layup. YOU could not actually WATCH the test.
So all you got from the test was either YES (he did all 3) or NO.
Well what does the NO tell you? Maybe he can’t run or runs in the wrong direction. Maybe he can’t dribble. Maybe he can’t shoot a layup. You have no idea what to work on. Same with Grade 3 MCAS.
* * *
This is fixable. Usually, there’s a one-time cost in changing any test. You can’t do apples-to-apples comparisons of 2011 (new test) to 2010, 2009, etc (old test).
But here you could just add a dozen simple questions to MCAS. It would take the average 3rd grader just a few extra minutes on MCAS day. 4+7, 14+38, 173+399, 4*14, 22-19, etc.
You could provide 2 scores — the score on the existing test, and the score on the extra 12 “just the basics” part.
For kids who do fine on the new Grade 3 “basics” math test, great. A teacher could focus on the more complex questions they got wrong.
For kids who failed the Grade 3 “basics” MCAS, though they need concrete intervention. (I think the average need is 100 hour dosage of tutoring). For them, put the same basics questions on a Grade 4 test. You could find if the intervention worked.
I only have experience with the middle school and high school English exams.
<
p>As far as the 10th grade English exam goes, it’s not a difficult test. Students write a lengthy composition based on a literature prompt. The composition portion is perfectly reasonable, asking the student to write an organized, persuasive essay based on a well articulated theme. Students who are cognitively intact or who have mild to moderate special needs are able to successfully write such an essay. The length between long composition assessments, however, is not productive. Students write a long comp in 7th grade and in 10th grade. I can tell you as one who teaches sophomores that the length of the 10th grade composition comes as a bit of a surprise to many students. I think it would be wise to ask students to write a long comp of increasing complexity each year, with their 10th grade effort a sort of capstone of skill. Of course, the fact that their SAT composition must be written in 25 minutes sort of throws all that out the window….lol.
<
p>The reading portion is also reasonable. The passages are taken from a variety of genres at Flesch-Kincaid readability levels generally +/-1 to 1.5 years on either side of 10th grade. The multiple choice questions are straightforward–4 to choose from, 2 clearly wrong, 1 attractive wrong, and 1 correct. Not rocket science. Students who have not been taught to close read or annotate can get tricked by the attractive wrongs, but, overall, these questions usually pose the least difficulty for most students.
<
p>Now the ORQs, or open response, are the part of this assessment I find most problematic. They have managed to reduce reading a passage and answering a question in a paragraph form to a one-size fits all formula. If you don’t “know” the MCAS formula, you won’t get 3 or 4 points for your answer–even though your answer may be both reasonable and accurate. Indeed, a reasonable and accurate answer can even earn a 0. Why? Ya didn’t follow the freakin’ formula! This is the portion of the 10th grade exam that is freakishly problematic. High school English teachers (as well as middle school teachers) are chasing the Golden Chalice of the Correct Open Response Formula in order to eke out those extra points because it’s all about the points, remember. (Don’t just quote, paraphrase, comment, and quote. Quotes alone can get you 0 points….) This is the best we can do? Good grief.
<
p>English is the only skill-based content area, so it makes sense that the assessment in English should look a lot different from the other areas; however, you’d never really know this if you looked at their ORQ scoring standards. Candidly, I think the ORQ portions of the exam should be halved and replaced with ten-percent summaries of non-fiction text, but that’s just me.
<
p>Now, all this changes if you are an ELL. Unlike the math test, you have no accommodations for your English test. That means if you are of limited English proficiency, you’re on your own. Because I’m an English teacher, my district requires that I be MELA-O certified in order to properly assess the progress of my second language students and to ensure that they continue to receive the services they are entitled to. But here’s the problem: if you land in a Massachusetts high school from another country with little or no English, you are toast–unless your family is of extraordinary means. The probability that you can pass your English MCAS landing here after the age 14 is low indeed. That means, then, that you will receive a certificate of attendance instead of a diploma–no matter how innately intelligent you are. No high school diploma for you. Recently, we’ve found that several of the private colleges have discovered the ELL/MCAS/certificate black hole that a lot of talented students fall into and have begun treating the certificate of attendance (due to an ELL language issue) like a diploma. IOW, they’re ignoring it. So what’s the point? Good grief.
…is that while I agree the 10th grade exam is reasonable, the elementary level exams almost seem more difficult relative at least to where I personally would expect a lot of students to be.
By grade 3 each student should have acquired, through a targeted literacy program in their elementary schools, grade level reading proficiency. Many districts do not have targeted literacy programs, so their test scores reveal that. In the districts that have them, though, the MCAS results are generally excellent–as they should be.
I recall seeing an 8th grade social studies MCAS many years ago (I think I was substitute teaching) and it had an essay question about the impact of McCullough v Maryland. Now, I am a lawyer and a history undergrad major who wrote a thesis on the Jacksonian era, so I would love to live in a world in which every eighth grader could discuss this decision, its context and its effects. And I absolutely think it should be a part of classroom instruction. But to include this on a statewide test where the primary endpoint is whether students are learning basic skills…. well, it just struck me as overkill.
MCAS has always been, and shall always remain, a bad idea.
<
p>One does not, and cannot, improve upon, touch up, or reform a program intended since its inception to disembowel public education. The destruction of public education was always its aim.
Fluoride in the water supply.
At least CARE understood what MCAS was all about:
<
p>http://www.fairtest.org/files/…
<
p>As did Chuck Kollars:
<
p>http://www.ckollars.org/mcas.html
I opposed MCAS at the beginning, but in trying to develop an progressive educational view, a view that accomodates what the people (it’s not just policy-makers) want, I’ve come to accept it.
<
p>I also don’t mind teaching to the English Language Arts (ELA) portion of Grade 10 MCAS because these skills can be taught using regular materials as well as old MCAS questions. I also use old MCAS questions to teach skills and strategies.
<
p>I have little hope of the MCAS test being a better test, however. As James Popham continually points out, test questions sample a pathetically small portion of knowledge on a skill. I’ve never counted context vocabulary questions, for example, but I can tell you there are very few and they are not always well-written. Popham also points out that frameworks are always too broad, which is why these tests fail to provide feedback.
<
p>Passage-based, open-response questions (ORQ’s), for example, are supposed to address reading and writing. Yet these are two discrete skills. Consider the framework for one of the 4 ORQ’s:
<
p>How can you tell whether the student failed to identiy and apply knowledge about non-fiction or just failed to convey it completely? Sure, one could guess if a kid can’t write it, he doesn’t know it, but it’s unclear.
<
p>Success on a ORQ is measured with 4 as a perfect score and 1 credit for trying. Zeroes are non-responsive. The 4 point scale was intentionally created to keep a mass of kids from ending up in the middle as a 5 point scale would do. (My guess is that a 5 point scale would produce a normative curve, something the test makers want to avoid (everyone is supposed to eventually be above average)). How do you read the scores of a kid who received 2’s on two ORQ’s questions and 3’s on the two other ORQ’s?
<
p>At least the ELA MCAS doesn’t necessarily lead to dumbing down teaching. Our freshman biology teacher says that the test has turned bio into a lot of rote learning. I predict history will be worse.
<
p>I don’t have a lot of faith in being able to improve MCAS tests. There are too many, too broad, frameworks, and an increased number of questions would be necessary to really get an accurate picture of student skills and knowledge or the quality of curriculum and teaching in a school.
<
p>By all means, make some small adjustments, but let’s not fool ourselves into thinking that MCAS can give us anything but a very fuzzy, incomplete picture of what’s happening in school. One improvement I would make: get rid of the NCLB requirement and test only in grades 4, 7 and 10.
so I agree that cutting back the NCLB requirement from every year in grade 3 through 8 to just three grades (as we used to do in MA) would be a great step.
<
p>Obama and Duncan say they are prioritizing the reauthorization of NCLB, so the time to weigh in with Congress on issues like this is now.
<
p>For information on the work being done by FairTest and the Forum on Educational Accountability to push for rational changes like this, see here.
…though I prefer open-response to multiple choice. You’re correct about writing being a discreet skill. I struggled for a while in AP US History, not because I didn’t know my facts, but because my teacher wasn’t a big fan of my writing.
I think there continues to be more concern and debate on this issue than meets the eye (or gets reported by the Globe). I get calls and emails from current and retired teachers and parents often. Among teachers and many parents dissatisfaction remains high. It’s just that politically there seems to have been virtually no space to discuss or move forward with sensible changes.
<
p>For example, I was a proud and enthusiastic member of the MCAS subcommittee of the governor’s Readiness Project. My colleagues on the subcommittee brought a range of expertise and perspectives to the project. All were thoughtful, experienced, good folks.
<
p>We collected and considered a large body of research and came up with a good consensus report that said there was an urgent need for change. A number of us on the subcommittee, however, felt that the final Readiness document ignored the spirit and letter of our recommendations, missing an opportunity to make needed changes.
<
p>To quote from the subcommittee report, which is still available here
<
p>
<
p>We recommended that the current system be evolved to include:
<
p>
<
p>Carl Sciortino’s bill is one good effort to address the shortcomings of the current system and begin the development of a better one, or at least initiate a substantive debate in the legislature, something that has never really happened. Sabutai’s proposal looks to be in a similar spirit.
<
p>In sum, there is no dearth of good ideas or even a lack of agreement on the need for something better. There’s just a lack of will on the part of our political leaders to really tackle this critical issue.
There are a number of problems with MCAS and more specifically the graduation requirement.
<
p>1. MCAS data is essentially autopsy data. Teachers don’t get the results of the spring test until the next fall. By then it’s too late to do anything about it. Good schools are using their own interim assessments during the course of the year and making instructional adjustments during the year. If you wait until the tests results come, it’s too late.
<
p>2. It narrows the curriculum. What gets tested gets taught, especially in those schools who have poor test scores. Schools with good demographics have broad rich curriculum experiences for students. Schools with poor demographics have a drill and kill curriculum emphasizing reading, writing and math only.
<
p>3. The 8th grade and 10th grade tests are essentially the same test. That’s why you see 23% of the students statewide failing the 8th grade Math test, but only 8% in the 10th grade test.
<
p>4. Let’s look at the 10th grade failure rates in Math. The Math test is a little harder than the ELA test. By race, White 3%, African American 17%, Hispanic 19%. Other factors, low income 16%, Limited English Proficient 37%, Students with Disabilities 28%. So basically if you are White, not disabled, not low income and your first language is English you get a diploma.
<
p>5. The MCAS is a significantly harder test than other state tests. Thus the graduation requirements are more stringent here than in any other state. The ability of Massachusetts schools to make AYP (acceptable yearly progress)under NCLB is much more difficult here than elsewhere in the country. This, despite the fact that Massachusetts consistently scores number 1 in almost every area of the nations report card.
<
p>Recommendation: Keep MCAS, but significantly change the graduation requirement to include multiple measures of student achievement. Using the Rhode Island model, statewide testing counts for a percentage of the graduation requirement, along with local assessments, student grades, and a locally governed student portfolio or performance assessment. Essentially each community has a local graduation board which requires students to show what they have done in high school that should result in a diploma. Let’s get back local control.
I was going to give you a 6 rating, Jim, but that seemed too self-aggrandizing, given your title.
<
p>But reducing the weight of the test in graduation decisions and incorporating multiple measures is part of the idea behind Carl Sciortino’s more recent MCAS bill (H.3660) and was something we discussed and advocated in the MCAS subcommittee of the Readiness Project. A link to a summary of H. 3660 can be found here at the Citizens for Public Schools web site, under “Solution.” We on the MCAS subcommittee were kind of guided away from advocating “reducing the weight”or at least using that language, for some reason I don’t really understand, but I know I brought up the Rhode Island model as something to consider.
It should be adopted because it will tell teachers which kids can add/subtract/multiply one/two/three digit numbers.
And that’s useful info.
educational reform. Period. Without MCAS, the state cannot hold schools “accountable.” Curriculum frameworks would have limited enforcibility. It would be impossible for charter proponents like the Globe editors to compare charter schools to public schools. Editorialists and politicians would have nothing to bemoan.
<
p>The state would have no control over education in municipalities!
<
p>How could this be good for our children?!
..and do worse on MCAS and there is a shortage of about 500 beds for homeless 18 year old high school students. See: Homeless in High School
its an ugly competition between states
Seriously. And part time jobs doing the upkeep, cooking, etc. and paid internships.
<
p>It takes a village to raise a child – and our villages, too many of them, are broken and have spit these kids out like water melon seeds.
<
p>Just think – 800 legal orphans a year…just from the legal system, not the throw away kids.
or at least the extent to which MCAS has become so pervasive. I’m curious if there are any teachers out there who defend the status quo. I’m fairly certain I’ve never met any teachers who think so.
<
p>The governor and legislature would do well to pay attention to that fact. Perhaps it’s time to begin listening to the voices of those who spend their time closest to our children.
or is it the teachers union that is opposed.
<
p>
<
p>Stock Brokers are also probably universally opposed to the SEC.
<
p>
And what it is isn’t particularly helpful, valuable, or meaningful in the any pedagogical sense. It’s window dressing, for the most part, and not very good window dressing at that.
<
p>No, she’s right. The vast majority of teachers oppose MCAS in its current configuration.
does not reflect what teachers think?
reflects a kneejerk reaction to the MTA. It depends on what you mean by being against MCAS. The MTA supports mending, not ending, MCAS.
<
p>The MTA supported reform of the assessment systemwhen Representative Carl Sciortino and Senator Pam Resor brought forward The MCAS Reform Bill – H561.
<
p>The legislation, according to the MTA,
<
p>Some people refuse to believe that the MTA is interested in anything other than preserving teachers salaries and benefits. We’re all teachers. Most of us care about our students. The MTA does too. Unfortunately, our union is written off as exclusively self-interested by too many policy-makers.
<
p>
My feelings is teachers are professionals. But the MTA does everything in it’s power to make teachers day laborers and will try and break the MCAS system if they can.
<
p>
As someone who went through a suburban 1970’s high school and now has two kids in a suburban high school. I can see a huge improvement in the quality of instruction. I think the MCAS test is largely responsible for this.
<
p>The MCAS test sets a standard for graduation. Your high school diploma means something. Your diploma is not a certificate of attendance.
<
p>
<
p> But I hear rumbling about what MCAS is doing to vocational schools, i.e. making them academic and not vocational, that’s wrong and a change should be made.
<
p>
It would be interesting to see polling. Have you see any?
<
p>Most of the national polling I’ve seen of teachers has been along these lines. The only handy link I found was back in 2003, though.
<
p>When offered:
<
p>Standardized tests
<
p>1. Much more harm than good
2. Necessary evil; schools need standardized ass’mt
3. Necessary, valuable, reliable
<
p>Most teachers pick #2.
commend MCAS, not all math teachers. I can’t recall a teacher in another subject praising MCAS. Our biology teacher panned the test recently, and she teaches both freshman and AP Biology.
<
p>FWIW, Teachers, in my experience, are not necessarily better informed about educational issues than non-teachers of the same socioeconomic demographic.
Here’s some information on teachers’ views of standardized testing from a 2009 Public Agenda survey:
<
p>Ninety percent of teachers view the current overemphasis on standardized testing as detrimental to education. Overall, 58% of respondents said too much testing is a major drawback, while 32% said it was a minor drawback.
<
p>They divided respondents into three categories: contented, disheartened and idealists.
<
p>”Q6. Based on your personal experience, please tell us whether [too much testing] is a major drawback, a minor drawback, or not a drawback for you.” Eighty-three percent of the contented said it was a drawback (49% major, 34% minor); 95% of the disheartened agreed (70%, 25%), as did 94% of the idealists (52%, 42%).
<
p>In response to other questions, three quarters of the teachers said that “student test scores are less important than a lot of other measures.” Two-thirds agreed that “Tying teacher rewards to their students’ performance” would not be an effective improvement approach.
I think it’s a shame if it would take “really big, politically significant changes” to make the test fulfill its claims. I don’t really know how to categorize the following:
<
p>*Have the test written by education professionals in-state, and not a private company in New Hampshire whose profit model depends on a large number of students failing the test.
<
p>*Offer results to teachers in a timely way, and not just the partial window offered starting this year that make results of questionable value.
<
p>*Test the curriculum, not next year’s.
<
p>*Write a culturally neutral test.
<
p>*Hire test scorers with the education required to successfully offer a fair score.
<
p>If these are “really big, politically significant changes”, I think that says a great deal about the integrity of the regime. If the current administration of Massachusetts were serious about the integrity of this test, I’d love to sit in on the type of effort Goldstein describes.
begins with the logistical limits of standards-based testing rather than question writing. There is simply not enough time to sample all of the content students are supposed to have learned.
<
p>As James Popham writes,
There’s no way to learn my curriculum in 160 days, much less test it in 90 minutes.