A “highlighted member” of American Test Publishers, Measured Progress is the New Hampshire company that is best known round here for “writing the MCAS”. The MCAS, the formal assessment used to measure students’ progress here in Massachusetts, was originally written in-house by personnel of the Massachusetts Department of Education. For various reasons, the government soon privatized the job, awarding it to Measured Progress. Oddly enough, that company had recently bulked up by hiring many workers late of the Massachusetts Department of Education. So the test was written by the same people as always; now they were just doing it out-of-state, and for more money. (As the company has expanded, they’ve added education bureaucrats from other states as they pursue contracts elsewhere.) In 2000, they declared themselves a “non-profit,” but this interview in Bloomberg Magazine wherein their CFO talks about “tripling margins” demonstrates the superficiality of such a change…you can still make six figures in a “non-profit”.
As with all privatizations, this government function now passed to an entity with entirely different motivations. As we saw with the Big Dig, private companies will do the minimal job for maximum profit, and Measured Progress is no exception. More insidious, still, is the fact that the fortunes of Measured Progress depend on students failing their tests. And the choice between corporate financial health and students’ education appears to be an easy one.
For the real money in education is in consulting. While Measured Progress makes a lot of scratch selling and correcting the MCAS and similar papers, the growing part of its business model is “consultants”. The Department of Education demands more and more engagement of consultants by local districts. Meanwhile, Measured Progress and others have stepped in to take advantage of this forced spending by districts that can’t buy their own paper. For prices that range upward of $10,000, a consultant will come on in and tell you how to beat the test that the company writes! So when you think of the budget troubles in your district, do not forget that the state is forcing many of these towns to pad Measured Progress’s bottom line by shipping in consultants who just as often tell staff what they already know. Of course, the MCAS has proven to be a boon to private industry, as many companies have gotten into the act.
Paranoid ravings? Well, Measured Progress is very aggressive in keeping the MCAS secret. For a taxpayer-funded document, taxpayers have little access to the fruits of their money. As a teacher I am not allowed to look at the exams as students take them, or even reveal the questions for one year. Losing a question booklet occasions investigations and fines much worse than those following the loss of students’ personal information. However, I’ve been through enough cycles, and discussed the test with enough students, that I have collected some examples of Measured Progress’s choices. In the MCAS itself:
- One common tactic is to ask a question about the very last line on the curriculum. Given that the MCAS is given in two phases in March and May, the test is always administered before the curriculum is finished. Thus, Measured Progress keeps the scores low by asking questions about material students have yet to see.
- Another tactic is the old cultural bias routine. Science and mathematical questions abound with cabinetry, gardening, and other manner of suburban pursuits irrelevant to urban youth.
- Finally, some questions are outright not in the curriculum. There are many cases of questions post facto not counting toward scores upon protest, but come next year there are still out-of-curriculum questions.
It bears mentioning that Measured Progress takes a “trust us” approach to scoring. Answer booklets are never returned, just a sheet of what the company says were the correct answers, and which answers the student chose. The veracity of these, in many circles, is considered suspect.
Finally, there is a strong disconnect between the scores of Massachusetts students on national tests and the MCAS. Bay State students score among the tops in the nation on the SAT or the NAEP, yet fail their in-state standardized tests at a high rate. I’m all for tough standards, and I want Massachusetts to demand a lot of its schools and its students. However, this is just another item in this trend toward finding ways to make students fail.
(Of course, when things get really bad, you can just have your friends in the Department of Education suddenly and arbitrarily move the goalposts so more students fail.)
How much of this is a conscious effort to low-ball our students for revenue? Only the company elders know for certain. However, Measured Progress is at best a company whose incompetence in writing tests have led them to stumble into a very lucrative fleecing of the state of Massachusetts. At worst, they have a conscious policy to enrichen the company at the expense of the students of Massachusetts.
I would hope that Deval’s new “Readiness Project” will address the fact that our understanding of student performance is held by a company that stands to win as our students fail. However, in my conversations with the staff of this project, I’ve been disappointed with their ignorance of this situation — some Readiness staffers were surprised that the MCAS is given by a private company.
(Cross-posted at my My Second Home)
post. I hope this results in some questions being asked and answered on why this was ever outsourced in the first place.
how quickly people forget what it was like BEFORE Ed Reform in Mass. How many of you remember?
There was A LOT less money spent on education in the early 1990s and before, and yet, the children of this state actually received a pretty good education. But, it wasn’t enough.
So, the state made a HUGE investment in education but not without some sort of requirement which showed actually results and that the money was being well spent. For the most part, this has worked.
The problem has become that schools began “teaching for the test” in order to ensure the extra funding continued. So, it is a double-edged sword. Sure, you can drop MCAS. But will it be much longer after that that taxpayers go, “Well, OK, we want to go back to 1992 percentages for funding education …”
People should really be careful about what the advocate for because the end results might be something WAY BEYOND what they want.
but I didn’t live here at the time, so have no idea what it was like in MA before. I actualy didn’t even know that the testing was a requirement resulting from the investment. It would be good to hear more about the history in the state. I only have experience with the parenting part of teaching, so I trust others with teaching experience to analyze what’s going on.
Excellent questions. There was a subcommittee on assessment so we will see what they say.
Yes, there was a Subcommittee on Assessment. 6 member of that subcommittee released a statement earlier this week saying that “Patrick and his advisors have ignored the advice of the grass roots team they assembled.” The full statement is here: http://www.fairtest.org/MA-MCA…
I got there through this one.
page about MA MCAS. This is the link I believe you were going for.
yes, that’s it – thank you
to hear that six people are so dismayed about participating and seeing no results from their efforts. Thank you for telling us about it.
I can’t decide which if those words better describes your post. While I can think of a couple of specific improvements, overall I think people get overly scared by the idea of a test. There are some good points here, but I’m going to address your bullet points.
<
p>Secrecy – I’m not sure why the year buffer, as I thought they were put online sooner than that. High stakes means for some high temptation to cheat so secrecy and security make a lot of sense.
<
p>Curriculum – Different teachers have a chance to cover different material and different amounts of material. In my experience there is a range of coverage, but plenty of students still pass. Remember, the goal of the MCAS is to pass, not to get a perfect score.
<
p>Cultural bias – I’ve never understood this with regard to Science and Math, using as they do objective formulae. In other words, the perimeters of an inner-city basketball court and a suburban rectangular flower bed are both going to be found by 2L + 2W.
<
p>A few years ago my State Senator, Sue Tucker, invited adults from her district to take a sample MCAS then discuss it. Not only did I get alomst every question correct, but I remembered which HIGH SCHOOL class I learned it in. It was not a matter of having since attended college and had more life experience. The one concern was a “graduation” exam given in 10th grade did ask a couple of questions that I would not have known in 10th grade, but did by the time I was really ready to graduate.
<
p>I also think people need to relax about “teaching to the test”. If the subject is presented correctly the test will take care of itself. I have believed this ever since I sat for the AP US History exam. My teacher not only tried to predict the essay questions, but bragged about her track record for guessing correctly. She told us that there was no way that the document essay would be after World War II, but sure enough it was about the 1960s Civil Rights Movement. There were a few groans when this was discovered, but the class came away with the same passing rate as previous classes when she had guessed more accurately. This was because she covered the Civil Rights Movement, not because of a test, but because it’s just something you cover in US History.
I don’t think it’s paranoia…education is big business. I don’t think it’s paranoid to impute onto Big Testing the same motives that are routinely accepted as given for Big Pharma and Big Oil.
<
p>Secrecy – If all tests are given during the same 2-week period, why withhold the questions for 50 weeks beyond that? Testing security doesn’t make sense.
<
p>Curriculum – I’m saying either test what has been taught, or re-arrange the curriculum so we teach what will be tested. Some of these questions may as well be in Estonian for all the students have learned them. I don’t expect everyone to get a perfect score, but I think everyone should have a fair chance at answering the question.
<
p>Bias – One of the greatest difficulties for students in grades 4-9 is determining relevance to the problem. If you’re inserting foreign ideas due to cultural bias, that handicaps students trying to throw out the junk. The basketball court you mention isn’t a typical MCAS problem. The typical MCAS problem is 6-8 sentences, of which maybe 3 are relevant. If you include things with which students are not familiar, you’re directly confusing them.
<
p>The MCAS and MTEL don’t seem that hard to me. Then again, English is my first language, and I grew up in a middle-class suburb. My family strongly participated and monitored my education. However, when school funding, scholarships, and jobs are on the line, I think it’s only fundamental to ask that these tests be fair. And the writers of these tests have a vested interest in making sure that they are not. I just hope that people making education policy are aware of that — thus far it seems that they are not.
I just have a hard time believing anyone would WANT students to fail. I also think that plenty of kids do pass, so if failure is the goal they are doing a terrible job. When MCAS first came out there were some disconnects between what was taught and what was tested, but I’ve seen quite a bit of improvement. In my school system (I substitute teach.) we use math texts written specifically for the MCAS. If there is anyway you can provide an example of a biased question I would like to see it, as I’m having a hard time coming up with one. I will point out though that one of the skills we teach in math is how to distinguish and discard irrelevant information.
<
p>Education is an industry people by human beings, and some of them ain’t so nice. If your goal is to make money (not to help students), and you make money when students fail…well, then, the desire writes itself. It is true that schools are much more careful teaching what the MCAS tests (“aligning with the standards” in pedagogy-speak).
<
p>I’m all for distinguishing relevant and irrelevant…where I have the problem is including information that will necessarily confuse a student. For instance, three years ago was a math problem on planning a rooftop garden, and the question was much more difficult if you couldn’t picture a rooftop garden, understand why it had to be laid out in a particular format, and understand where the restrictions are coming from. Save some upper-middle-class urban types — they types who write these exams — who is that familiar with this idea? This is a question that set up our most at-risk groups.
It’s one thing for an outside company to write the specific questions, but shouldn’t the policy makers in this state be determining content? I always assumed that was the case and if not then they need to step up and take charge. I also thought educators with teaching experience had input into writing these exams, and if not, they should.
<
p>Still trying to picture a question with bad information. I may try to find previous exams online to see what I can find. I actually figured rooftop gardens were more prevelant in the city since most suburban houses I know have sloped roofs.
<
p>Maybe some find this overly idealistic, but I thought the whole point of school was to be the great equalizer (at least socio-economically). In other words it shouldn’t matter what background you come in with, you have the same opportunity to ultimately match your better-off peers so long as you put in the effort. Of course, that means a commitment to make the schools excellent as well.
…which is why I want an equal-access test. Christopher, do you think you’d find more rooftop gardens in the Back Bay or Mattapan? Our at-risk students may live in a city with lots of rooftop gardens, just as I may live in a town with a million-dollar house for sale. Doesn’t mean that we see them that often.
<
p>The state did determine content in the form of the curriculum frameworks. There’s very little on food webs in the curriculum, but dang if doesn’t show up on the MCAS on an almost yearly basis. Measured Progress has decided to test food webs, so now it is taught out of proportion to its place on the curriculum.
<
p>These are good questions here Christopher. I’m glad you’re asking them, and the reason I threw this post up there is because I don’t have the impression that folks at the DoE are.
This will be a problem with any word problem, no? The entire purpose of w word problem is to see if the problem solver can filter out irrellevant information– that the garden is supposed to be on a farm, on a roof, a community garden at the vacant property next door, or on a flying saucer– and determine tha is an area question, assuming that is what it was.
<
p>We already know that lots of urban students in poorer districts have a lot of problems stemming from things that are hard for a school to fix, like parental involvment or even nutrition.
<
p>Could it be that the perceived bias of the test is really just picking up those problems, as it is supposed to do?
<
p>it extends way beyond that. Go read this book for starters (easy, awesome read: it really only touches on the subject, but shows the full extent of its dangers). The problems Sab has talked about, of question sample bias, extends way back in American history and even has its roots in the Eugenics movement.
<
p>Off Topic: Since when did Google Book include the entire freaking book, still with copyright protections? That’s downright scandalous! It almost pains me to link to it. If you want to be a good doobey, I bought my used hard cover copy for $3 plus shipping on Amazon when I had it for assigned reading back at college.
They leave out pages in between. Try reading 3 of every 4 pages of a book — you’ll find it’s terrible. 14, 15, 21, and 22 are missing in the link above, for example.
<
p>By including so much, it helps people decide if it’s the “right” book but still provides tremendous incentive to buy it.
I didn’t check page by page, just merely realizing it went all the way to the last page. That is a terrible way to read a book. That said, including 100s of pages of a book, if without permission, violates fair use, that’s for sure.
Google gets permission for every book. There was a huge outcry from publishers when Google launched this project, and since then, Google’s tried to be good about getting permission.
<
p>With this book in particular, if you look over on the right it says:
It’s understanding what the word problem is saying to begin with. If you’re unfamiliar with half the objects in the word problem, PLUS you don’t speak written english, you speak roxbury, you’re not gonna be able to get those word problems correct, even if you know how to smart and can add.
<
p>Add up the intimidation value of written english, then throw in the fact that half of the nouns in that word problem are things you don’t see in your daily life, but a kid from Sudbury does. It’s a little unfair.
<
p>Now, of course a bunch of the problem is teaching poor kids better english. But that doesn’t mean they should be flunking the math section.
I’m not even sure I spelled it correctly, but we read a poem by that name in high school, I believe by e. e. cummings. He uses deliberately nonsense words, but the context makes clear what parts of speech are involved so the overall poem still somehow makes “sense”. Real English should be constantly reinforced in EVERY context. Kids now are using instant message abbreviations in essay writing which to me is unacceptable. When I substitute teach at the elementary level a student might ask, “Can me and him…?”, to which I say, “Me and him aren’t going to do anything – try again!” It sometimes takes a few tries and even a whispered hint from another student overhearing the exchange, but I don’t let it go until I get “he and I”. The student may then complain that this is Math class and not English, to which I respond with something to the effect of, “Do I look like I care!?”
<
p>(Now I’m just trying to decide whether the word “smart” in the first paragraph is a mistake on your part or deliberate slang you used to make your point.)
But yeah I agree.. point is, that kids are gonna fail math questions because of weakness in english. I’m not making a larger point here, simply saying that if you have word problems that involve stuff that kids from the suburbs understand and kids from the inner city don’t, even if those things are irrelevant, it’s still a harder question for the city kids than the suburban kids.
<
p>If they were all perfect syntax and logic calculators, then we wouldn’t have to test them at all 🙂
Arthmetic is great, but the whole point is to be able to put it in context. If you can’t do that, for whatever reason, why should you be passing the course?
Lewis Carroll wrote Jabberwocky. It’s from “Through the Looking-glass, and what Alice found there.” For somebody who, upthread, claimed to remember even which class they leaned something in, to not know the authorship of such a famous poem seems a bit glaring.
<
p>At any rate, for context, here’s the poem in question:
<
p>
<
p>The point being, though, that a lot of the nonsensical words here are to lend a feeling, an atmosphere, to the overall “story” being told. Their meanings here are not important.
<
p>But supposing there was a question on the MCAS depending on the meaning of “mimsy”. These sorts of things can and do happen. Large portions of the MCAS-takers or SAT-takers or whatever, may not know crucial details about a problem that the writer took for granted.
You’re right, that’s a bit embarrassing. It was a one-day thing, but I guess that shows I could pass, but not get a perfect score, which is what I’m trying to emphasize. I stand humbly corrected:)
The rooftop garden excuse is a bit of a stretch. Do you have another example sabotage sabutai?
We’ve had a number school zombies:
<
p>1) schoolzombie87 (Thu Apr 05, 2007)
2) SchoolZombie’87 (Thu Sep 06, 2007)
3) schoolzombie1987 (Fri Oct 19, 2007)
4) schoolzombie (Fri Jan 11, 2008)
5) The School Zombie (Wed Feb 27, 2008)
6) Skewl Zombie (Tue Apr 22, 2008)
<
p>When will it end?
Virtually all tests have a vested interested in people failing. As a criterion-referenced test, everyone is supposed to become proficient eventually, but if the test were given and everyone passed, the test itself would be considered too easy. Following the logic of education reformers, when everyone passes, it will be time to make the tests harder. So failure is integral, and paradoxically, a positive outcome.
<
p>If most tests are interested in failing students, all tests are interested in producing a spread of scores. An underlying assumption of all testing is that achievement is variable.
<
p>The trick to test-making is using items that create a spread of scores, to make sure that everyone doesn’t get the same score.
<
p>Two of the most reliable ways to create a spread of scores is with test items linked to socio-economic status and native intelligence. I can’t give a specific example of the former, but I saw an example of the latter on the MCAS Math 10 when I proctored. It involved bending a shape or cutting it in half or something. Kids born with excellent spatial intelligence have an advantage. The problem is that the test item confuses an ability with a learned skill or knowledge.
<
p>On MCAS ELA 10, the context vocabulary questions are generally awful. Unlike the verbal part of the SAT’s, these questions can’t be figured out in context. And the context strategies we do teach don’t help.
<
p>Mark
I’m glad you got all those questions right. That obviously means every other person in Massachusetts should, too.
<
p>Seriously, these anecdotes are slightly useless. Teaching to the test doesn’t mean teachers will ignore the parts of the curriculum they don’t think will be on the exam, it means two things: they can’t teach what’s not on the state standardized curriculum and they can’t spend as much time on parts that aren’t focused on it. Maybe your teacher was an expert on the Cuban Missile Crisis, for example, and wanted to spend a couple days on that – but it wasn’t on the curriculum. That means her students are going to miss out on what she knows best. More importantly, it also means that students have less opportunity to learn what they’re interested in, because schools can’t have as many options as they did before since they have to focus so much effort on teaching classes related to the MCAS.
In my mind the MCAS should not be used to re-create a curriculum, but should be checking to see whether students are learning what should be taught anyway. I actually did not attend an MA public high school (and that was before MCAS anyway), but my school did not need a test to require teaching these things anyway. These questions were by no means extraordinary, so yes I do think it is reasonable that others should get them correct. I understand that there are different capacities to remember material, but my point is it is reasonable that they were covered even without an impending test.
<
p>As to your Missile Crisis example, why shouldn’t a teacher teach that? That is certainly a key event in recent US History and if the US History MCAS is approaching there may very well be questions about it. Teachers SHOULD be able to use their judgement about how much to focus on a particular topic.
My Missile Crisis example was just an example – pick any other fairly important topic you could come up with. I think the fact that students are learning is far more important than WHAT they’re learning. Of course, we should have certain standards, but all too often we force kids to learn about subjects and books they just aren’t interested in – when there are other things out there that would hold their interest and be valuable to them. Hence, teaching to the test.
<
p>So, the big question is this: who gets to pick what should and should not be taught? I’m far less interested in learning about world and civil wars than I am the history of cities in America, yet we spend far more time on specific battles in our curriculum, even, than we do on probably the entire history of urbanization, then the move to suburbs and exurbs, put together. Yet, the funny thing is, that history is far more important and relevant to today than what ship was sunk, and where, 60 years ago, or how many people died at the battle of Gettysburg.
<
p>The bottom line is that teachers are going to come in with their own expertise – I’d rather them teach about things they’re passionate about. Students are going to come in with their own interests – I’d rather we try to cater various classes to meet as many of their interests. But as long as there are more people than not who think along these lines
<
p>
<
p>we’re always going to be turning far more students into uncreative robots than is acceptable.
<
p>And as long as we’re doing that
<
p>
<
p>this statement only ever going to be a wild dream.
<
p>Look, we need kids that are inspired. America is what it is today because, for decades, we’ve had the most creative economy. We’re losing our edge as we’re losing our base industries, meaning we’re inevitably going to fall behind. We see that every day as the US is now sliding behind in patents, graduates and every other high-skill measure.
<
p>I’m not inherently against exams like the MCAS, because I think they can have their place, but the current one we have is flawed, the people who are deciding what’s on it and what isn’t is flawed and it’s not leaving enough room to allow our schools and our school districts to grow a large base of highly intelligent, curious and creative thinkers.
I am definitely biased toward the wars, politics, etc. than social history. I tend to take what I would call a “Great People, Great Events” approach to history. I’m not interested in super-specifics like the casuality count at Gettysburg, but I do get tired of reading the surveys that suggest an outrageously high percentage of high schoolers can’t even place the Civil War in the correct half century. As for cities, I would love if each system taught its own local history and its relevance to the larger picture. For example, when I teach Rev. War/early republic history, I try to remember to mention who from my town fought in the war and attended our ratification convention. I’m all for variety, but there are certain basics everyone should know.
the history of cities is important:
<
p>Currently, in America, one of the major reasons why we use so much damn gas and energy is because of the flight from cities to suburbs, and now from suburbs to exurbs. City and town planning barely even exists. We zone out pretty much anything that could make an area less boring, or attract a little diversity, while zoning in as many Earth-killing McMansions as possible, with yards so large that one wonders if we even have neighbors anymore.
<
p>So, let’s sum this up: things like “planning” doesn’t really exist, we encourage people to buy homes as far away from their work as possible and, just in case they aren’t using enough oil, then encourage them to heat as much square footage as possible, all the while inoculating us from such things as “culture” and “fun.”
<
p>Except, of course, our population doesn’t even realize how insanely stupid we are, or how destructive – and, in many ways irreversible – these policies are. Why? Because we can’t spend 3 days on it at schools.
<
p>Sure, it’s great to learn about the Lowell Mills, the Lynn shoe factories, local war heroes whatever rocks your socks – and we all do that, probably by the third grade. I’m not saying that shouldn’t happen. But unless people learn about how we, as Americans, lived 100, 70, 40 and even 20 years ago, we’re all fracking doomed to failure, as far as I’m concerned.
…is how I have to wedge in discussions and lessons about the current political campaign when the curriculum police aren’t looking. “Sorry, I’d like to talk about Obama, McCain, and solar power but the frameworks say I need to tell you about the Hundred Years’ War”.
Many interesting points. A couple of questions:
<
p>
<
p>I think that pharmaceutical companies do have a responsibility for determining proper dosages. Afterall, they perform the studies and get FDA approval. If doctors are then prescribing outside of tested doses, there could be unanticipated and negative side effects.
<
p>Doctors are great, but they aren’t medical researchers. You need both.
<
p>I’m trying to understand your statement that MP tells teachers how much of its (MP’s) education students should get. I thought MP only works with the tests. Are you saying that it also develops the curriculum standards, materials, lessons, schedules, teaching methods?
<
p>
<
p>I can find no evidence to back up that statement. MP states that it was the original developer of the MCAS. A similar statement is published in the book “State Assessment Policy and Practice for English Language Learners”, on the bottom of p. 231. See the sentence beginning Starting in 1997, when the MCAS was first being developed….
<
p>Was the MCAS truly written by the MA DoE? Are you asserting that the state could do a better and less expensive job writing and administering the test than is done by MP or any other private company?
<
p>Thanks, and I’ll look forward to hearing more.
but my friend wrote questions for the MCAS ELA 10 test for years. A handful of teachers developed the questions with the input of psychometricians from whatever company they were working with.
<
p>These questions are field tested and scrutinized for cultural bias, though culture is so multi-faceted that there is alway some bias.
<
p>Mark
But certainly better. The MCAS was written by the DoE; I was actually an adviser to the Board of Education at the time (and, along with my committee, actually made some subtle changes on certain sections and policies that were eventually accepted).
<
p>Sab is right when he talks about the private business model. They’re going to do the job as cheaply as they can, irregardless of achieving the maximum results (so long as those results are ‘good enough’). We’ve seen this with infinite amounts of industries. One of the most amazing PR jobs in this country to date is the lie that corporates are more efficient than government; it’s absolutely laughable, but people still believe it. I don’t know how many more Blackwaters, Halliburtons, Enrons and HMOs people need to find out about before they change their minds, but they haven’t changed them quite yet.
Rye, when you get into the private sector and the third dimension of time comes into play into efficiency (and also performance results in that if you stink at what you do, you lose the client or lose your job), you can post these comments. If you stink, you get fired. That doesn’t happen in the public sector…
Have a corporation on the brink? Let’s give the CEO a multi million dollar bonus! Corporates are wicked efficient!
<
p>I’ve worked in the private sector, Mike, and I’ve seen all sorts of incredibly waste. Sometimes it was so much that I wanted to scream. When there’s plenty of money coming in, you’d be surprised at how many sheets of paper and staples that get needlessly tossed into the waste basket, so to speak. I’ve worked in nonprofits and I’ve seen how they do so much with so little.
<
p>Want to talk private vs. public waste? Just look at the overhead of HMOs versus Medicare. ‘Nuff said.
<
p>Fair weather warning: I’ve yet to read a “you’re too young to know what you’re talking about” comment that I couldn’t swat back with 2x the force. I’m 24, not 14. If you have a problem with my arguments, by all means, use facts and prove me wrong. But attacking my arguments just because I’m a twentysomething and prepared to think differently than the old fogies, challenging old ideas and stereotypes, I think, is kind of silly.
<
p>Longtime BMG readers are not surprised to see your anti-MCAS argument. And I think to your credit, you’ve proposed other ways of assessing….
<
p>Still, I was surprised to see that you think ACCESS to the old tests themselves is a problem. I actually think it’s a strength!
<
p>Anyone can read all the 2007 MCAS tests here.
<
p>And the 2006 tests here.
<
p>Etc.
<
p>For example, the first question on the 2007 Grade 7 English MCAS is a writing prompt.
<
p>”Heroes have special qualities that people admire. Heroes give us examples of the courage and strength it takes to face difficult situations and challenges in life. Think of someone who is your personal hero. In a well-developed composition, describe this person and explain two qualities you most admire about him or her.”
<
p>While it seems like you want FASTER access to the tests themselves, overall there is easy online access in a *reasonable amount of time, no?
I’d like faster access…within two weeks of the test being given. I’d like to be able to access and discuss the tests before the scores are enshrined on students’ records.
I’d like the scores to be “turned around faster.”
<
p>Current set-up: Tests in May, results in late August. In late August, it’s harder to respond to the test results themselves in terms of changing curriculum, schedule, classes, etc for the coming year.
<
p>I want: Tests in May, results by July 15th. That gives teachers plenty of time to make changes to their year-long plan. It also gives a principal a chance to act to help the lowest scoring kids.
<
p>For example, if two dozen fourth graders are still reading at the lowest level, a principal might want to schedule those kids for an hour a day of before or after school tutoring, or even during school tutoring in lieu of something else. He needs to phone those 24 parents, solve the logistical issues (bus, etc).
<
p>He’ll get more parent buy-in if he can say “Your son is a weak reader. That shows up in his MCAS. He scored in the lowest level. We need to make him a good reader or he’s in for a heap of frustration and his drop-out likelihood down the road skyrockets. Your son can become a good reader. We have a plan…..”
and give this man a 6.
….isn’t that like the emcee of the talent show, after some dorky kid squawks out a passable song, jumping in with the energetic-and-boosterish “AND LETS ALL GIVE IT UP FOR”….
<
p>now i feel like the dorky kid. again. i mean, Rye has never given me a 6, it’s really YOUR 6…. 🙂
Sometimes high quality gets passed over, mired in mediocrity.
<
p>If the subject now read
what I want (5.50 / 2)
instead of
what I want (6.00 / 6)
<
p>than that dorky feeling would be warranted and well deserved. Now’s your chance to shine, bro.
My suggestion:
<
p>Company 1 writes the test.
Company 2 grades the test.
Company 3 consults on how to beat the test.
<
p>Company 1 in this case would be Measured Progress, Inc. They write the test, keep the booklets, etc. They get paid for that.
<
p>Company 2 would simply be shipped the student answer sheets and the answer key provided by Measured Progress Inc. Company 2 would grade the tests, compile the data, and be available for an audit.
<
p>Company 3 would do the consulting. While they have every incentive for kids to fail [more consulting business], they’d control neither the questions nor the grading. This would remove their ability to unfairly induce kids to fail.
<
p>This is the kind of checks and balances system organizations use when the vendor has an incentive to provide less-than-excellent service, just as Measured Progress does now. It might cost a little more in overhead, but the product will certainly be better too.
<
p>
<
p>P.S. Anyone else find it ironic that the company is interested in measured progress. Don’t want too much progress too soon!
… is done in construction all the time. RFP Design, Writing of Specifications, Conceptual Design, Preliminary Design, Final Design, Construction, and Construction Management all might be handled by a different mix of contractors.
<
p>It looks like a non-profit monopoly. See Guidestar (account registration required): 2006, Stewart Kahl and Richard Rizzo, respectively the President and CFO, each pulled down over $300K in salary/benefits, with at least 5 other employees making over $100K each. $72 million in Revenues; a balance sheet with $14 millioni equity.
<
p>Big money in non-profit it seems, on a “sell the razors cheap, make money on the blades” approach, the company has the successful strategy of low-balling the testing and cashing in on the prep material.
What society memes prompted the formation of Measured Progress.
How owns Measured Progress and what is the history of it’s founders. What did they do before. Do they have a common political slant, a particular corporate slant.
Is this another continuation and/or extension of Charolette Iserbyt’s dumbing down of America or has it now moved on to the newest social engineering “Compliance” meme.
Yeah, which Satan worshipping think tank did this come out of.
Excellent post, and I commend you for putting the issues out there for discussion.
<
p>What’s needs to be added is that the over-emphasis on testing results in on obsession with scores and “improvement” that goes way beyond simply meeting the standards. Even affluent districts where the students generally do fine have become overly concerned with keeping their rankings high. Why? Real estate, of course.
<
p>I teach in Brookline, and the test scores of my students remain quite high. Of course- they are predominantly white, affluent, and with very well-educated parents. Yet, I have been instructed to spend weeks prepping the kids for the MCAS by reviewing old test questions. Ultimately, who benefits? Not the kids; they miss out on good instruction for the weeks I’m cramming test questions down their throats. But by keeping our test scores up, property values remain high, and real estate brokers do well.
While I completely agree with your argument, that these scores drive towns to teach to the test so they perform well on the exam – thinking it’ll impact real estate prices – I’d actually like to see a statistical analysis on how changes to scores on tests like the MCAS and SATs actually effects property values: I’m guessing it’s less than people would imagine.
<
p>Are these rankings more psychological for towns than a real phenomenon, so long as the school systems are decent? Do towns think MCAS scores are actually more important to property values than they really are – especially when most people who move in and out of towns don’t have school-aged children and could simply care less.
<
p>People want to live in Brookline, for example, for various reasons: awesome public transportation, lots of fun things to do, as safe as it gets, well managed, beautiful, closer to Boston than probably most of Boston, with a good school system that will prepare all its students for college. Bonus points (according to my friends who live there) for pissing on the BU kids, keeping as many of them out as possible. Whether the town finishes 15th on the MCAS or 40th, I really don’t think it would really make a dent in property values, though I’d be curious to learn more.
One of the best pieces of advice I got about finding a teacher job was to ask local real estate agents about the school system. Man, they knew that stuff backwards and forwards! Says a lot about how often they hear the question…
of course they’ll know those kinds of answers. I’m not trying to say it doesn’t impact the prices at all – and I’d never suggest that, beyond impacting prices, it could help seal the deal. I’m just suggesting that as long as the schools are good, I’m skeptical that one town finishing a few towns ahead of the next will cause a seismic shift in prices, as some of the more histrionic types may believe.
<
p>Even if some people were Earth-shatteringly concerned by a statistically small difference in MCAS scores, I’d be willing to bet that 3/4 of the other people looking at the same house don’t have school-aged kids and are far more interested in the quality of the property, things to do nearby, commute distance and safety of the area. Those kinds of people help keep plenty of towns with school systems mediocre and worse very expensive.
<
p>Now, if the school system is going down the tank and there’s negative press in abundance, all bets are off and real estate property will still take a hit – but even that a) won’t show in MCAS scores for years, if ever, and b) won’t stop the fact that, if the town has other things going for it, it still probably won’t make a huge difference in the prices. Heck, just look at the real estate prices in desirable cities like Boston and Cambridge versus their drop out rates. Plenty of people are spending hundreds upon hundreds of thousands to live in shabby condos in both locations – and it’s not for the schools.
<
p>I agree that few people would generally care about a few places in the rankings… but if a town happens to slip, even a single spot, for three years in a row, local media could create a meme which would only hurt things. If Obama polls ahead 14%, then 12%, then 10%, what’s the meme — that he’s winning by a significant amount, or that he’s faltering?
<
p>On another note,
<
p>
<
p>Not the same house. People looking at 3, 4, and 5 bedroom houses almost certainly have kids or are expecting to do so. Sure, some pensioners might buy 3 bedroom houses, but by and large the number of bedrooms is closely correlated with the number of kids.
<
p>Same neighborhood? Possibly. Same house? Nah.
<
p>I haven’t done a statistical analysis on this, but I have looked at properties in Springfield versus similarly-sized properties in Wilbraham or Longmeadow (schools with very good reputations). The difference in price is about $100,000 on a $150,000 house in Springfield (i.e. a $150,000 house in Springfield will sell for around $250,000 in one of those communities).
<
p>Education isn’t the only factor — people want to avoid perceived crime problems (even though those neighborhoods in Springfield have no crime problems) and many people simply don’t want to live near anyone not their same race or ethnicity. But the schools clearly drive demand, probably more than any other factor. And consider that those people paying more for their houses are also willing to pay more for their commuting costs and their property taxes.
<
p>My company went through a hiring spurt a few years ago and I tried to get people to buy houses in Springfield. Every single person with kids’ first question was “how are the schools”. They didn’t reference MCAS scores, but the MCAS scores now make it possible to easily quantify differences between districts.