Sorry if this has already been posted, but I wanted to take a final look at how the polls did predicting the Warren-Brown race. Poll data courtesy of RealClearPolitics.com:
Actual election result: Warren +7.4
Suffolk/7News: Warren +7. Despite their national embarrassment of pulling out of 3 states they claimed were non-competitive for Romney, when those states turned out to be, um, pretty competitive after all, Suffolk nailed this race. Poll done 10/25 – 10/28.
Public Policy Polling: Warren +6. Also did well predicting the final result. This one used the most recent data of the group at 11/1-11/2 and the largest sample size, 1,089 likely voters.
WBUR/MassINC: Warren +6. Predicted the result despite less recent data, taken 10/21-10/22.
Rasmussen: Warren +5. Despite their often Republican lean, their final poll on 10/25 was reasonably close. However, earlier polls skewed more Republican; for example, their late September poll showed a tie while many others had Warren with a comfortable lead.
Western NE University: Warren +4. Correct result, slightly underestimated the margin. Data from 10/26-11/1.
Boston Globe: Tie. Taken 10/24-10/28. Helped feed the media meme of a horserace — and encouraged volunteers to GOTV; but not very helpful in predicting the result.
UMass/Boston Herald: Brown +1. Worst of the lot, and using recent data, no less: 10/31-11/3. This poll skewed heavily Brown in September also, showing Brown up 4 when every other poll except Rasmussen’s tie showed Warren with a lead. I’m guessing there was a problem with their overall voter sample weighting, their “7-item turnout scale” likely voter methodology, or both.
I wonder whether these results are consistent with the house effects of the respective polls. For example, does UMass/Boston Herald generally report 6 more points for the GOP than the end result?
I wonder this idly because I’m not willing to do the research to find out.
Again, all data from RealClearPolitics.com:
In the 2010 special election, with the final result Brown +4.7, PPP had Brown +5 and Suffolk had Brown +4. So they’ve both got a good track record in our last 2 Senate races in their final polls.
Rasmussen had Coakley +2 in their last poll, but that was the week before those others, when almost everyone but PPP and Suffolk still had Coakley up. The last Boston Globe poll on RealClearPolitics was first week in January showing Coakley up 17. However, that was a race which really did appear to swing considerably in the last couple of weeks, so I don’t blame polls relying on data more than 7 days old for not being predictive. The mistake was in not anticipating the race could swing so much in the final days, not in the data they had from several weeks before the vote. Don’t see UMass as polling that one.
In the 2010 gubernatorial, final result was Patrick +6.3 over Baker with Cahill getting 8. Suffolk’s last poll on that one had Patrick +7 and I don’t see PPP polling in there. So Suffolk’s 3 for 3 on those elections.
Boston Globe/UNH had Patrick +4, reasonably good and quite a bit better than the tie they forecast on this one. Rasmussen had Patrick +2 — again correct result although somewhat undercounting the Democratic margin. No UMass/Boston Herald, at least in the RealClearPolitics aggregation.
Western NE had Patrick +8, a very slight over on the margin, vs. 2012 very slight under on the margin.
So some polls seem to have a bit of a house effect but others don’t.
is such an embarrassment. That is all.
I’ve said it before, I’ll say it again…Rasmussen does this every. Single. Cycle. They skew Republican so that they might drive a “the Republican is doing really good!” narrative in the hopes that it pulls the election towards the Republican in any given race. Then weeks before the election, they pull their methodologies in line with everyone else so they can be accurate in the final result and get schmucks to hire them next cycle.
Really, I’m am hoping people are not going to fall for it ever again. This pattern is very set and has happened many times, now.
if Republicans want to throw bad money after bad money and stay in their little bubble so they can keep being surprised when they lose…
Context such as who has a good track record and who doesn’t; and context such as if one poll looks like an outlier, that be noted in coverage instead of news reports treating it as the best piece of info on the current state of the rate because it’s new. Is it too much to ask that Nate Silver’s publicized success this cycle translate into a little more statistical context in news reports about polls going forward?
if reporters starting taking the geeks seriously, they’d put the media-punditocracy complex in serious danger. Don’t look for it to happen any time soon. đŸ™‚
though it’s possible that they did that in Massachusetts, Ras continued skewing Republican right up to the end in several key swing states. His final polling had Romney well up in CO, VA, and IA, and ties in WI and OH. It seems possible to me that, rather than having a dastardly plan to skew GOP early and then etch-a-sketch his way to respectable results late in the game, he may just be a crappy pollster.
national Pres. poll also tilted from Romney up a few to a virtual tie on Nov. 5
he’s far from the only national tracker that showed that pattern.
Globe/UNH was odd throughout this Senate cycle. They yo-yo-ed around and had a lot of undecided to the end. Herald/UMass Lowell was way off in the last poll, but some of their earlier polls were pretty consistent with the other pollsters like PPP and Suffolk as I recall.
By the way, didn’t someone predict that once the Globe poll showed it tied, the Herald would turn up with a poll favoring Brown?
If his house effect is predictable, then it’s just a matter of a small arithmetic adjustment to transform his polls from crap into gold.
a constant predictable bias or inability to understand electoral shifts. As far as I know, Rasmussen is among the pollsters who are not calling cell phones, and it’s possible that their model to account for that may get worse next cycle. To quote Nate Silver’s post from yesterday:
…why are there so many polls in the first place? I mean, really, what news value to they have? Polls are useful internally to help a campaign figure out messaging and what not, and yes, I can understand releasing an internal poll that shows good news for your campaign so you can spin the narrative. However, the polls tell me nothing about what I really need to know as a consumer of news, like what are the comparative positions of candidates on various issues. Most infuriating are the national presidential polls which certainly everybody knows are close to meaningless given the electoral college, but only Nate Silver and a couple of others analyzed on that basis. Thus, we were presented with a nailbiter of a presidential race whereas in reality the President was comfortably ahead for almost the entire time.
that we pretty much don’t hear about, but there is also a multitude of polls taken by the media, independent pollsters, and various political groups. I guess that some of them are meant to be informational, that is the true “snapshot” polls. I also think that a lot of them are driven by campaigns and supporters for purposes of moving the electorate in one direction or the other, the push polls we occasionally hear about. There are also polls I call prestige polls, sponsored by various media to lend cachet to themselves as a source to be looked to. An interesting thing is how are the various pollsters determining their filters and mix of Ds, Rs, and Is. Are they developing it themselves, or is the info coming mostly from the campaigns and major parties? If the latter is true, then that may be why Republican leaning polls came up with such bad results since they were using foundations from flawed sources with an inherent, and incorrect bias. You know, GIGO (Garbage In, Garbage Out). Just a thought.
Of election coverage, which is depressingly similar to a pennant race. Who has pulled ahead? Who’s falling behind? Instead of being an interesting minor component to the coverage, it seems to be the driving force. In which case media need a lot of data points to report on, regardless of whether they’re properly analyzed. So, companies that want to be in the business of market research for corporate customers no doubt know that presidential polling will give them lots of free news coverage.
Until media consumers demand different coverage by tuning out the penant-race coverage, I fear we’ll keep seeing this. Nate Silver may be our best hope on that, having shown most of the breathless horse race meme to be worthless.
The overwhelming amount of election coverage is horse-race focused. Who’s ahead or behind, who’s got momentum, how will such-and-such play in Ohio. There is precious little focus on actual policy proposals, and what there is generally is bad.
Like he said-she said bad. “President Obama charged that Romney’s tax plan doesn’t add up. Mike Jones, a top Republican strategist, said it does too and Obama’s just running away from his own failed record.” Little independent analysis of what is and isn’t true.