Yes, you read that right. Despite four recent polls from Suffolk (taken 9/13-16), Public Policy Polling (9/13-16), Western New England College (9/6-13), and MassINC (9/15-17) that all show Elizabeth Warren with a lead of between 2 and 6 points, the Herald’s latest effort (taken 9/13-17) shows Brown up 49-44 with likely voters. The margin is one point larger, 50-44, with registered voters.
Weird, huh? Most of the polls were taken during the same time period, so either the Herald is wrong, or the other four are wrong. So here’s one observation: both the Suffolk and the MassINC polls had 36% Democratic respondents, which is right on the money according to the statewide registration figures (I couldn’t quickly find the PPP breakdown). WNEC asks its respondents to self-identify rather than using party registration; it had 37% self-identified Dems. The Herald’s poll, like WNEC, also asked respondents to self-identify which party they affiliate with. Without “leaners,” the poll had only 28% Democrats, which falls well short of the actual party registration figure, whereas it had 12% Republicans, which slightly exceeds the actual party registration figure of 11%.
So, I’m no pollster, but it seems to me that the Herald poll under-sampled Democrats and slightly over-sampled Republicans. Which could certainly explain why the Herald’s result is so far out of step with the other four polls released over the last couple of days. If anyone cares to explain why I’m wrong about the sample, or has another explanation, I’m all ears.
5.5%. Dusting off the old college hat, 5% and under were considered the golden standard in the poli-sci courses I took. None of the other polls had margins of error that high and one of them was as low as 3.3%.
All that said, instead of trying to find all kinds of reasons to explain it away, let’s take it as one of the five polls released this week that shows a very close race. Momentum is clearly on our side, but momentum means squat in September if we don’t ensure it continues into October and November. There are lots of doors to knock on and voters to talk with.
Get your eyeballs off the screens and your tush off the chair. Let’s win this thing the old fashioned way. Let’s earn it.
Mass. registered Republicans tend to vote in higher percentages than Democrats and the unenrolled. The percentage of election day voters who are Republicans is typically higher than the percentage of registered voters who are Republicans.
That said, it does appear that the Herald poll undersampled Democrats and probably oversampled independents. Unenrolled voters tend to vote in much lower numbers than their 50% (or so) registration would suggest.
I could find an exit poll for the 2010 and 2008 general elections. In 2010, the turnout apparently was 42% Dem, 16% R, 42% U. In 2008, it was 45% Dem, 13% R, 40% U. Have you got data saying different?
I don’t have the numbers on hand but your numbers square with my memory and reinforce the comment that I was making. 11% of Registered voters are Republicans, but more than 11% of the election day voters are Republicans, according to exit polls. Over 50% of Registered voters are unenrolled, but only 40% of election day voters were unenrolled in 2008.
both exit polls show Dem turnout substantially *above* the registration number of 36%, while the Herald’s poll is way *below* that number. That’s my point. The GOP numbers are far less dramatic.
You mentioned they slightly over-sampled the GOP in your post, I posited that they did not over-sample Republicans based on voting patterns, and so must have over-sampled the unenrolled, who do not vote as reliably. I’m not sure what you think I’m trying to say here.
I was thrown off by your first line: “Mass. registered Republicans tend to vote in higher percentages than Democrats and the unenrolled.” That doesn’t seem to square with the exit poll data, which suggests that Dems vote quite a bit higher than their registration numbers; GOPers a little higher; and unenrolled quite a bit lower.
Assuming the standard 95% confidence interval in a poll’s margin of error, that still means there’s a 1 in 20 chance that a poll will have inaccurate data even if properly modeled. While I generally try not to assume that the poll with results I dislike is the poll that happens to fall in that 5%, when you’ve got a slew of polls showing one result and one lone poll showing another result, all taken around the same time, that is a possibility.
Whatever the reason, none of these polls are predicting results — this race is too close and mid-September is too early. I agree 100% with the comments that this is a close race and grassroots work will be critical in deciding outcome. I’ve just put in for time off on Election Day so I can help with getting out the vote on November 6.
this just motivates me to put in more time on my end too.
*looks around to see if Kate pounces on her again*
Is the coast clear?
That does square with the surge in the other polls being a “Dems are coming home to Warren” analysis which seems to be the general consensus.
So, a big part of this is turning out the Dems, along with convincing at least half the undecided unenrolleds.
We can do this.
By painting Brown for what he is – republican
It looks like the Herald changed how they conducted their poll to undersample Democrats. The Herald conducted a poll in December 2011 the poll sampled 33% Democrats (which could still be argued as undersampling), from 33% they went down to 28%. A good question to ask the pollster. Why did you change your sampling parameters to lower the amount of Democrats?
Sept. 13-17, 2012
RVs by party – Unleaned: D 28% of RVs, I 56%, R 12%
Leaned: D 55% of RVs, I 8%, R 36%
LVs by party – Unleaned: D 28% of LVs, I 56%, R 11%
Leaned: D 56% of LVs, I 7%, R 36%
Dec. 1-6, 2011
Party breaks – Unleaned: D 33% of sample, I 50%, R 12%
Leaned: D 56% of sample, I 11%, R 32%
for my state University to be partnering with the Herald of all freaking outfits…ugh.
Marty, Marty, Marty.
These are VERY good questions. I hope the University is thoroughly chastened about their model come election day, and not just because I want Warren to win, but because I’ve been squidgey about this partnership since they announced it. All well and good to create the political arm and such, but…they couldn’t find a better partner?
Suffolk leans right, WHDH and Andy Hiller is a right winger, while I do often question the timing of the polls, based on Suffolk’s history I do trust their numbers. UMass Lowell needs to understand this if they want to build a program and have people take them seriously.
We did not “change our sampling parameters.” See my explanation in a new comment below. And note how the “leaned” D numbers are consistent from December to now.
Democrats better focus and leave it all in the streets.
#yafeelin’me
supporters to feel like they’re winning, particularly if they feel like they’re winning, but know that it’s close. Far from making them less likely to go out there and campaign, I think it makes them that much more likely, because they know their effort is that much more likely to come with an award attached (victory in the end, and more favorable results while door knocking in the interim).
I produced the poll for UML. I’m sorry I wasn’t able to chime in sooner to try to clear up some misconceptions.
We don’t ask party registration but self-described party ID, so comparing our party breaks to state registration figures is apples:oranges. We ask two questions about party ID near the end of the survey, amid demographics and after all the political questions. First:
Now it is true that this time, this initial question yielded fewer self-described Democrats (28%) and more independents (56%) than in our earlier polling (last Sept and Dec), when we got around 33% D and 50% I.
But when we asked self-described independents and “something else” a follow-up question – “As of today do you lean more to [RANDOMIZE: the Democratic Party (or) the Republican Party]?” – we end up with 55% “leaned” Democrats and 36% leaned Republicans. And those are just about identical to the leaned party breaks we were seeing last year.
What does this tell us? First let me point out that near the start of the survey we asked a presidential trial heat followed by the Senate test. (We deliberately followed ballot order.) And we got 22% of Obama voters saying they’d back Brown if the election were today.
I suspect that after a good number of respondents just told us they were ticket splitters – by definition these are weak partisans, or firm independents – some may have been less inclined to initially self-identify as Democrats. But when we “lean” the independents we end up with the same share of Democrats as we have before.
That tells me we didn’t have “too few Democrats” in our sample.
In fact, only 12% of the initial Democratic identifiers said they’d vote for Brown – but 30% of Democratic-leaning independents backed Brown. (These start getting to be small subgroups, 156 and 140 respectively, but that difference is statistically significant.)
Another piece of evidence is Obama got just as much support, in favorable rating and vote preference, as in our two earlier polls and other recent polls. If we had “too few Democrats,” Obama should have taken a hit too.
We checked demographics a whole bunch of other ways too and are confident our sample wasn’t skewed. Without getting into any further detail, I think what differences there are between the UML poll and others recently mostly can be chalked up to methodological differences. We try to be a model of transparency so see http://uml.edu/polls for extremely detailed methodology statements on all our polls.
I’ll try to check in again if anyone has further questions but I’ve got a very busy day ahead so it probably won’t be before the weekend.
Mike Mokrzycki
mjmsurvey.com
However, I have to say that what troubles me is that both times UMass-Lowell/Herald has polled the race, the results have seemed out of step with other independent polling that was going on. Back in December, you reported that Warren was +7 – that is the highest that she has *ever* polled, and at a time when other polls showed Brown ahead, or Warren with at best a very modest lead. And now, there are four polls showing Warren ahead, including three in exactly the same timeframe, and yours again is substantially out of step. Particularly when Suffolk, which as you know exactly nailed the race in January of 2010 and also did very well in the 2010 MA general election, agrees with the other current polls and is way off from yours, it’s hard not to wonder whether something is up with your methodology…