One possible explanation for the divergent results comes from the questions asked just before the vote preference question. Both the Suffolk and MassINC polls begin by asking voters whether they have favorable or unfavorable impressions of each of the candidates. The MassINC poll then immediately asks about vote preference, while the Suffolk poll also asks the following:
Q9. Does Senator Scott Brown deserve to be re-elected or is it time to give someone else a chance?Q10. What is the first word or phrase that comes to your mind when you hear the name Scott Brown?
Q11. What is the first word or phrase that comes to your mind when you hear the name Elizabeth Warren?
Q12. Does Elizabeth Warren have the experience to be a United States Senator?
Q13. Is Scott Brown a leader in the United States Senate, or a follower?
Q14. If the General Election for United States Senate were held today and the candidates were Republican Scott Brown and Democrat Elizabeth Warren for whom would you vote or towards whom would you lean at this time?
The two open-ended questions (Q10 and Q11) are not troublesome, but the order of the other items is. While each uses neutral wording, they may have the collective side-effect of reminding respondents, three times, that Scott Brown is an incumbent senator. They may also raise doubts about Elizabeth Warren’s experience while planting the suggestion that Brown has been a leader in the Senate, not a “follower.”
And it’s not just liberals questioning it, it’s poll outfits like the one Romney and Brown employ:
The issue on the Suffolk survey is similar to a criticism leveled earlier this month by Mitt Romney campaign pollster Neil Newhouse about a recent ABC News/Washington Post survey. Newhouse argued that just before measuring the Barack Obama-Romney sentiment, the pollsters asked a series of questions that “introduced specific negative information about Governor Romney.” These included a set of questions about three candidates — Romney, Newt Gingrich and President Obama — as well as several more specific items about Romney. These included a question about whether, given his “work as a corporate investor … Mitt Romney did more to create jobs or more to cut jobs,” and a question asking whether Romney “is or is not paying his fair share of taxes” having “paid about a 14% federal tax rate on income of about 22 million dollars last year.”
Asked to comment by the The Huffington Post, Newhouse — who is also the pollster for Scott Brown — said that the criticisms he leveled against the ABC/Washington Post poll would “absolutely” apply to the Suffolk poll, “though not to the same degree” as the ABC/Post poll.
Suffolk, of course, is pointing to the “but look how accurate we were in the special election!” argument to back their poll up. But in polling, past performance is not always indicative of future accuracy. When your poll is already an outlier, you should expect to have your methods examined with a fine tooth comb. Especially from other pollsters who have everything to gain from your loss of credibility, even if it hurts a candidate they are working for.
David Nir also argues that Suffolk isn’t one of the stronger poll outfits, to begin with.