The Real Clear Politics (“RCP”) race-by-race average of polls is the second best place for political junkies wanting election predictions to get their fix. (The best guide remains Nate Silver of 538.com.) Yet a Pew Research study released in October raised the possibility that polls which over-rely on landline telephones overstate Republican support, by around 2-4 %. 1 in 4 households now use only a cell phone, and constituencies that favor the Democrats like young people and minorities are more likely to be cell only. In 2008, RCP underestimated Obama’s margin of victory by about 2 percentage points. Nate Silver has already pointed out some notable failures of polling this year: http://fivethirtyeight.blogs.n…
We know that Pew Research is going to take a systematic look at how accurate polls proved to be during this political season. Yet if we glance at some key gubernatorial and Senate races, there are a number of instances where the RCP polling average was off. The Rasmussen polls were particularly bad in skewing Republican.
Senate races
RCP had Barbara Boxer up 5 points over Carly Fiorina in California in its final poll; Boxer’s actual margin was 9.3%. Rasmussen predicted a 3 point spread for Boxer in its final poll. RCP predicted Ken Buck would beat Michael Bennet in Colorado by 3 points. Bennet won by a percentage point. Rasmussen had Buck up by 4 in its last poll. In Pennsylvania, RCP had Toomey up 4.5 points over Sestak. Toomey’s margin of victory was 2 points. RCP had Joe Manchin ahead by 4.5 points in its poll average for West Virginia. Manchin beat Raese by 10.1 points. In Nevada, RCP had Sharon Angle ahead by 2.7% in its final poll average. Harry Reid won by 5.6%. In Illinois, RCP had Republican Kirk ahead by 3.3% in its final average. His winning margin was 1.9%. In Wisconsin, RCP had Johnson up 7.7 points over Russ Feingold in its final average. Johnson’s margin was 4.9%.
Gubernatorial Races
In Massachusetts, the RCP average had Patrick up 4.5% over Baker in its final average. Rasmussen showed Patrick just 2 points ahead. Patrick’s actual winning margin was 6.3%. In California, the RCP average had Brown up by 6.6%. Brown beat Whitman by 12.2%. Rasmussen had Brown up by 4. In Colorado, the RCP average had Hickenlooper up by 4%. He beat his nearest competitor, Trancredo, by 13.9%. In South Carolina, the RCP average had Haley up by 10.7%. Her winning margin was 4.3%. In Maryland, RCP had Democrat O’Malley up by 10.8%. He won by 13.5%. (Rasmussen had O’Malley up by 10%.)
RCP was pretty accurate in New Hampshire and Ohio, and overestimated the Democrat’s margin of victory in Minnesota. (Lynch won by 8 points and Kasich beat Strickland by 2.7%. Dayton squeaked by with a half percentage point margin in Minnesota.) It’s interesting to note that Rasmussen had Kasich winning by 4 % in its last poll, and it had Lynch up by 6 points.
Overall, it was a particularly bad day for Rasmussen, which generates a lot of polls and thus heavily influences RCP averages. It’ll be interesting to see what Pew has to say, but I expect that its report will come down pretty hard on the robo-polling orgs like Rasmussen, which cannot reach either cell phone users or non-English-speakers.
Last weekend, I was asked whether I agreed or disagreed with the following statement: “The country is headed in the right direction.”
<
p>There is simply no way to distinguish the following two opposite feelings:
<
p>
<
p>I passionately believe that the country is headed for utter disaster. I equally passionately believe that Barrack Obama and the Democratic Party is our only hope for a non-violent avoidance of that disaster.
<
p>The media is full of polls that accurately (I suspect) report that Americans oppose the health care reform bill. What nobody seems to report is the number of those who oppose it because it doesn’t go nearly far enough towards government-sponsored single-payer health care.
“Do you think the country/state is headed in the right direction or on the wrong track?”
<
p>We are clearly still in dire straits, but the operative word is “headed”. I’ve told polsters this cycle that I strongly believe under our President and Governor we are HEADED in the right direction (albeit not at our destination yet).
<
p>On health care, the question I didn’t like was, “Do you approve of the way the President is handling the health care issue?” A negative answer to this question was interpreted as opposition to the President’s ideas on this matter. However, I also was tempted to answer in the negative because I felt he wasn’t fighting hard enough for such things as the public option.
The media and others constantly remark how Congress’ approval rating is always horribly low, and I always shake my head because it’s a metric that is, I believe, misinterpreted. If the question is “Do you approve or disapprove of the job Congress is doing?”, the results to me mean pretty much nothing.
<
p>The reason is the same. “Congress” is made up of the House and Senate and 100 individual senators and 435 individual representatives. Congress as a singular entity does not act in a unilateral fashion – it’s full of chaos, conflicting votes, bills, press releases, etc. etc. Generally I think most people will always have a negative opinion of how the “Congress” is performing. I think it is more useful to ask about the performance of individual members (like Dem/GOP leadership) to get an idea of what people think about general legislative agendas.
<
p>Of course, there still remains the problem of people agreeing in a poll for very different reasons. In a theoretical poll, an ultra right-wing conservative and an ultra left-wing liberal may both say they “disapprove” of Congress, or the Dem Congressional leadership, or Obama – but of course for very different reasons. But they get lumped together in the poll, and it becomes misleading to people trying to interpret or spin poll results.
<
p>Similarly (and I know I’m beginning to ramble), it irks me that the media and others lump all “independent” or “unenrolled” voters into one voting bloc, when in reality these voters generally vote with one party or the other, and most definitely do not represent a homogenous political ideology, agenda or persuasion. To treat them as a voting bloc on par with registered Dems or Republicans is just incorrect, but it happens all the time.
<
p>And then I found 50 bucks…
I seem to recall Bush outperforming the polls against Kerry in 2004. Possibly the landline effect explains things shifting the other way.
Pollsters have been seeing the landline problem coming for years, but until recently they didn’t think it was widespread enough to significantly impact polling results because the people being missed by the polls were not a large and politically different enough group to do so. I think we may have reached the point where this is no longer true. As an under-35 voter, I can tell you I know plenty of people around my age who do not have landlines. I personally have never been called for a poll or received a political robocall or other contact on my (cell) phone, which is my only phone.
<
p>On the upside, all these polls will give the GOP false hope that will be crushed by the growing thousands (millions?) of cellphone-wielding Democratic voters on election days to come…right? It’s like an unorganized electoral sneak attack.
With the advent of caller ID, we (and many others) do not answer calls from numbers we do not know. I figure if it is important, they’ll leave a message. If they don’t, well, it is like the call never happened.
<
p>Credit John Walsh for forecasting this and doing something about it, as told in today’s Globe story
<
p>
…I do the same on my cell. If I had a landline with caller ID, very unlikely I’d pick up a poll call.
<
p>Speaking of phone calls, I hope Tim Kaine had a nice long phone chat with John Walsh this week. I remember hearing the pitch about Patrick’s online/card voter ID system with personal contact networks as the central mechanism and thinking “wow, what a great idea – why hasn’t anyone done this before?”. Probably because it takes a lot of work, and could fall short and waste a lot of time and effort.
<
p>Clearly, Walsh and the Dems made this work, and there are a lot of downticket Dem candidates who should be sending them a nice thank-you giftbasket over the holidays, especially Bump.
Western New England called on my land line a few days before the Governor’s election. I didn’t know who they were on my caller ID, so I didn’t pick it up. We usually screen calls that way, only picking up when it’s someone we know.
<
p>There were so many robocalls this election that we just assumed it was another call from Tom Wesley.
One of the most infamously incorrect polls in history was the one conducted that year by Literary Digest, which until then had a perfect track record of predicting the winner of the presidential election. Alf Landon beat FDR in a poll which used telephone lists in an era where not everyone had a home telephone and those who did were wealthier and not big fans of the New Deal to say the least. It looks like we’re potentially again seeing a case where use of technology is not representative of the population as a whole. As for 1936, FDR won a huge popular landslide and every state but Maine and Vermont.
….says it all, thanks Bob.
We need elections not because they do a better job at determining the majority opinion, but because we need a decisive and authoritative method. Elections self-select a subset of motivated citizens who are able to vote that day, which can be manipulated by GOTV campaigns. I’ll take a proper random sample over that any day.
GOTV campaigns=democracy in action. When results are manipulated by voting machines and Republican dirty tricks like phone jamming and blanketing black neighborhoods with leaflets telling them to vote on Wednesday instead of Tuesday, that is not democracy in action. Polls only sample likely voters, and sometimes only registered voters. There’s no way they can every predict with any accuracy, though some do come close.
They don’t determine majority opinion, but plurality opinion, as happened in the governor’s race. And if you want to take GOTV out of the equation, you could look at compulsory voting. What do you feel about that idea?
I’d be screaming bloody murder if I were not one of the ones sampled in this scenario. I’d love to get near-unanimous turnout, but GOTV is an assistance, not a manipulation. It’s not as if you can’t vote unless invited to by a campaign’s GOTV operation.
IF they were an assistance, they’d get out people regardless of their likely vote, but they try to only get out people they think will vote their way. They are a manipulation.
<
p>And the Greeks, who invented Democracy, didn’t think elections were Democratic, they thought (correctly) that they were Oligarchic, and they used sortition not elections, to ensure a true Democracy. Sortitions were random, like samples are supposed to be. They randomly chose their representatives, and that was essential to fairness:
<
p>And I suspect screaming bloody murder that you were not selected was considered uncivilized. I’m not suggesting that we randomly select our Reps and Senators, but merely pointing out that elections are not the holy grail of democracy, and putting too much stock in them, considering them sacred and unerring, is neglecting the point of government, which is not to give people what they want, but to preserve and strengthen civilization.
…”Just powers are derived from the consent of the governed” and not just a few randomly selected of the governed. I’m very familiar with the Athenian system you describe. It’s closer to how we select juries, but not appropriate IMO for legislative bodies. It would be great if the Secretary’s office did a general GOTV, but it’s also a legitimate campaign operation. If everyone just voted without prompting it wouldn’t be necessary, but you drive out your vote if you want to win. As long as all candidates and parties are allowed to do it there’s absolutely nothing wrong with it.
Here are a couple other wonk sites:
<
p>Stochastic Democracy
Princeton Election Consortium
<
p>I’d rank them both well above RCP, personally, in terms of polling analysis. Probably above 538 as well (Nate Silver delivers steak plus a lot of sizzle).
I been curious about this cellphones-only vs. landlines effect for a while. I found this article about a SurveyUSA poll of the Kitzhaber (D)/Dudley (R) race for governor in Oregon before the election.
<
p>SurveyUSA Poll results
<
p>
<
p>Below are the final results. So it looks like the cellphone plus landlines poll was an outlier and wrong – cellphone-only households may lean Democrat but they don’t vote. A SurveyUSA poll a week earlier that used only landlines had the race in a dead heat – much more accurate. Speaking of outliers – check out Rasmussen.
<
p>Final Results Kitzhaber – 49.1 Dudley – 48.2
RCP Avg. Kitzhaber – 46.7 Dudley – 44.3
Portland Trib Kitzhaber – 46 Dudley – 43
SurveyUSA Kitzhaber – 48 Dudley – 41
Rasmussen Kitzhaber – 46 Dudley – 49
In the last poll before the election, SurveyUSA used both landlines and cellphone-only households in their sample.
<
p>“So it looks like the cellphone plus landlines poll was an outlier and wrong – cellphone-only households may lean Democrat but they don’t vote.”
<
p>Here’s the SurveyUSA poll from the week before (Oct. 15th) using only landlines.
<
p>
the accuracy of a poll is determined by their likely voter model. That is why the polls differ. Each polling organization determines this model differently. The closer the likely voter model is to the actual turnout in terms of demographics the more accurate the poll is.
<
p>If people with cellphones only don’t vote, then they should be discounted in a good model. If they turn out then they should be included more heavily. But to exclude cellphone only people completely is highly inaccurate and biased. This is Silver’s point, and it’s a good one.
that included cellphone-only households in the sample. Are there others that you know of? More details about the samples used on these two polls are on the SurveyUSA link. Perhaps, there is a clue there about why the results were so skewed.
<
p>It could be that cellphone-only households are less likely to vote or that cellphone-only households are mostly in urban areas or . . . name your variable. I have no idea what the likely voting statistics are for cellphone users vs. landline users. Maybe there is no difference. I am, by no means, advocating the exclusion of cellphone-users from poll samples. I’m just saying that, in this particular poll, the predictions were much less accurate in the cellphone+landline poll than in a poll by the same organization conducted a week earlier which used only landlines.
I guess it’s a sickness on my part – largely Schadenfreude – but I enjoyed the fabulists and fantasists taking polls as proof. Of course, we knew what Baker and various Herald writers would do with internal and external polls showing a clean sweep for the GOP of MA commonwealth-wide offices.
<
p>Not only was Baker himself particularly pathetic in believing and touting these, but by far the worst prognosticator based on them was RMG’s Eno. His calls two days before the election show one danger of being in the delusion-based community.
<
p>Such failures won’t stop people from bringing them on talk shows or quoting them. Then again, that’s entertainment, boys and girls!