That poll, Cohen says, was message testing, not voting. “It didn't have Sonia winning, but it did show that she could close the gap” that she started with as an unknown. Because Wilkerson hadn't bothered to collect the 300 signatures to get on the ballot, both ran sticker campaigns. Chang-Díaz came within 6% points, about 700 votes, short. She did, in fact, close the gap, but still lost.
Even there, Cohen rues the short campaign time and resulting lack of people on the ground. He says that at many polling places, the Chang-Díaz camp did not have enough workers to be there to provide stickers for voters entering.
Since her loss, Chang-Díaz has not stopped campaigning, according to Cohen. She continued to raise money and speak publicly, as well as work to put a solid organization together. “In a lot of ways, this has been a two-year campaign for her,” he says.
Leaving out the favorability and equally subjective question of whether Wilkerson deserves re-election, the how-I'll-vote question appears for the first time in this poll. The chart here shows this month's figures.
The Chang-Díaz folk claim a combined lead on voters who lean/are probable/definitely voters for the candidates. Those are Chang-Diaz 47.2%, Wilkerson 29.7% and undecided 23%.
That combines the undecided with a margin of error of 4.8%. Cohen is quick to say he wants Chang-Diaz to win but knows that four weeks before a party primary is a long time in politics. Many things could happen. “I can't say Sonia's definitely going to win, but I like the results,” he adds.
In his polling of 417 likely Dem voters, he broke down the respondents many ways, including into four major neighborhood divisions (South End/Back Bay, Roxbury/Dorchester, Fenway/Mission Hill, and Jamaica Plain). He sees strong support for Sonia even where she does not have a solid lead in one place. He is sure there won't be runaway leads for Wilkerson in any one place as there were last time.
He also confirmed what many of us have long suspected, that the insider party buzz bears little relationship to what voters think and how they act. As Cohen puts it, “The Ward Committees are totally disconnected from the voters.” He cited the South End, where Chang-Díaz led five-to-one, but the two committees endorsed Wilkerson.
“The insider chatter has almost no relationship to the voters,” he adds.
Mike, you’re trying to get me in trouble again =p
<
p>A correction of your diary, though: You posted “he says” below the paragraph about me, which may not be an obvious mistake to most readers. Ah, the dangers of cross posting =)
<
p>But, yes, you posted a solid post. Here’s my thoughts on it in full.
True enough. I made it clear.
<
p>I had added the paragraph largely to link to the existing BMG post on this. I’m always willing to get to, or me, in trouble, but this was unintentional.
click here for more, but here’s a taste:
<
p>
Thanks for the info. That further fleshes out the picture, although as Dan says, it doesn’t say who will win.
<
p>As you well know, we’ve been trying to get Dianne to join us for a Left Ahead! podcast. She scheduled, then postponed with no firm date. Even though I favor a change to Sonia, I’d love for Dianne to come on and pitch her re-election. We don’t spring trick questions or surprise antagonists on guests, so she’d have the mic.
<
p>Cynics have said she won’t risk defending herself and that I am naive to suppose otherwise. That’s looking likely now, but we have four weeks to go…
to see if I wanted to talk to him about the results. I haven’t had a chance to call him yet, as I am quite curious. I do appreciate his note very much.
<
p>Still, I stand by my comments. You start with a selection bias since no candidate releases an unfavorable poll. After that though, the crosstabs matter big time, and without (a) looking at them and (b) having a good enough sense for the district and the trends and the particulars of history and current media, there’s no way to know if the turnout model is at all reasonable.
<
p>Furthermore, since the pollster is being paid by the candidate, there’s no way to know if there was influence. This doesn’t mean that Sonia bribed Dan or anything like that; still we’re all human and it’s hard to “forget” just who is buttering our bread.
<
p>I hope to get in touch with Dan over the next day or so, but I doubt I’ll be able to reach any other conclusion since I don’t know the district nearly enough to know if the crosstabs are any good. On top of that, if the crosstabs are complex [four ethnicities, three levels of income/wealth, four age levels, etc etc] as I would expect them to be for a diverse region like (I think!) that district, then a sample size of only 417 is a bit worrisome, although at least that statistical error shows up in the study [MOE], unlike some of these other ones which are outside of the statistics themselves.
If any of the voters in Suffolk 12 are like me, they do not answer their telephones if it is an unrecognized number on the caller i.d. Not that everybody has caller i.d., but most people have an answering machine, and I usually let collectors and telemarketers leave a message. Also, more people than ever before rely on cellphones exclusively for their personal business- or so I’ve heard. Is it assumed that a tendency not to want to talk to pollsters, or even to pick up the phone, is distributed evenly among Wilkerson and Chang-Diaz supporters?
<
p>P.S: Although opposing Wilkerson’s reelection, Yvonne Abraham indicated in her recent column (which has her actually visiting some of the less gentrified areas of Ward 12) that support for Wilkerson still appears strong in some quarters, particularly among “politically connected” expectorators:
<
p>
I have no idea how the W v. SD pollster handles it, but in general here’s what I know…
<
p>
<
p>The question is, is there any reason to think that Wilkerson supporters are any different than Chang Diaz supporters to behave this way? As long as the pollster is making sure that he’s got the right amount of responses for each demographic, this problem increases his costs but probably doesn’t cost him any accuracy.
<
p>
<
p>This is more difficult, because the people who tend to use cellphones exclusively are the young, the poor, and those who have moved or expect to move a few times within a few years. Aside from the bias of not being nearly as likely to vote at all, are there any other biases here? Tough to say. My gut is that these folks are less likely to vote for the incumbent than the population at large, but I’ve got no evidence for that. Ultimately though, even if it does create an error, it’d be on the order of a percent or two and wouldn’t account for the enormous lead this poll shows.
You’re a very smart, largely theoretically-focused guy (at least when it comes to polling, in which AFAIK you have no hands-on experience). Dan is also very smart, and is intensely focused on the practical. You will have a great conversation.
<
p>But come on — sample size of 417 worries you for a state Senate district? Major polling organizations routinely publish polls of around 1,000 “likely voters” that are supposed to represent the entire country. Sample size should be the least of your worries.
The size of the population you’re polling has nothing to do with the needed sample size to get good results. What does matter is the complexity of the sample you’re polling.
<
p>I’d bet [though I don’t know] that the complexity of the demographics in the senate district in question exceeds the complexity of demographics in most states.* The challenge is in making sure you’ve got enough samples in each “slice” of the data, and this data ought to have quite a number of different slices because the area is so complex.
<
p>
<
p>* consider the extreme example of Mississippi:
62% white, 37% black, <2% Hispanic. I’d bet that’s simpler than Suf-2.
1.4% foreign born. I’d bet that’s simpler than Suf-2.
3.6% Non-English spoken at home. I’d bet that’s simpler than Suf-2.
72.3% home ownership, almost all single family housing. I’d bet that’s simpler than Suf-2.
98.7% of the 82 counties in Mississippi are plurality [or majority] Baptist. I’d bet that if you divide Suf-2 into 82 sub-regions that the same religion won’t be plurality in 81 of ’em.
<
p>The demographics of Mississippi are likely to be far simpler than Suffolk-2, which is why a sample size of 417 is far more troubling for Suffolk-2 than it would be for Mississippi.
David’s right on this matter. Maybe the sample size should be higher, but it’s a problem with the entire industry, not this particular study. Still, though, most of these polls make for a strong snapshot in time. Dan’s certainly done himself justice in the interviews his done over it. I don’t know if I’ve ever seen a pollster subjected to this level of scrutiny to prove that his or her polls were accurate – and, yes, that includes campaign polling (just ask Eileen Donoghue, who released her campaign polls – showing her down! – during her campaign, not a single blogger or newspaper person questioned them, even though they showed she had tremendous momentum and was catching up… and was an internal poll they released).
I didn’t say Mississippi, or any other particular state. I said the entire nation. You can’t seriously contend that the demographics of a single, relatively small Senate district are more complex than those of the entire nation.
<
p>Further, Ryan is right that a sample size of 417 (which gives a MoE of right around +/- 5%) is industry standard for districts of this size. Nothing peculiar there. So, again, you have done nothing to show that this poll is uniquely untrustworthy due to sample size.
<
p>Seriously, talk to Dan.
Totals by candidate preference – which is all the poll seems to be reporting (on the SCD website, anyway) – are more than covered by a sample of 417. It’s only if you want to start reporting the results for some subset of the sample that you run into the problems of complexity that stomv points out. For instance, it’s not clear that the pollsters could say something like “Among Hispanic voters…” or “Among voters 18-34…” because we don’t know how many of these voters are in the sample.
<
p>If the sample was stratified to ensure that it contained representative proportions of the sexes, races, ethnicties, ages, and other characteristics of interest, then the poll could credibly report results along these lines. But because there could be, for all we know, two Hispanic persons in the sample, a crosstab of candidate preference by Hispanic ethnicity wouldn’t fly, owing to the fact that the sample in this case would effectively be 2, and the standard error would blow away the mean. I highly doubt they went to the trouble of stratifying the sample for a state senate race, and in any case n=417 is likely insufficient to generate a stratified sample truly representative of the diversity of the district.
<
p>But since all they’re giving us is totals by candidate preference, the sample size of 417 is fine.
The public would need to know what questions came before the relevant ones, in order to give these numbers any weight. It’s a given that if there’s ten questions about Wilkerson’s problems and Sonia’s positives and then you get to the “who you’ll vote” for question, you get different results then just asked asked straight.
<
p>Also, it’s not touted in the press release or the interviews but, looking at the graph of the results, the seems to say that only 16%-17% are definitely going to vote for Wilkerson?!? IMHO, that just doesn’t sound realistic.
<
p>The good news is, time will tell.