| I should clarify that I’m talking not talking about the use of robo-calls to deliver messages or solicit support – that’s already in wide use. (“Over-use” may be more precise.) My topic is the use of robo-calls to assess whether a given race is competitive or one candidate has established a lead. That is, to decide if more money would make a difference, whether it’s time to stand pat with a winning hand – or pull the plug on a loser. Campaigns, and their backers, have to make such decisions every day, but they often have nothing but guesses to guide them. Big campaigns use “real” polling, but it doesn’t make sense for a candidate spending $10,000 of his own money to run for township supervisor to spend $5000 of it on polling. Better to put the money into another round of mail, with the result that the day after the election he’s likely to look back and find it was wasted in a blowout – or that he never really had a chance.My firm, PPC, has conducted over fifty of these cheap little polls, and we’ve learned to accept a number of limitations. For one thing, we limit ourselves to a handful of questions – no more than six or eight. We don’t ask voters about their views on issues, nor do we try to probe for “meaning” behind their choices. Instead, using a narrowly selected list, we dial (say) 1000 likely Democratic primary voters, and ask “When you vote next month for county commissioner, if you plan to vote for Bev Baten, press one. If you’ll vote for Carol Koenig, press two.” Then we say “thank you” and we hang up. (This is a real example for this August: Ingham County, district 9.)We’ve been working with Voiceshot.com, which offers a simple (“crude”) interface for recording the choices and downloading the call results. I’m sure there are other firms offering similar services, but the ones we’ve explored haven’t seemed like an improvement over Voiceshot, so we’ve stayed put. The cost is 12 cents per “completed” call, which unfortunately includes reaching voicemail. Typically, of 1000 numbers dialed, we are charged for 600 ($72), and we get useful data from between 100 and 150 repondents, depending on how visible the race is. (In the case of the county commission race between Baten and Koenig, we got about 100 completes, in spite of the fact neither was well known or had done much campaigning by the time of the poll.) PPC’s charge for conducting such a poll is $400, including the list and a small amount of analysis. There’s no reason a campaign couldn’t conduct a similar effort using volunteers for under $100.New techniques always start out in the shadow of older techniques, and their initial use is sub-optimal because it’s assumed they are direct replacements. Only after considerable use do their peculiar stengths and weaknesses become apparent, steering their use away from outdated convention. In the case of micro-polls, PPC has moved away from the kinds of samples used in more conventional polling. Our samples are generally well-defined, rather than broadly representative. Another difference is size – we generally aim for 100 to 200 completes, rather than 400 or more.The reasons arise from experience, not theory, so I don’t feel a need to justify these practices, but I can try to explain them. First, although robo-calls start with a refusal rate no worse than conventional polling, each additional question causes people to hang up. Not only does this reduce sample size, but it raises questions of comparability between the early questions and the final ones. So PPC doesn’t ask if people are (say) Democratic primary voters – we select such households in the first place and procede directly to the paydirt questions. We also worry that people may “qualify by lying”, that is, some may press whatever buttons seem to be indicated in order to hear the entire poll, resulting in a contaminated sample. So instead of asking people “Do you vote in nearly every election?”, we ask the database instead.Because we know who answered the questions (actually, we know which household, but not which individual voter) we can tease out crude cross-tabs, looking for major correlations between the answers and the data associated with the phone number in our database. In the Baten-Koenig race, not finding anything striking was a bit of a surprise, since Baten has been involved in East Lansing politics for decades. With older voters leaning slightly more toward Koenig than younger ones, it was clear Baten doesn’t have any residual support. (The Koenig strength among older voters was non-significant, but it serves to dispell the opposite possibility.)We aim for fewer completes than conventional polling mainly because we don’t fully trust the results anyway. Completing 100 calls gives us a “margin of error” of about +/- 10%, while 400 would improve that to +/- 5%. But I don’t trust robo-polling to give me such accuracy, regardless of the sample size – there are simply too many risks of bias. So what’s the point of paying for the additional calls? I figure micro-polling has an inherent accuracy, if well-conducted, of about +/- 10%. If somebody needs greater accuracy, they should feel free to spend ten times as much on a real poll.Another, very practical, reason for accepting the smaller sample size is that in small races we simply can’t GET any more data. In the case of the county commissioner example above, we literally called everybody who met the qualifications for whom we had a phone number. Most city and township offices present similar problems.Micro-polling works best if we combine the horserace result with a generous dollop of local knowledge. In the case of the Baten-Koenig commissioner race, the poll was conducted before either candidate had done much campaigning, but we know that Bev Baten was a moderately conservative member of the East Lansing city council for 8 years, and lost re-election in 2007, largely because she was regarded as ineffectual and negative. Carol Koenig is not well known and has never run for office before. When the poll results came back with Koenig leading 61% to 39%, it was obvious that Koenig is in a commanding position and merely needs to run a competent campaign to win easily. That’s perfectly obvious now, but it wasn’t obvious before we ran the poll. We might have found voter sympathy for Baten, or a wellspring of support among older Democrats in a low-turnout election, or that she was beloved by her immediate neighbors – none of which appear in the results.Micro-polling isn’t useful – at least in PPC’s hands – for assessing issues, or for finetuning campaign tactics. Its real strength is where you find yourself wondering what’s going on among a well-defined universe, and where the answer is needed to make large strategic decisions.Five years from now, I bet it’s ubiquitous. |