How Polls Should Be Interpreted, or Not
1 General Advice
Polls do not tell us what’s going to happen on
election day. Especially polls taken months in advance. Typical polls
don’t even ask about election day; instead they ask what would happen
“if the election were held tomorrow” which of course it won’t be. A
lot can change between now and November.
Polling data may give us a general outline of what
“could” happen. However, even if the outline is correct, the
details probably won’t be. The more detailed the prediction, the less
likely it is. See item 7 below.
Polling data may give us some idea of where things
stand at the moment. Recent trends may shed some light on which
tactics are working and which are not.
Do not over-react to the results of a single poll.
The average over many polls (here, or here) is much more meaningful.
Anybody who looks at poll numbers and
grows complacent (or, conversely, despondent) is a fool. The fact is,
issues matter, candidates matter, and campaigns matter. The whole
point of campaigning is to cause a shift in public
In particular, it is important to contest every race. Vigorously.
Even if you don’t win this time around, you lay the groundwork for
winning in some later cycle. You have to win the argument before you
can win the election.
Some politicians seem to have given up on trying to shift public
opinion; instead they obsess over turning out their base and/or
suppressing the opposing base. I consider this political malpractice
on a epic scale. It’s not a winning strategy in the long run. And
even if it works in the short run, it damages the foundations of
There is such a thing as poll porn. Every so often
we are allowed to take a break from the trench warfare and enjoy some
poll porn. Don’t take it too seriously.
Polling data (along with historical data) can be fed
into statistical models that try to predict the outcome of the
election. This is a very, very complicated undertaking. It involves
considering thousands or even millions of possible scenarios. Nate
Silver knows how to do this. Very few other people do.
: Modeling Possible Election Outcomes
In 2016, the raw polling data was correct. The problem
is, a great many people misinterpreted the polls, including people who
should have known better (e.g. the Hillary campaign) and people who
had no business interpreting polls at all (e.g. James Comey). A few
people got it right, including Nate Silver. I knew he was right,
partly based on my own models, and (!) because he showed his work. I
said so at the time, as loudly as I could.
There is a huge market for “internal” polling,
which is notoriously rife with abuse. In contrast, most public
pollsters have a brand and a reputation that they need to defend.
Before reading any poll, check the pollster’s
rating. Over time, bias and/or will be detected.
markets have a fair bit of predictive value. Unlike polls, they ask
questions that focus on the real election (rather than the
counterfactual “if the election were held tomorrow”).
The action item is clear: Run up the score. Run
it up everywhere. Run it up so high that no “polling miss” will
make a difference, and no amount of cheating will make a difference.
Flip the senate and the state legislatures also.
2 Sources of Information
There are tremendously many pollsters. To find out which are
reputable and which are not, check the FiveThirtyEight Pollster Ratings.
It’s painful to track down the primary source for each poll, so you
may find it helpful to rely on aggregators. Most of them
allow you to filter out which race(s) you are interested in:
presidental primary, general election popular vote, presidential
state-by-state, senate, or whatever. Some good, widely-cited
In addition, we have pundits, who look at polling data and
mix in their opinions. For example, in mid-2020, the Biden was ahead
by several points in Florida, but most pundits rated it a toss-up,
because the state party is so screwed up that they were likely to find
a way to blow it.
A useful aggregation of aggregators and pundits, with a convenient way
to visualise the results, can be found at 270ToWin. There is a pulldown that
allows you to select which data-source you want to map out.
Then there are the prediction markets: