What good are polls except to fill the gogglebox?

The Wilson Quarterly surveys polls.

Despite unanimous poll results predicting an Obama victory in the New Hampshire democratic Primary, pollsters still determined Hillary Clinton was going to emerge the ­winner.

The New Hampshire debacle joined a list of major embarrassments that includes the disastrous Florida exit polling in the 2000 presidential election, which prompted several networks to project an Al Gore victory, and the national polls in the 1948 race, which led to perhaps the most famous headline in U.S. political history: “Dewey Defeats Truman.” After intense criticism for previous failures and equally intense efforts by pollsters to improve their techniques, this was not supposed to ­happen.

New Hampshire gave new life to many nagging doubts about polling and criticisms of its role in American politics. Are polls really accurate? Can surveys of small groups of people give a true reading of what a much larger group thinks? What about bias? Don’t pollsters stack the ­deck?

Iowahawk also chimes in on the margin of error of what is a very human process:

As a rule of thumb the margin of error of a sampled probability is:

Margin of Error = 1 / sqrt(n)

So if the sample size is 400, the margin of error is 1/20 = 5%... if the sample size is 1000, it's about 3%.

Obviously this works well for controlled experiments, defective widgets, jellybeans, and coloured balls in a jar. It's also good for undergrad exam questions. But what if the things you are studying don't quite fit the curriculum of QUAN101, what if the subjects are more wormy... say, like voters?

Using the analogy of coloured balls in an urn:

• What if 40% of the balls have personally chosen to live in an urn that you legally can't stick your hand into?
• What if 50% of the balls who live in the legal urn explicitly refuse to let you select them?
• What if the balls inside the urn are constantly interacting and talking and arguing with each other, and can decide to change their color on a whim?
• What if you have to rely on the balls to report their own color, and some unknown number are probably lying to you?
• What if you've been hired to count balls by a company who has endorsed blue as their favorite color?
• What if you have outsourced the urn-ball counting to part-time temp balls, most of whom happen to be blue?
• What if the balls inside the urn are listening to you counting out there, and it affects whether they want to be counted, and/or which color they want to be?

If one or more of the above statements are true, then the formula for margin of error simplifies to

Margin of Error = Who the hell knows?

Related Link: