- Joined
- Oct 30, 2015

- Threads
- 18

- Messages
- 436

- Likes
- 430

This is an OP I posted elsewhere some time ago, but I think it's worth repeating.

Every time there’s a new poll published, political forums go into an entirely predictable pattern of posting:

Great, accurate poll. My party is doing well.

Crap, unreliable poll. My party is not doing well.

To a certain extent, this is understandable, but could be avoided by standing back and asking a few simple questions.

If you want to know if a pot of soup needs more salt, you just need to stir it well and then taste a spoonful, not eat the whole pot. If you want to gauge the opinion of an entire population, you don’t have to ask everyone, just a representative sample. Quota samples will reflect the composition of the larger pot by pre-selecting quotas of groups to ask (gender, age, social class, geography, etc.) to fit the census breakdown along those lines. So, if 30% of the population are working-class women over 40, then the sample should reflect this. Random samples will just ask 1,000 (or whatever the sample size is going to be) people, and then adjust to reflect. So, if they find that only 200 of the sample are working-class women over 40 but it should be 220, then they weigh the responses of that 200 so that each of them counts as 1.1 person. Small representative samples will always be better than big, self-selecting ones.

Then they work out the margin of error (MoE), which is always an approximation as it is based on an ideal situation. For example, statistical theory confirms that for a sample of 1,000 with a 100% response rate on a question where the population is split 50/50, the poll will have an MoE of +/- 3% 19 times out of 20. Clearly, this never happens, but the theory acts as a guideline for an MoE that isn’t too far off.

Despite what some people think, genuine, serious polling companies have no interest in being wrong, so polls that are conducted by one of them are more likely to be right than polls conducted by randomwebsire.ru. Equally, most serious media organisations want to be as accurate as possible, so, like them or not, a poll commissioned by RTE is more likely to be accurate than one commissioned by a company that wants to sell a product.

So, why do polls on, say, voter intention in Ireland, that are published within a week of each other often show very different trends? We’re back to these questions:

Clearly, the question asked will have a strong influence. Just think about the difference between being asked ‘Which party is currently doing the best job?’ and ‘Which party would you vote for if there were a GE tomorrow?

Equally, time plays a role. Would you expect the same response rate the day before a big scandal as you’d get the day after?

So, most polls are accurate enough to within the MoE for the questions that were asked at the time they were asked. If you don’t like the result, then the problem is probably more with you than with the poll. And always remember, polls are not predictors of election results, they are samples of how salty the soup is at the moment you test it.

If you’d like more detail, this link is very useful:

A Journalist

Every time there’s a new poll published, political forums go into an entirely predictable pattern of posting:

Great, accurate poll. My party is doing well.

Crap, unreliable poll. My party is not doing well.

To a certain extent, this is understandable, but could be avoided by standing back and asking a few simple questions.

- What questions are asked?
- When?
- What is the sample size?
- How was the sample selected (random, quota, self-selecting)?
- What was the mode (face-to-face, phone, e-mail, other)?
- What is the margin of error?
- Who did it?
- Who paid for it?

If you want to know if a pot of soup needs more salt, you just need to stir it well and then taste a spoonful, not eat the whole pot. If you want to gauge the opinion of an entire population, you don’t have to ask everyone, just a representative sample. Quota samples will reflect the composition of the larger pot by pre-selecting quotas of groups to ask (gender, age, social class, geography, etc.) to fit the census breakdown along those lines. So, if 30% of the population are working-class women over 40, then the sample should reflect this. Random samples will just ask 1,000 (or whatever the sample size is going to be) people, and then adjust to reflect. So, if they find that only 200 of the sample are working-class women over 40 but it should be 220, then they weigh the responses of that 200 so that each of them counts as 1.1 person. Small representative samples will always be better than big, self-selecting ones.

Then they work out the margin of error (MoE), which is always an approximation as it is based on an ideal situation. For example, statistical theory confirms that for a sample of 1,000 with a 100% response rate on a question where the population is split 50/50, the poll will have an MoE of +/- 3% 19 times out of 20. Clearly, this never happens, but the theory acts as a guideline for an MoE that isn’t too far off.

Despite what some people think, genuine, serious polling companies have no interest in being wrong, so polls that are conducted by one of them are more likely to be right than polls conducted by randomwebsire.ru. Equally, most serious media organisations want to be as accurate as possible, so, like them or not, a poll commissioned by RTE is more likely to be accurate than one commissioned by a company that wants to sell a product.

So, why do polls on, say, voter intention in Ireland, that are published within a week of each other often show very different trends? We’re back to these questions:

- What questions are asked?
- When?

Clearly, the question asked will have a strong influence. Just think about the difference between being asked ‘Which party is currently doing the best job?’ and ‘Which party would you vote for if there were a GE tomorrow?

Equally, time plays a role. Would you expect the same response rate the day before a big scandal as you’d get the day after?

So, most polls are accurate enough to within the MoE for the questions that were asked at the time they were asked. If you don’t like the result, then the problem is probably more with you than with the poll. And always remember, polls are not predictors of election results, they are samples of how salty the soup is at the moment you test it.

If you’d like more detail, this link is very useful:

A Journalist