Sample Size for Online Surveys

The Big Sample Size Question: How many survey completions is enough?
A common question among people who are new to conducting surveys is how many survey completions do I need? There is no single answer to this question, but some simple principles which will allow a user to come to an appropriate answer.
First, lets go back to basics. What is a sample? A sample is data which is representative of the total audience. In most cases (except for some business to business surveys where the customer universe might be very small), it is simply not practical (or possible) to survey the total audience or population. That would be a census. A sample is a representative part of the total population.
So how big does the sample need to be? We need to understand who the audience is first, and the purpose of the survey. Lets say we want to understand how a representative sample of the UK population thinks about petrol prices in the UK. The sample is most likely to focus on drivers, although we may be interested in the total adult population.  There is one vital consideration here. If we want the survey to be nationally representative, and to be able to compare answers between gender and people in the different regions of the country, we will need a large sample.  In the UK there are 12 so called Government regions, which are used to structure national samples.  The table below shows the % of the population accounted for in each region, and the number of respondents that would be provided in a nationally representative survey of 1,000 survey completions.

Region

%

Number

North East

4.3

43

North West

11.4

114

Yorkshire/Humberside

8.5

85

East Midlands

7.1

71

West Midlands

9.0

90

East of England

9.1

91

Greater London

12.2

122

South East

13.7

137

South West

8.5

85

Scotland

8.7

87

Wales

5.2

52

Northern Ireland

2.8

28

When we are comparing groups of people (sub groups we need a minimum of 50 – 100 in consumer surveys and 30 – 50 in business to business surveys. In other words, as long as there are at least 50 – 100 responses per region, it is reasonable to compare the result.
Therefore, it is likely we will want a sample size of 1,000. This will be sufficient to  compare regions assuming we have interview completions spread approximately according to the table above. 
When do I need more than 2,000 survey completions?
You may have seen surveys conducted and publicized in the news, by polling companies or PR agencies.  Increasingly, a sample size of 2,000 has become a default setting. Why? Well, to a large extent journalists have come to the view that 2,000 interviews is some how more credible than 1,000 interviews. Infact, this is not automatically the case. If a poll is being conducted a sample of 1,000 is perfectly adequate for reporting “the national view”, assuming the 1,000 completions are taken from a nationally representative sample.  Some surveys are not nationally representative or are skewed to certain groups (e.g. women.). In these circumstances, even 2,000 responses, if not nationally representative, is likely to be no more robust than a sample of 1,000 nationally representative completions.
But there are times when 2,000 interviews would be useful. For example, if we are keen to compare income groups or gender within regions, we may need a higher number of interviews. If we specified 2,000 nationally representative interviews, our sample in the West Midlands, should increase to approximately 180 – more than enough to compare results by gender with confidence.
Is 1,000 or 2,000 the magic sample number? 
In general, a larger sample size is more desirable, but it is not always necessary.
If you need to run a nationally representative survey, and you have the budget, then 1,000 or even 2,000 survey completions is often advisable.  But it can be overkill. Infact, there is no magic number, no one size fits all, when it comes to sample size.
There is one piece of important information. It is not the size of the audience which determines the sample size, but the minimum level of variation for the sub groups we can tolerate which determines the sample size. So, if we want to compare all regions in the UK, it is likely we would need 1,000 interviews. But if wanted to compare one group, say Mums with Children between 5-12, it is likely that 200 – 400 completions would be sufficient. This would be sufficient to compare responses say between those with children between 5 – 8 and those with 9 – 12 (assuming we have structured the sample to represent both children’s age groups).
Do the same rules apply for business to business surveys?
Sample sizes for business to business surveys tend to be much smaller than consumer surveys. One reason is that business samples tend to be more homogeneous than is the case for consumer surveys. Finance Directors in large corporates tend to be relatively less diverse in opinion when talking about accounting software than say consumer preferences for holiday destinations. Its also true that the business “universe” is smaller. For example, the number of large companies (more than 1,000 employees in the UK) is fewer than 1,300 companies in 2008 according to the Office of National Statistics. Similarly, certain industry sectors may only have hundreds of companies. It is more likely that in business to business surveys, it is necessary to structure the sample by practical considerations. It may be possible to get 500 responses from HR managers across a broad range of business size and industry sector. But if you are trying to size the potential market spend in the oil and gas sector, it may be necessary to aim for a near census of perhaps a few dozen companies.
Most of us are not trained statisticians. Infact, most people designing and using market research are not trained statisticians. Deciding the right sample size is also, in the real world, about balancing practical considerations of what is possible, affordable and credible. But using a few basic principles, most people are capable of arriving at an appropriate answer for planning a reasonable sample size.

Surveygoo Books Available on Amazon and iBooks Store

Stacks Image 31309
Visit Amazon to buy | More info on Book
Stacks Image 31314
Visit iBooks Store to download Free

Start Using Our Survey Tool Now!

See Price Plans