Downloadpdf (PDF: 103 KB)

Random Sampling

  • NAR member surveys are typically sent out to a randomly-selected group of members – generally 50,000.
  • This ensures that the survey sample represents the larger member population.

Response Rate

  • Response rate is calculated by dividing the number of responses by your total sample size (everyone the survey was sent to).
  • A standard response rate for our random-sampled surveys is 5%.

Margin of Error

  • The margin of error is the statistic that expresses the amount of random sampling error in a study.
  • The larger the error, the less confidence you should have in results being representative.
  • For example, with a MOE of +/-7, if results show that 10% of people like a program, you can confidently say that anywhere between 3% and 17% of your entire population feel that way.
  • We aim for margins of error around +/-3%, but typically cut off reliability at +/-5%.
Illustration: Response rate plus margin of error determine reliability of survey results

Key Takeaways

  • When looking at survey results, it’s important to look at both the response rate and the margin of error, as they combine to paint a picture of how reliable and representative the sample is. Non-response bias is already addressed by random sampling.
  • Smaller overall groups need higher response rates in order to be representative.
Advertisement