If you do your homework before you start the survey process, you’ll find that surveys can save you time and money, and lead to the answers you’re seeking with regard to your members and the association’s strategic direction. View a sample survey: Florida REALTORS® membership surveydoc (DOC: 863 KB)

NAR’s Research team does occasional ad hoc research work for REALTOR® associations for a fee. If you’d like to discuss the options for this, please email surveys@nar.realtor.

If you need to undertake a survey on your own, NAR’s Research team offers the guidelines below.

Guidelines for Conducting a Survey

  • Ask yourself:
    • What specific business decisions do we need the results of this study to guide?
    • What are we going to do with this information? Again, be specific.
    • What is our objective for this survey? Phrase it, “To determine/measure X in order to Y.”
    • What questions are we trying to answer?
  • Determine your target audience. This will help you select the best way to design and distribute your survey. If you have more than one demographic, you may need more than one survey – or you may need to enlist a research professional to help you capture information you can analyze by all the necessary demographics.
  • Calculate how many people must respond to ensure the data you collect is statistically valid. Validity is based generally on response rate and final sample size, and assumes you have invited participation from either all of your members or a randomly-selected sample of them. The higher the response rate and the larger the sample size, the more reliable the data. A solid final sample size will ensure that any data analyses will provide a strong basis for good decision-making. A valid response level depends on the size of your membership. The smaller your membership, the greater the proportion of your members you need to hear from. For instance, if you have only 400 members, you would need to hear from at least 196 of them to achieve a reasonable margin of error (+/-5%). However, if you have 10,000 members, you only need to hear from 400 to get the same margin of error. Assuming you’ve surveyed all of your members, or a randomly selected sample of them, a good rule of thumb is you need at least 400 responses for 1,000 members or over, or at least 150 for less than 1,000.
  • Plan your timing. The timing of your survey launch can impact the response rate. A poorly chosen launch time can decrease the response percentage. Survey launch timing can also change the answers you get, and ultimately the decisions you make. Keep your survey open for at least a week to ensure that you pull in a broad range of people and opinions. And plan to send at least one reminder at a different time of day than you sent the original invitation before you close the survey.
  • Be careful how you introduce the survey. Disclosing the exact subject matter may bias your results by drawing in those most interested in (or least interested in) the subject matter. Be vague and general about the content of the survey.
  • Consider an incentive to increase responses. Include the people who return completed surveys in a drawing for a prize. Or offer a free copy of the (non-confidential) result highlights to those who complete the survey.

Surveying Methods

 Next, decide on your data collection method. Below, we discuss the unique advantages and disadvantages of several popular methods.

Web Surveys

This is by far the easiest, least costly option in most cases.

Advantages

  • Extremely fast and can gather several thousand responses within a few hours
  • If no analysis is needed, there is practically no cost involved once the setup is completed
  • You can show pictures; some web survey software can also show video and play sound
  • Can use complex question-skipping logic, randomization, and other features
  • Can use colors, fonts, and other formatting options
  • Many people will give more honest answers to questions about sensitive topics when giving their answers to a computer
  • Some web survey software can combine the answers with preexisting information about individuals taking a survey
  • Can be taken on a smart phone or a computer. Virtually all REALTORS® use both, so response bias among REALTORS® is minimal.

Disadvantages

  • May not reflect the population as a whole unless you are sending to the entire population of interest or to a randomly selected sample
  • People can easily quit in the middle of a lengthy questionnaire
  • If your survey pops up on a web page, you often have no control over who replies
  • Depending on your software, there is often no control over people responding multiple times to bias the results
  • Your survey invitation may bias the results if you give too much information about the nature of the survey

Personal Interviews

Advantages

  • Lets the interviewee see, feel, and/or taste a product
  • Longer personal interviews are sometimes better tolerated over the phone or via Zoom

Disadvantages

  • Can cost more per interview than other methods
  • Depending on the location, interviewees may not represent the target population, which may create a non-representative sample
  • These are best for QUALITATIVE, exploratory work—you're highly unlikely to be able to conduct enough to get a full, representative sample from in-person interviews

Telephone Surveys

Advantages

  • People are usually contacted faster over the telephone than with other methods
  • If using computer-assisted telephone interviewing (CATI), the results can be available minutes after completing the last interview
  • Skilled interviewers can often elicit longer or more complete answers, and ask for clarification of unclear responses
  • Some software can combine survey answers with pre-existing information

Disadvantages

  • Many telemarketers give legitimate research a bad name by claiming to do research when they start a sales call; consequently, many people screen calls or decline to participate
  • Few people are home during the day; this limits calling time to a window of about 6–9 p.m. (when dinner or a favorite TV program is interrupted)
  • You cannot show or sample products by phone
  • Requires many dials to complete a single interview, so cost can be higher than other methods

Mail Surveys

Advantages

  • Can be less expensive than personal or telephone interviews
  • The questionnaire can include pictures (not possible over the phone)
  • Respondents can answer at their leisure, so the survey seems less intrusive

Disadvantages

  • Takes longer to receive and process
  • In populations of lower education/literacy, response rates are often too small to be useful; even in well-educated populations, response rates vary from 3% to 90%
  • Printing and postage makes up fully half the cost of a mail survey

Scanning Questionnaires

Advantages

  • Can be the fastest method of data entry for paper questionnaires
  • A scanner is more accurate than a person in reading a properly completed questionnaire
  • Can be a good way to get information from attendees at a trade show or other event where you’d like them to fill out a quick form and return it onsite

Disadvantages

  • Best suited to "check the box"-type surveys and bar codes; scanning programs have various methods to deal with text responses, but all require additional data entry time
  • A scanner is less accurate than a person in reading a poorly marked questionnaire
  • Requires investment in additional hardware to do the actual scanning
  • As with mail surveys, requires investment in printing questionnaires

Summary of Survey Methods

Your choice of survey method will depend on several factors. These include:

  • Speed: Web surveys are the fastest, followed by telephone interviewing. Mail surveys are the slowest.
  • Cost: Personal interviews are the most expensive, followed by telephone and then mail. Web surveys are the least expensive for large samples.
  • Internet usage: Web page surveys offer significant advantages, but you may not be able to generalize their results to the population as a whole unless you receive an adequate number of responses and are certain the survey has been sent to your entire population.
  • Literacy levels: Illiterate and less-educated people rarely respond to mail surveys.
  • Sensitive questions: People are more likely to answer sensitive questions when interviewed directly by a computer.
  • Video, sound, graphics: A need to get reactions to video, music, or a picture limits your options. You can play a video on a web page, in a computer-direct interview, or in person. You can play music on a web page, in a computer-direct interview, in person, or over a telephone. You can show pictures on a web page, in a computer-direct interview, in person, and in a mail survey.
Advertisement