Chapter 4. Determine the Data Collection Method
Chapter 4 in PDF format
Selecting the appropriate mode of data collection for a survey is dependent on three factors: (1) the objectives of the survey, (2) the concepts to be measured, and (3) the population. In addition, the timeframe available for data collection and the budget influence the mode of data collection. Survey instruments can be either self- administered or interviewer-administered. Data collection modes range from in-person, telephone, mail, to the Internet.1
The response rates often differ markedly by mode. In-person surveys usually produce the highest response rates, followed by telephone and mail, respectively. If a high response rate is desired, in-person or telephone surveys are the best data collection modes, but the costs are higher than for mail surveys.
When selecting the mode of data collection, it is also important to remember that the majority of respondents are older persons. For mail surveys, ensure that written questionnaires have a font size that is readable for older persons who may have vision problems. For in-person and telephone surveys, it is necessary for interviewers to speak clearly and to make sure that respondents are able to hear the questions. Additional information on interviewing older persons is in chapter 7.
Back to top
In-person surveys allow the interviewer to build rapport with the respondent. Often, respondents may feel ill at ease, and a well-trained interviewer can make them feel comfortable by clearly explaining the purpose of the study and the specific topics covered. Complex interviews are best administered in-person. Interviewers are able to probe for more complete responses when necessary and observe body language during the interview.
In-person surveys require interviewers who are trained on how to administer questionnaire items, their meaning, when to probe and how to respond to frequently asked questions. One limitation of in-person surveys is that respondents may give socially desirable responses, especially to questions on sensitive topics. Another consideration is that in-person surveys take time to administer, especially when respondents are not located in close proximity. Therefore, budgeting for an in-person survey should include costs for not only interviewer time for administering the questionnaire, but also for traveling and for travel expenses, such as gas and vehicle upkeep.
Back to top
Using telephone surveys allows multiple interviewers to conduct the survey in a relatively short period of time. Additionally, computer-assisted telephone surveys accommodate complicated skip patterns. Moreover, interviewers are available to respond to frequently asked questions and probe according to the data collection protocol. Quality control measures, such as monitoring interviewers during survey administration, ensure that interviewers adhere to the study protocol.
The optimal length of telephone interviews is 20 to 30 minutes. As with in-person surveys, telephone interviewers need training on how to administer the questionnaire, answer respondents’ questions, and how to handle difficult situations. Having to deal with missing or inaccurate phone numbers may increase the level of effort needed to complete a telephone survey. In recent years, there has been a diminishing return rate for telephone surveys due to screening calls. This is an important limitation that reduces the cost effectiveness of telephone surveys.
Back to top
Mail surveys are less intrusive than in-person or telephone surveys, especially when asking for sensitive information. Respondents are able to answer at a convenient time for them. Mail surveys are less expensive than in-person and telephone surveys. Telephone numbers are not necessary, but accurate lists of addresses are needed. With a mail survey, there is no interviewer support, but mail surveys often include a toll-free number for respondents to call for additional information. If doing a mail survey, insure that you provide time for respondents to return their completed interviews. In addition, nonrespondents can be called or sent postcard reminders to participate.
Some limitations of mail surveys include the following: they are not conducive to complex questions; there is no control over who really completes the questionnaire; and the turnaround time for completion is much longer than for in-person and telephone surveys. The response rates for mail surveys are often dramatically lower than for in-person and telephone surveys.
Back to top
Internet surveys have become more popular in the last few years. They have some advantages, such as the ability to handle complex skip patterns, and the use of visual stimuli to guide respondents, and elimination of data entry. While increasing numbers of older adults are using the internet, Older Americans Act service recipients who tend to be older, poorer, and frailer may be less likely than their healthier and wealthier counterparts to have such access. Internet surveys of individuals also experience low response rates, and those that do participate may not be representative of all persons of interest to the study. For example, those with a propensity to participate may have higher levels of income and education.
1 An excellent resource for conducting telephone, mail, internet, and mixed mode surveys is:
Dillman, D., Smyth, J. & Melani, L. (2008).
Internet, Mail and Mixed Mode Surveys.
Hoboken, New Jersey: John Wiley
Back to top