In the last column, we began a discussion of good practices in survey research, building on the insights provided by Floyd Fowler in his excellent books on writing and administering surveys.
Mr. Fowler makes some very specific recommendations for wording survey responses based on his many years of experience and his review of research on collecting unbiased, accurate information. When discussing asking people to indicate the frequency of something, one of the sets of response options he provides is: Always, Usually, Sometimes, Rarely, Never. Allowing people completing a survey to select one of these answers makes both answering the question posed, and interpreting the data collected, more straightforward.
When asking people their feelings about something, things get a little trickier. A set of verbal options that seems to work well, according to Fowler, is Delighted, Pleased, Mostly satisfied, Mixed, Mostly dissatisfied, Unhappy and Terrible. Arguments can be made for other response options, such as a continuum of cartoon facial expressions (think: a range of smiley faces, some much less “smiley” than others).
Asking people about their satisfaction with something can be a little trickier. It’s important to decide if you want to provide a midpoint among the answer options, essentially allowing people to be equally satisfied and dissatisfied. Some survey writers believe that there are few issues on which people are equally satisfied and dissatisfied, and that providing this option makes it easier psychologically for people to select responses to questions without really thinking about what they’re doing. Others argue that it’s possible that some people could truly be “middle option choosers” based on their satisfaction level, and not providing a midpoint for the scale distorts the data collected because it forces them to pick an answer that doesn’t align with how they feel. A set of response options Fowler presents with a mid-point is: Completely satisfied, Mostly satisfied, Mixed-equally satisfied and dissatisfied, Mostly dissatisfied, Completely dissatisfied.
It is important to reiterate that many words that might be used in survey questions may need to be defined. The best examples are terms that refer to spaces where people work. A workstation might be a work area, an office, a workspace or workplace, or something else. Whether people are being asked about the space where they sit most of the time to work or where they have been assigned a seat or for a whole complex of thousands of offices needs to be clear, for example. Definitions of options, repeated frequently, provide that clarity. Also, it’s easy for technical terms to make their way into a survey because all pre-use reviewers of a survey may be familiar with them. To most, “circulation” is something blood does, not people, for example. Need to decide if a term needs to be defined? Call a few people in an entirely different field from yours who will be frank with you. Ask them what they think a word means. This gives you another reason to be glad your brother is an accountant and your sister is a doctor.
The number of people who need to answer your questions is related to the analyses you’ve planned for the data after it’s collected. You might want to do some types of tests on the information collected from the accounting team and other tests on material collected from the whole organization. Googling the name of the statistical tests planned along with text such as “sample size needed” produces (in all instances tested) information on sample sizes required for a test to be meaningful.
“Googling the name of the statistical tests planned along with text such as “sample size needed” produces (in all instances tested) information on sample sizes required for a test to be meaningful.”
Writing a survey that collects information that accurately represents a situation of interest requires attention to many details, as described in this article and the previous one on this topic. All of the upfront effort, however, increases the value of the insights derived from questions asked.
Note: Information on survey response options was drawn from Floyd Fowler’s book Improving Survey Questions, Design and Evaluation, published in 1995 by Sage.
Sally Augustin, PhD, a cognitive scientist, is the editor of Research Design Connections (www.researchdesignconnections.com), a monthly subscription newsletter and free daily blog, where recent and classic research in the social, design, and physical sciences that can inform designers’ work are presented in straightforward language. Readers learn about the latest research findings immediately, before they’re available elsewhere. Sally, who is a Fellow of the American Psychological Association, is also the author of Place Advantage: Applied Psychology for Interior Architecture (Wiley, 2009) and, with Cindy Coleman, The Designer’s Guide to Doing Research: Applying Knowledge to Inform Design (Wiley, 2012). She is a principal at Design With Science (www.designwithscience.com) and can be reached at sallyaugustin@designwithscience.com.