People are complicated. There are so many factors that influence how they’ll respond to any situation, and so quantified predictions of their behavior are unlikely to be accurate. There is a continuing parade of new influences on people’s thoughts and behaviors that cognitive scientists need to recognize when making projections.
After people’s responses to environmental changes are quantified, it’s unlikely that whatever differences in productivity or whatever else was found in that first measured setting will be precisely replicated in a second.
Here, we’re not talking about things like more energy efficient light bulbs that save x or y or z% on energy bills after humans decide to use them. We’re talking about assessments of things such as new workplace configurations that people toil in and where their rate of effective collaboration changes by c%. These are situations in which differences measured are deeply tied to human brains, not metal coils. And, as we’ll discuss in a future article, there are ways to calculate those changes in collaboration and other slippery factors that seem so elusive.
Robert Wachter wrote an intriguing article for the New York Times, published on January 16 and titled, “How Measurement Fails Doctors and Teachers.” In it he states, “The only way to understand whether a high mortality rate, or dropout rate, represents poor performance is to adequately appreciate all of the factors that contribute to these outcomes – physical and mental, social and environmental – and adjust for them. It’s like adjusting for the degree of difficulty when judging an Olympic diver. We’re getting better at this, but we’re not good enough.”
Although Mr. Wachter was talking about measuring the performance of healthcare and educational systems, which is different from assessments of design decisions, his basic principle about contextualizing data is relevant to design discussions. Mr. Wachter’s mention of degree of difficulty suggests the need for a similar “adjuster” for instances when people are trying to extrapolate from data collected in one designed/re-designed setting to another.
It is a transferability factor. The transferability factor would be multiplied by results obtained in the first environment to obtain probable results in a second. Only when the transferability factor is 1 could the findings obtained in one environment be expected to be directly analogous in another. The transferability factor would be fiendishly difficult to calculate and could be more or less than 1. Data management systems may one day allow its easy determination, once all relevant information is collected, but that day is not today.
What would go into calculations of a transferability factor? Things like differences in national and organizational cultures, specific types of work done, ambient factors such as ventilation rate and lighting levels, worker views of each other or something else…the list is very long. Each element has an influence on outcomes such as performance, some more than others, however, and each can vary from one situation to another.
Although we don’t yet have a handle on project-by-project transferability factors, it is prudent to talk about general implications of environmental changes. If some sort of performance measure that is relevant and valuable to people beginning a new project responds desirably to an environmental change elsewhere, it seems reasonable to try that modification in the new context at well, as long as conditions seem fundamentally similar in both situations. The quantified results in the tested site can be reported to decision makers at the new one, but they cannot be presented as probable, that’s only possible after the transferability factor is calculated.
Some qualitative studies indicating pleasing changes in a location may not have quantified results of the type so desired by financial types, but they still can showcase desirable interventions, if properly conducted (and no scientific studies are worthy of any sort of discussion if they’re not). Many times, aversion to qualitative research results from lack of knowledge of the rigors of these methods; sharing information on how thorough qualitative studies are conducted is regularly a prerequisite for their acceptance.
For a specific project, it can be useful to determine how well – or not – design decisions seem to have panned out, again within the specific physical, social, cultural and financial environments in which they are made. Findings can aid in future planning. It’s important to recognize that at another time, even for the same organization, data obtained after replicating a design decision might be different from that original set.
Numbers are interesting. They can tell a compelling story, but, at least when humans are involved, they are more like context-specific performance art than a best selling novel whose words remain the same wherever they’re read.
Sally Augustin, PhD, a cognitive scientist, is the editor of Research Design Connections (www.researchdesignconnections.com), a monthly subscription newsletter and free daily blog, where recent and classic research in the social, design, and physical sciences that can inform designers’ work are presented in straightforward language. Readers learn about the latest research findings immediately, before they’re available elsewhere. Sally, who is a Fellow of the American Psychological Association, is also the author of Place Advantage: Applied Psychology for Interior Architecture (Wiley, 2009) and, with Cindy Coleman, The Designer’s Guide to Doing Research: Applying Knowledge to Inform Design (Wiley, 2012). She is a principal at Design With Science (www.designwithscience.com) and can be reached at sallyaugustin@designwithscience.com.