Statistical Researcher Meets Qualitative Researcher: A Tale of Cultural Misunderstanding
Recently a colleague and friend I very much respect asked me to conduct a couple of focus groups for him. He’s a numbers guy; however, he was fascinated to hear about the fishbowl focus groups I had been conducting at school sites around issues related to teen alcohol and drug use. (These groups have led to policy changes at some schools, and even contributed to breaking up a drug ring at one site—but that’s a separate story.) He didn’t make the mistake that a few of my statistically-inclined colleagues make, of assuming that “anyone can run a focus group—what’s the big deal?” My friend understands that there is as much rigor in a well-run focus group or stakeholder interview as there is in a well-crafted survey or assessment.
And then he handed me his focus group protocol and questions. All 17 pages and 26 questions of it.
I stared at him in disbelief. The average focus group I run has five to seven questions, with a lot of probes to delve deeper into responses. My protocols and questions might run four pages. Maybe.
And then I looked at the questions. “How often do you and your friends…?” “Where do you and your friends…?” “Over a period of x number of months, how many times did you and your friends…?”
No. No. No. NO.
In talking further with him, my friend thought that by calculating the number of “respondents” (that is, focus group participants) that answered each question in each way, he’d get to the data he wanted.
And that’s when light dawned for me.
My friend and I live in two different countries, speak two different languages, live by completely different cultural values, and haven’t got more than a clue about the reality in which the other is operating. Heck, we live on two different planets. I’m not sure we don’t breathe different compounds.
He’s looking for the concrete: the facts, ma’am. Just the facts. In this case, the behaviors. Then he can generate policy answers to the problems identified.
I’m looking at nuanced perception, at life experience, at belief systems and how they impact behavior.
Both are incredibly important: the facts tell us what is happening in real time and space. Perception and belief tell us why. People don’t operate based on reality. They operate based on their cultural orientation, their perception, and their life experience.
“Here’s what we do,” I told my friend, as we struggled to find common ground. “We conduct a survey with these young people that addresses knowledge, attitude, and behavior before they come into the group. And then we ask the questions about perception and belief around these topics. We ask about how they come to believe what they believe. We ask THEM to interpret the data from the survey, and to give us their insights regarding why we got the responses we got. In a focus group, a trained facilitator can do this without compromising confidentiality, shaming, or putting participants on the spot. And the information we get—TRUST ME—will be far richer than either of us can anticipate right now.”
And so it was.
Focus groups and interviews can be very rich sources of data in and of themselves, and can really enhance our understanding of patterns, trends, and the outcomes we are getting with our programs and services. They help answer the “why?” and “how?” behind the facts. They help us tell a story that policy-makers, supervisors, managers, and concerned others can use to make better informed decisions that better meet the needs of the people they serve.
My friend is struggling to learn my language. I often struggle with his, as well. We aren’t in competition. We both know it’s important to be bilingual in statistical and qualitative research. In the meantime, our discussions and yes, even our arguments, enrich each other, and that in turn enriches the work we do.
