The Best Laid Plans… Qualitative Research Design During COVID-19

“It is important to reclaim for humanity the ground that has been taken from it by various arbitrarily narrow formulations of the demands of rationality.”

Amartya Sen

[Ed. note: This post originally appeared on Social Science Space’s sister site, MethodSpace, where additional ideas on online teaching can be found.]

Qualitative research seeks to identify and incorporate the complexities of participants’ lived experiences and feedback into the research process itself—not just the findings—in ways that challenge researcher power and the imposition of interpretive authority. Researcher and research design responsiveness requires what is sometimes referred to as an emergent design approach. This means that elements of a study’s research design, such as participant selection and data collection methods, are carefully reconsidered in relation to emergent understandings and realities of participants’ views and experiences. As Ravitch and Carl (2020) state,

Since participants’ experiences and mediating contexts are difficult to anticipate, identify, and articulate fully in advance of the implementation of research, researchers need to respond to these in real time once the research is under way. In fact, the primary criterion of qualitative validity is fidelity to participants and their experiences rather than a strict adherence to methods and research design.

Qualitative research design includes: 1) site and participant selection; 2) data collection methods; 3) data analysis strategies and techniques (Ravitch & Carl, 2020). In this post, I address the first two domains in relation to COVID-19. The researchers I know—both seasoned and new—are quite concerned about the data collection aspects of research design given sudden changes in the world. What follows are things I’ve been discussing with students and colleagues since the novel oronavirus arrived and changed the world, and therefore our research in that world, in incalculable ways that will unfold for years.

In this moment of the global COVID-19 pandemic, the ideas of emergent design and researcher and design responsiveness take on new meaning and import; they can serve, I argue, to connect more traditional qualitative methods with participatory frameworks and critical and humanizing methodologies such as trauma-informed methodology and Chronic Illness Methodology, which I describe below.

Site and Participant Selection Changes

Suddenly, researchers have a range of new concerns about our research. These include participant access given that most of the United States, indeed the world, is practicing social distancing at home. Concerns range from 1) identifying participants given that people are no longer congregating together physically in naturally occurring groups and organizations, and 2) asking people for time to do interviews, focus groups, or other asks for time (this is true even when people had already secured participants) given the burdens placed on everyone by this pandemic. These concerns are central given that life has radically changed since completing their research designs—people are displaced, working from home, out of work, looking for jobs, juggling family responsibilities sometimes including kids and elderly parents during the workday, unwell or taking care of people who are ill, and so on. This global moment, importantly, necessitates learning a set of new skills necessary to design and conduct valid, humanizing research online.

Below are ways to consider these design changes, embedded in live examples of research design changes made since coronavirus. For example, Beckett designed a case study that was to examine leader agility in complex adaptive systems in a Fortune 100 company. He had just finalized his interview protocol and received IRB approval when COVID-19 halted the workplace, creating sudden access issues given international pandemic flux. He was going to interview executives from 3 countries face-to-face, which can no longer happen. As well, his topic and research questions needed to change given the enormity of this new global development and how it relates to his topic.

Beckett needed to pivot. He decided that rather than interview a dispersed group of 40 business leaders once about critical incidents of team agility, he will engage more deeply with a smaller group in the US. Beckett chose a US-based leadership team (12 people) and shifted the focus to a case study on leading with agility through coronavirus. This still builds on his intended topic and theoretical framework, and in fact, deepens what he’ll be able to learn during a live critical incident of crisis leadership. Beckett adjusted his interview protocol to reflect the new research questions and shift in participant demographics. And because they believe it will be generative in real-time to learn from his research, the team agreed to participate in a more intensive data collection process. Beckett submitted an addendum and interview protocol to IRB and they approved the switch.

Another example is Camilla, a doctoral student who’s a school principal and who just passed her dissertation proposal defense on exploring black girls’ perceptions and experiences of safety in an under-resourced urban middle school. With COVID-19, the school closed making access to students difficult. As well, the issues she wanted to explore—racial microaggressions and racialized stress—suddenly feel less urgent than finding food and shelter for students and their families. Camilla believes the research is even more important now—it provides an opportunity to check on the students’ well-being while in transient living situations. She’s exploring opportunities for virtual data collection with less girls more intensively for a shorter duration. She was planning to interview thrice weekly for three months, but since she can’t access the larger group over time given that the structure of school is no longer there, and moreover, given that they’re struggling in myriad ways. If not, Camilla will pivot again and interview her colleagues—fellow principals, teachers, and school counselors—to understand their experiences and ideas for exploring and supporting student well-being and safety in the current pandemic. Camilla is clear about keeping her existing theoretical framework (literature review). This pivot makes that possible and, importantly, it will contribute to a much-needed literature on crisis leadership in complex adaptive schooling systems.

It’s important to state that issues of equity in relation to participant access and representation are central to deciding if and how these choices make sense or create validity or ethical concerns. One plus of design pivots may be greater access to a wider range of participants, though in pandemic it is hard to know. Importantly, this raises issues of representation—if the most vulnerable in society are unable to engage in research because their lives are exponentially turned upside down, their stories and anything built from our research excludes them.

Data Collection Method Changes

Most of the world cannot leave our homes which raises questions about conducting an entire data collection process using online methods. People wonder if they should 1) postpone data collection; 2) conduct interviews and focus groups online (and/or change to a study using extant sources), 3) employ different data collection methods since interviews are virtual and working from home precipitates issues of privacy and confidentiality (on both sides of the screen), all of which must be addressed and explicated in the research design.

A colleague conducting research on difficult conversations about family dynamics is now forced to conduct online interviews since participants are home, which she worries about for disclosure and confidentiality reasons. She worries about asking participants for their time and to be interviewed about family dynamics while home. She asked the women who had already agreed to be a part of her study if they were able to join a virtual focus group to discuss options once everyone needed to do at-home social distancing. Together, they developed a new data collection plan in which they: 1) started a Google doc for group-think generally and for specific weekly email prompts for data collection purposes, 2) created a WhatsApp group and gave written permission to use all chats as data, and 3) shared transcripts from the first interview—and instead of doing a second round of interviews, the participants will comment on their own interviews, layering in additional insights and concerns, and then they will have an online focus group to discuss what they were all thinking and feeling after they read their individual transcripts. This is a creative work-around that enables the research to continue. It’s a great example that even with new challenges come new research design possibilities. An emergent design mindset supports this.

In contrast, some work-arounds are ethically problematic. Jono could no longer access participants for his dissertation research because, as a transient population (migrant workers), they’re currently scattered given COVID-19. He asked if he can simply re-analyze his old data from a year-old pilot study that had different research questions and address the new questions with these data. This is not an ethical solution because everything maps from and back onto the research questions, and thus, analyzing old data from a different set of research questions and for a new population would skew the analysis and undermine validity.

Researcher and Design Responsiveness as Ethical Stance in Global Crisis Times

Trauma-informed methodology foregrounds learning about trauma and its intra-psychic and interactional effects, cultivating a research environment comfortable to those who’ve experienced trauma, and recognizing the resilience and resources of individuals and communities who have experienced or are experiencing trauma (Pak & Ravitch, in review). Trauma experiences can influence behavior and responses in interviews and focus groups. Memory loss, lack of focus, emotional reactivity, and different versions of the same story can be signs of trauma as it is exhibited during interviews and focus groups. While we cannot essentialize, it is important to raise awareness of these possibilities for any individual.

Trauma-informed methodology has always been important for the broader population given its foregrounding of the affective and social-emotional dimensions of interviewing and the need to be intentional in relation to possible trauma histories playing out in the present. I argue that trauma-informed methodology should become much more widely used in current research studies. Given that we are, I believe, in a moment of collective trauma—both our own trauma and vicarious trauma—we must attune ourselves to both its inner and outer reverberations, for ourselves and our participants. It is important to engage people with an understanding that all traumas are not the same, and while the pandemic is shared trauma, it lands into the lives of already-vulnerable populations in ways that cause more severe diffusion effects.

As well, many already have trauma histories completely separate from COVID-19 that must be considered with compassion. Interviewers and focus group facilitators should be familiar with possible signs of trauma and not assume a participant is being evasive or dishonest if their responses or communication styles depart from more familiar ones. Additionally, it’s vital to consider the intersection of trauma with aspects of social identity and structural inequity including culture, gender, ethnic, race, social class and caste, religion, immigrant status and so on.  

Related to better aligning research with our shared humanity, I draw on Melissa Kapadia-Bodi’s Chronic Illness Methodology (2016), which is an intentionally embodied, relational, critical approach to designing and conducting research. Chronic Illness Methodology is a methodological approach that “views research as an embodied project, acknowledges researchers’ and participants’ bodies and lived experiences as central to the research process, encourages participants to take up space in the research process, supports an active concern for participant well-being throughout research and writing, and enables a critical focus on participants’ layered and societally contextualized stories of their own lives… Chronic illness methodology is for all bodies.”

Critical humanizing qualitative methodologies, in these times, help us to engage cosmopolitanism—the idea that we are all connected, dependent on, and responsible to, each other as humans—as we rethink and work to dismantle deficit orientations and false, socially constructed binaries relating to illness/wellness and ability/disability as the embodied struggles of the human family (Appiah, 2006). Engaging this kind of critical and humanizing methodology seems, to me, the only way that our research can proceed with humility and authenticity in these fraught and challenging times. We must un-learn so much now as we re-learn forward; all of which requires an emergent design mindset and a collectivist orientation.

Recommendations

• Moving data collection wholly online creates specific validity and ethical issues that need to be identified and addressed as part of research design (see Ravitch & Carl 2020 for resources). Be intentional in planning for and rehearsing online data collection situations (interviews and focus groups) so the research experience is generative, positive, engaged, and enriching.
• Read about trauma-informed interviewing and Chronic Illness Methodology. Build a working understanding of strategies for developing data collection instruments and techniques for approaching participants with care given the ubiquity of trauma right now.
• Approach study participants with respect, humility, and appreciation for their time. Make every effort to schedule around their needs (e.g., childcare, work schedules). Let them know that an ideal interview scenario will provide them a confidential space.
• Develop a brief script to begin online interviews and focus groups that 1) addresses, with compassion, the current difficult moment of Coronavirus as well as changes in the interview format/process caused by the pandemic. Do this before the general framing of interviews and focus groups (Welcome, informed consent, etc.).

Most of all, at this strange and scary time of distancing in the world, I wish you good health, safety, peace, and goodness. Qualitative researchers generate powerful stories of healing, connection, and transformation. Right now, we can be truth-listeners and truth-truthtellers for the world, which means we have a unique set of skills to serve as a light in dark times. If you’d like to stay in touch, or have questions, follow me on Twitter @SharonRavitch.