Ann Sloan Devlin on Timeless and Dynamic Research Design

New technology has, and is, changing a lot of the mechanics of social and behavioral science research, but how much is the underlying enterprise itself changing as a result? This is a key question Ann Sloan Devlin, author of the newly released The Research Experience: Planning, Conducting, and Reporting Research, addresses in the interview below. In the book, technologies such as Qualtrics, Google Scholar and Amazon’s Mechanical Turk, are addressed and in he remarks here Devlin notes that while having the latest tools available is a welcome development, knowing which to use at any moment and how to analyze the results are essentially timeless questions regardless of platform.

Devlin is the May Buckley Sadowski ’19 Professor of Psychology at Connecticut College, and in her more than three decades at Connecticut has taught on cognitive processes, industrial and organizational psychology, and research methods. Her research and teaching focus is on environmental psychology.

In addition to The Research Experience, which came out in April, Devlin’s books include 2015’s Transforming the Doctor’s Office: Principles from Evidence-based Design and 2010’s What Americans Build and Why: Psychological Perspectives. In 2012 she received the Helen B. Regan Faculty Leadership Award, which recognizes Connecticut College faculty who exemplify the college’s commitment to shared governance, democratic process and campus community development. She received the John S. King Memorial Teaching Award in 2006.

 ***

Apart, of course, from the subject matter, what is it about social and behavioral research that’s different from other sorts of academic research?

In truth, I’m not sure that social and behavioral research  is different in any fundamental way from other sorts of academic research, but I’ve often thought that social and behavioral research has a way of speaking to our everyday experience that is more accessible than in some kinds of academic research.

How different is this book in 2017 than it would have been in 2007? And what might you expect it to look like in 2027?

The major difference I see (from 2007) is in the tools that are available for students to use, specifically computer and survey software. Thinking back a generation (or two) ago to when I was in school, the computing power we now have in our laptops took a computer the size of a large room to produce. Now we have powerful computer programs like SPSS on our laptops that enable every student to run sophisticated analyses. What hasn’t change, however, is the fact that you still need to know WHICH analysis is appropriate for which research question.

With regard to survey software, this is a fast-changing landscape. Instead of handing out paper versions of a questionnaire, students now have access to survey software such as Survey Monkey and Qualtrics. This means that students are not confined to bricks-and-mortar buildings to administer surveys; rather, they can send out links beyond participant pools to Facebook and other social media sites to solicit participants.

Related tools are online participant platforms such as Amazon Mechanical Turk. Amazon MTurk is an online paid crowdsourcing platform to acquire participants. Students at small colleges where the size of the participant pool has been a limiting factor can use such crowd-sourcing platforms to solicit participants for a relatively small fee. This means that small sample sizes will not be such a problem for students (or faculty members).

It is hard to predict what a research methods book will look like in 2027; the major research designs (e.g., between subjects design) will stay the same, but I think even more tools will be readily available to students, whether they are virtual reality, eye-tracking, or other tools from computing and behavioral neuroscience. As these tools become more affordable, I think there will be ways for students to incorporate them in their research, even at the introductory level. Given the rapid pace of technology, we can’t even imagine what some of these to-be-developed tools might be.

Explore The Research Experience: Read the preface by clicking HERE.

By the same token, are there concepts in a book on the research experience that are essentially timeless?

The answer to that question is a resounding “yes.” A source that is often cited is Campbell and Stanley’s 1963 book Experimental and quasi-experimental designs for research. This book is as relevant today as it was 50 years ago, and it will still be relevant 50 years from now. Why? Because the essentials of research design stay the same. What makes something an experiment (random assignment to condition and manipulation of a variable) doesn’t change.

Do new technologies, such as Qualtrics, Google Scholar, and Amazon Mechanical Turk, merely require developing new skills or do they require new ways of thinking?

These tools have a learning curve and require the development of new skill sets; I think today’s students are particularly good at mastering these tools. I’m not sure that the tools require new ways of thinking, but rather I think using these tools promotes asking new research questions that could not be answered outside of these platforms. An example would be the ways in which access to Qualtrics promotes asking questions to explore on social media (e.g., Facebook, LinkedIn, Tinder) or reaching age groups that are not available in the 18- to 22-year-old psychology department participant pool.

As you wrapped your arms round the entire experience in order to document it, what epiphanies (if any) did you experience?

In many ways, teaching reaching methods in 2017 is drastically different from 40 years ago; in many ways, it is the same. Students have so much at their fingertips, but having and knowing are not synonymous. Computations that were not possible for undergraduate students to perform now are readily available on SPSS. Yet, if you don’t know the difference between a nominal and an interval variable, you will likely have trouble figuring out what kind of analysis to run!

What has become more difficult for students and instructors alike is that there is more to master. Not only must students learn the fundamentals of research design, but there are also the new tools that are used to create surveys and collect data, such as Qualtrics and Amazon M Turk, respectively.

Your book is really comprehensive, but let’s get specific What area of the research experience is the most common where people have problems.

It is a real challenge to come up with a testable research question in a semester-length course and go out and actually collect and analyze the data to test that research question. In my experience, students have a hard time translating a research question into a testable hypothesis and understanding the appropriate statistical test(s) to use to evaluate the hypothesis. Another big stumbling block is identifying an appropriate dependent variable (DV) to use. Often DVs are tests (measures, for example, self-esteem), and students must actually OBTAIN these measures in order to use them in research. Finding such measures takes a bit of detective work because complete scales are often not included in published research articles (such scales take up too much space). Students need to stick to the recommended timeline (in the book) to make it to the finish line!

Would you describe your own initial forays into research? What could the Ann Sloan Devlin of 2017 have said to that earlier Ann that would have been the biggest help?

One of my earliest projects was based on the experience of my material grandmother who moved into a nursing home and passed away shortly thereafter. This experience led to my interest in housing for older adults and a project looking at the relationship between the physical design of housing for older adults and outcomes such as well-being and sense of community. What this project reflects is something that I still endorse. Often projects that grow out of meaningful personal experiences make the best research studies.