Academic Funding

Gathering Data for Policy Makers, Business and the Public

May 5, 2013 1710

Federal surveys have been getting more expensive to administer, in part because the number of people who actually respond to surveys has been progressively declining.  As a result, researchers are developing and implementing new methods for collecting the information that our policy makers, business leaders and the public officials depend on. Many of these new approaches are highlighted in volume 645 (January 2013) of the Annals of the American Academy of Political and Social Science.  These approaches use new technologies and combine strategies to collect needed data. Below is an explanation of some of the technical terms and new approaches presented in the Annals.

Multi-modal Surveys

Combining several different methods of approaching people for surveys is a promising approach to maximize response rates. For example, the American Community Survey now is conducted through a multi-modal survey. First, a statistically valid random sample of people to be surveyed is selected from an address database. These people are mailed a letter asking them to fill out the survey on a secure website. Response rates for the website are fairly high, especially considering that the cost of this first step is very low. If people don’t respond, they are mailed a survey they can submit by mail. Then, a sub-sample of those who don’t respond are called, and some of those who don’t respond by phone are visited in-person by survey-takers. While the last steps are more costly, since they are only used for a subset of the survey participants, the multi-modal approach increases the response rate while keeping costs down. The response rate for this survey has been consistently near 98 percent of all household units.

This approach can be further refined: one possibility for the decennial Census, for example, is a promotional campaign asking people to pre-register electronically so they can receive the questionnaire by email.

Other Improvements in Survey Design Can Improve Response Rates

  • Response rates improve significantly when the survey targets are contacted before the survey, with a letter or email that explains why the survey is relevant to the person being asked to take the survey.
  • Multiple calls or other approaches to the person also increase response rates.
  • Reducing the burden of the survey, for example by limiting the number of questions, filling in information in advance where it can be obtained from administrative sources, using a “matrix” approach so that not every person surveyed has to answer every question and using proxy reports by others who are knowledgeable when the person doesn’t respond, can all help improve the response rate and reliability of survey data.
  • Financial incentives can improve response rates.

Surveys Can be Adjusted for Non-Response

Survey reliability can be improved by weighting the answers of the responses from subgroups to ensure that the sample reflects as much as possible the population being surveyed. Similar statistical measures can be used to adjust where people choose not to respond to individual questions.

Data Can be Obtained by Linking to Administrative Records

Federal administrative agencies such as the Internal Revenue Service, the Social Security Administration, the Center for Disease Control (which houses vital statistics) and the Department of Housing and Urban Development, as well as state and local agencies such as school boards and Departments of Public Health all maintain important data. If cross-linked appropriately, with great care to protect individual privacy, administrative data can provide important information that bolsters the accuracy and reliability of surveys. However, it does have limitations; for example, health records will not capture those uninsured people who do not seek care and education records won’t include the homeschooled child.

Digital Data

Digital data refers to information that is created electronically by government, business and individuals. These data might be gathered when someone makes a phone call, purchases an item, conducts a web search, or drives on an HOV lane, to give just a few examples. Commercial firms collect digital data on everything from phone records to purchasing patterns to travel records. The government also collects digital data, especially for its security and anti-terrorism programs. Some of these data, again with proper privacy protections, could be used to provide the kind of information that business, in particular, relies on.

The American Academy of Political and Social Science, one of the nation’s oldest learned societies, is dedicated to the use of social science to address important social problems. For over a century, our flagship journal, The Annals of the American Academy of Political and Social Science, has brought together public officials and scholars from across the disciplines to tackle issues ranging from racial inequality and intractable poverty to the threat of nuclear terrorism. Today, through conferences and symposia, podcast interviews with leading social scientists, and the annual induction of Academy Fellows and presentation of the Daniel Patrick Moynihan Prize, the Academy is dedicated to bridging the gap between academic research and the formation of public policy.

View all posts by American Academy of Political and Social Science

Related Articles

Emerson College Pollsters Explain How Pollsters Do What They Do
Communication
October 23, 2024

Emerson College Pollsters Explain How Pollsters Do What They Do

Read Now
All Change! 2024 – A Year of Elections: Campaign for Social Science Annual Sage Lecture
Event
October 10, 2024

All Change! 2024 – A Year of Elections: Campaign for Social Science Annual Sage Lecture

Read Now
Neuromania – Or Where Did the Person Go?
Opinion
October 2, 2024

Neuromania – Or Where Did the Person Go?

Read Now
‘Settler Colonialism’ and the Promised Land
International Debate
September 27, 2024

‘Settler Colonialism’ and the Promised Land

Read Now
Webinar: Banned Books Week 2024

Webinar: Banned Books Week 2024

As book bans and academic censorship escalate across the United States, this free hour-long webinar gathers experts to discuss the impact these […]

Read Now
Research Assessment, Scientometrics, and Qualitative v. Quantitative Measures

Research Assessment, Scientometrics, and Qualitative v. Quantitative Measures

The creation of the Coalition for Advancing Research Assessment (CoARA) has led to a heated debate on the balance between peer review and evaluative metrics in research assessment regimes. Luciana Balboa, Elizabeth Gadd, Eva Mendez, Janne Pölönen, Karen Stroobants, Erzsebet Toth Cithra and the CoARA Steering Board address these arguments and state CoARA’s commitment to finding ways in which peer review and bibliometrics can be used together responsibly.

Read Now
Revisiting the ‘Research Parasite’ Debate in the Age of AI

Revisiting the ‘Research Parasite’ Debate in the Age of AI

The large language models, or LLMs, that underlie generative AI tools such as OpenAI’s ChatGPT, have an ethical challenge in how they parasitize freely available data.

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

4 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments