Academic Funding

Gathering Data for Policy Makers, Business and the Public

May 5, 2013 1738

Federal surveys have been getting more expensive to administer, in part because the number of people who actually respond to surveys has been progressively declining.  As a result, researchers are developing and implementing new methods for collecting the information that our policy makers, business leaders and the public officials depend on. Many of these new approaches are highlighted in volume 645 (January 2013) of the Annals of the American Academy of Political and Social Science.  These approaches use new technologies and combine strategies to collect needed data. Below is an explanation of some of the technical terms and new approaches presented in the Annals.

Multi-modal Surveys

Combining several different methods of approaching people for surveys is a promising approach to maximize response rates. For example, the American Community Survey now is conducted through a multi-modal survey. First, a statistically valid random sample of people to be surveyed is selected from an address database. These people are mailed a letter asking them to fill out the survey on a secure website. Response rates for the website are fairly high, especially considering that the cost of this first step is very low. If people don’t respond, they are mailed a survey they can submit by mail. Then, a sub-sample of those who don’t respond are called, and some of those who don’t respond by phone are visited in-person by survey-takers. While the last steps are more costly, since they are only used for a subset of the survey participants, the multi-modal approach increases the response rate while keeping costs down. The response rate for this survey has been consistently near 98 percent of all household units.

This approach can be further refined: one possibility for the decennial Census, for example, is a promotional campaign asking people to pre-register electronically so they can receive the questionnaire by email.

Other Improvements in Survey Design Can Improve Response Rates

  • Response rates improve significantly when the survey targets are contacted before the survey, with a letter or email that explains why the survey is relevant to the person being asked to take the survey.
  • Multiple calls or other approaches to the person also increase response rates.
  • Reducing the burden of the survey, for example by limiting the number of questions, filling in information in advance where it can be obtained from administrative sources, using a “matrix” approach so that not every person surveyed has to answer every question and using proxy reports by others who are knowledgeable when the person doesn’t respond, can all help improve the response rate and reliability of survey data.
  • Financial incentives can improve response rates.

Surveys Can be Adjusted for Non-Response

Survey reliability can be improved by weighting the answers of the responses from subgroups to ensure that the sample reflects as much as possible the population being surveyed. Similar statistical measures can be used to adjust where people choose not to respond to individual questions.

Data Can be Obtained by Linking to Administrative Records

Federal administrative agencies such as the Internal Revenue Service, the Social Security Administration, the Center for Disease Control (which houses vital statistics) and the Department of Housing and Urban Development, as well as state and local agencies such as school boards and Departments of Public Health all maintain important data. If cross-linked appropriately, with great care to protect individual privacy, administrative data can provide important information that bolsters the accuracy and reliability of surveys. However, it does have limitations; for example, health records will not capture those uninsured people who do not seek care and education records won’t include the homeschooled child.

Digital Data

Digital data refers to information that is created electronically by government, business and individuals. These data might be gathered when someone makes a phone call, purchases an item, conducts a web search, or drives on an HOV lane, to give just a few examples. Commercial firms collect digital data on everything from phone records to purchasing patterns to travel records. The government also collects digital data, especially for its security and anti-terrorism programs. Some of these data, again with proper privacy protections, could be used to provide the kind of information that business, in particular, relies on.

The American Academy of Political and Social Science, one of the nation’s oldest learned societies, is dedicated to the use of social science to address important social problems. For over a century, our flagship journal, The Annals of the American Academy of Political and Social Science, has brought together public officials and scholars from across the disciplines to tackle issues ranging from racial inequality and intractable poverty to the threat of nuclear terrorism. Today, through conferences and symposia, podcast interviews with leading social scientists, and the annual induction of Academy Fellows and presentation of the Daniel Patrick Moynihan Prize, the Academy is dedicated to bridging the gap between academic research and the formation of public policy.

View all posts by American Academy of Political and Social Science

Related Articles

Deciphering the Mystery of the Working-Class Voter: A View From Britain
Insights
November 14, 2024

Deciphering the Mystery of the Working-Class Voter: A View From Britain

Read Now
Doing the Math on Equal Pay
Insights
November 8, 2024

Doing the Math on Equal Pay

Read Now
Exploring the Citation Nexus of Life Sciences and Social Sciences
Industry
November 6, 2024

Exploring the Citation Nexus of Life Sciences and Social Sciences

Read Now
Julia Ebner on Violent Extremism
Insights
November 4, 2024

Julia Ebner on Violent Extremism

Read Now
Emerson College Pollsters Explain How Pollsters Do What They Do

Emerson College Pollsters Explain How Pollsters Do What They Do

As the U.S. presidential election approaches, news reports and social media feeds are increasingly filled with data from public opinion polls. How […]

Read Now
All Change! 2024 – A Year of Elections: Campaign for Social Science Annual Sage Lecture

All Change! 2024 – A Year of Elections: Campaign for Social Science Annual Sage Lecture

With over 50 countries around the world holding major elections during 2024 it has been a hugely significant year for democracy as […]

Read Now
Neuromania – Or Where Did the Person Go?

Neuromania – Or Where Did the Person Go?

David Canter bemoans how people are disappearing as ‘brains’ take over.

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

4 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments