NSF’s Three “Sensational Surveys” Survive and Thrive
Former Senator Tom Coburn, R-Oklahoma used the American National Election Studies as a favorite target in his attacks on the National Science Foundation’s support for political science research and in his annual publication of wasteful federal spending. Coburn cited media-sponsored polls as an alternative to ANES that would not waste taxpayer dollars. (For an interesting take on media polls, see Jill Lepore’s piece in The New Yorker.)
Take solace: In the three separate 2016 versions of wasteful public spending books — produced by senators Jeff Flake, R-Arizona; James Lankford, R-Oklahoma; and John McCain, R-Arizona — there are no mentions of ANES. In fact, support for the ANES and two other surveys, the General Social Survey (GSS) and the Panel Study of Income Dynamics (PSID), appears to have solidified more than ever. All three are included in The Sensational Sixty, a publication celebrating the most significant research projects funded by NSF in its 60 years.
Nonetheless, these three “gold” social science surveys, mostly funded by NSF’s Directorate of Social, Behavioral and Economic Sciences, or SBE, over the years have produced a number of questions regarding their support and place within NSF. In addition, the problems facing surveys in general, such as high cost, low response rates, and others, also provide food for thought. For these reasons the National Academies’ Committee on National Statistics now has a Standing Committee on the Future of NSF-Supported Social Science Surveys, chaired by Barbara Entwisle of the University of North Carolina.
A long history …
The ANES, GSS and PSID have a long history of NSF funding. The ANES has been around since 1948. In its early days it relied on support from the private Carnegie and Rockefeller foundations, but in 1976 the NSF took over its major funding with a grant to Warren Miller at the University of Michigan for “The Study of the American Electorate.” In its almost 70 years, the ANES has conducted presidential year pre- and post-election studies using face-to-face interviewing of a nationally representative sample of adults.
Howard J. Silver served for a quarter century as the executive director of the Consortium of Social Sciences Agencies. His blog appears monthly at Social Science Space.
According to its website, “the ANES produces high quality data on voting, public opinion, and political participation over time to serve the research needs of social scientists, teachers, students, policymakers, and journalists who want to better understand the theoretical and empirical foundations of national election outcomes. “ In addition, “the longevity of the ANES time-series greatly enhances the utility of the data, since measures can be pooled over time, and both long-term trends and the political impact of historical events can be identified.” Many books, including the classic volume, The American Voter, articles and dissertations have used ANES data. ANES also participates in the Comparative Studies of Electoral Systems, an international collaborative program of research among election study teams.
What also separates the ANES from exit polls are its pilot studies that have tested content and methodology for forthcoming Time Series studies. Furthermore, in recent years, to encourage involvement from the user community, ANES developed “The Online Commons.” Scholars can help shape ANES by submitting ideas for survey questions and future topics and identifying issues in data collection. For this year’s survey, ANES is accepting proposals until 3 p.m. Eastern on February 14.
Since 1972, the GSS — with NSF support to the National Opinion Research Center at the University of Chicago — has served, according to its website, as a barometer of social change and trends in attitudes, behaviors, and attributes of the United States adult population by conducting in-person, cross-sectional surveys of the nation’s adult household population. It also distributes up-to-date, important, high-quality data to social scientists, students, policy makers, and others.
The GSS collects data on a wide range of topics: behavioral items such as group membership and participation; personal psychological evaluations including measures of well-being, misanthropy, and life satisfaction; attitudinal questions on such public issues as abortion, civil liberties, crime and punishment, race relations, gender roles, social mobility, and spending priorities; and demographic characteristics of respondents and their parents.
Like the ANES, the GSS has stimulated comparative cross-national research, called the International Social Survey Program. In addition to its programs of basic research and data distribution, the program has carried out an extensive range of methodological research designed both to advance survey methods in general and to insure that the GSS data are of the highest possible quality. It too has tried to broaden the input to the survey, and is now accepting proposals for questions or short modules for inclusion in the 2018 GSS. The deadline is June 30.
The PSID is a longitudinal panel survey that began in 1968, originally funded by the office of the assistant secretary of planning and evaluation in the Department of Health and Human Services and focused on poverty issues. It soon expanded its focus and received its original NSF funding in 1979 with the University of Michigan directing the survey. Over the years, the study has also received funding from the National Institute on Aging and the National Institute of Child Health and Human Development. As a nationally representative panel survey with data on the same families and their descendants, the PSID , according to its website, “provides a unique opportunity to study evolution and change within the same families over a considerable time span.”
PSID data are used to systematically investigate a myriad of questions involving the study of life-cycle opportunities and trajectories over time. The data sets, the PSID website notes, have been central to investigate areas such as intergenerational relations; income, poverty, savings and wealth; demographic events such as teen childbearing, marriage and divorce, living arrangements and mortality; labor market behavior; and the effect of neighborhoods. The extended time series of data allows the estimation of robust, causal models, supports the study of economic behavior as well as the conduct of cohort analysis through the comparison of people’s activities from one time period to another. Like the ANES and the GSS, the PSID has generated overseas imitations that facilitate cross-national comparative research.
… of controversy
From the beginning the surveys have faced difficulties at NSF. Originally housed in a Measurement Methods and Data Resources division — something Otto Larsen in his Milestones & Millstones called the social sciences’ “strategic defense initiative” in the face of the Reagan presidency’s onslaught in 1981 since that administration had a fondness for databases — NSF moved the three surveys into the disciplinary programs.
This created another problem. With budgets tightening after the Reagan cuts, many of those seeking support from NSF for political science, sociology, and economics research complained that funding for the surveys squeezed out money for individual investigator initiated projects. This culminated in a debate during the deliberations in 1991 of the task force that proposed a Social, Behavioral and Economic Sciences Directorate in 1991. Sociologist Richard Berk argued that given the paucity of funds, NSF should get out of the database business. He argued that this was “lowest denominator research” that did not fit the definition of the agency’s role. Charles Plott, political scientist and economist, defended the databases, especially the ANES, which he argued “has had an important impact on political science.” The task force rejected Berk’s argument and SBE has maintained the databases during its history.
More recently, the SBE directorate, again faced with funding difficulties, has examined how to improve operations, increase accessibility, develop best practices, and facilitate collaboration and cooperation among the three surveys.
At a 2010 Principals Conference reported on by Sociology Program Office Patricia White, one suggestion was to develop a single portal to “facilitate data search, downloads, and basic analysis, mostly for experts and novices (intermediate skill ) users . The tools should facilitate searching and downloading from one or multiple data sets at thematic levels and by variable; provide users with information about metadata, such as the provenance of the data; permit users to construct dynamic codebooks that reflect the actual data downloaded; and meet current data management standards.”
In 2012, an SBE Advisory Committee working group on advancing SBE survey research, chaired by Jon Krosnick of Stanford, held workshops to provide advice regarding NSF’s best use of its resources to support research through survey data collection. The May 2015 report (PDF here) mainly focused on questions of data collection challenges, methodological innovations, the digital data explosion, and questionnaire issues. However, its major recommendation to NSF was to return to the pre-1980s situation “that all major SBE survey projects be funded not from the disciplinary programs in which they have been housed and instead be funded through a new program managed by a single program officer.”
SBE has also solicited proposals regarding “Metadata for Long Standing Social Science Surveys.” According to the announcement, the projects would develop tools to bridge data collection and dissemination by collecting and coding metadata associated with past and future ANES, GSS, and PSID surveys. The goal is to fund projects that will help make the many years of legacy data available to researchers who seek to answers to current scientific questions. One such project has provided a crosswalk between the data from the ANES and GSS.
So for the moment, these three “sensational” surveys remain an important part of the SBE infrastructure and funding stream. However, attacks and limits on SBE funding may continue and grow in the future. In addition, issues affecting surveys such as “invasion of privacy” concerns (see attacks on the American Community Survey) as well as the cost and response rate problems will remain. The NAS Standing Committee and those in the SBE community who value these surveys will need to continue their vigilance and support.
