Is Maximizing the Impact of Research Desirable?
Sometimes the desire to have impact can come at a price.
Of course, we must recognize the practical and ideological contexts in which arguments about research impact take place. Governments want to be able to justify their funding of social science by pointing to practical ‘returns’ on the ‘investment’ made. Universities have been required to demonstrate the impact of the research carried out by their members, not least for the purposes of the Research Excellence Framework. Social scientists, and the organizations that represent them, wish to highlight the impact of their work in order to defend its funding in an age of austerity.
This article by Martyn Hammersley is drawn from material in his new book, The Limits of Social Science.
Nevertheless, there are questionable assumptions involved in the idea that the impact of social research should be maximized. First, it seems to assume that the effects of sound research on policymaking and practice are always desirable. This is held almost as an article of faith by many researchers: very often people become researchers precisely because they believe that through research they can bring about social improvement of one kind or another, whether piecemeal or transformative. While, of course, they recognize that such change does not always result, perhaps even that it happens infrequently, they nevertheless assume that, if it had the influence it deserved, good research would always improve policymaking or practice, and that when this does not happen it amounts to a distortion of the relationship.
However, this assumption is of doubtful validity. It implies that research findings carry within them instructions about what ought to be done, whereas in fact any set of empirical findings can generate conflicting evaluations and prescriptions, depending upon the particular set of values adopted to interpret them. This clearly raises the question of what counts as ‘improvement’. And there is rarely general consensus about the priority that should be given to different values. Aside from this, no empirical findings about the nature of some problem, or even about the effects of previous policies or practices, can tell us with great certainty what the effects of future policy or practice will be: there is too much contingency in social processes. For both these reasons, the relationship between particular empirical findings and the production of social improvement (however defined) is highly mediated, to say the least. Indeed, the publication of research findings may sometimes even have consequences that researchers and others regard as undesirable.
The second assumption involved in the commitment to maximize the impact of social research is the belief that it is desirable to increase the contribution of research at the expense of the various other factors that shape policy and practice. For example, in the promotion of evidence-based policymaking and practice around the turn of the last century it was common for these other factors to be caricatured as irrational, for example as reflecting the influence of ‘tradition, prejudice, dogma, and ideology’ (Hargreaves 2007:12). From this point of view, making policy or practice more research-based necessarily results in it becoming more rational. And many researchers outside of the evidence-based practice movement have shared this view.
References
Dunne, J. (1997) Back to the Rough Ground: Phronesis and Techne in Modern Philosophy and in Aristotle, Notre Dame IN, University of Notre Dame Press.
Hammersley, M. (2002) Educational Research, Policymaking and Practice, London, Paul Chapman/Sage.
Hammersley, M. (2013) The Myth of Research-Based Policy and Practice, London, Sage.
Hargreaves, D.H. (2007) ‘Teaching as a research-based profession: possibilities and prospects’, in Hammersley, M. (ed.) Educational Research and Evidence-based Practice, London, Sage. (Annual Teacher Training Agency, given in 1996.)
MacIntyre, A. (1971) ‘The end of ideology and the end of the end of ideology’, in Against the Self-Images of the Age, London, Duckworth.
Turner, S. (2003) Liberal Democracy 3.0, London, Sage.
Weiss, C. (1979) ‘The many meanings of research utilization’, Public Administration Review, 39, 5, pp426-31.
However, this too is questionable. For one thing, research cannot provide all of the information that is required by policymakers and practitioners – they must necessarily rely upon other sources of knowledge (Hammersley 2002 and 2013): for information about local conditions not studied by researchers or about very recent events (research can never be entirely up to date). Furthermore, like any other source of knowledge claims, research is fallible. For this reason, policymakers and practitioners must engage in evaluation of the relevant evidence available to them, rather than accepting it at face value.
In addition, we must recognize that policymaking relies upon background experience and judgment about priorities, about what would and would not be ethically and politically acceptable policies, as well as about which ones are likely to mobilize sufficient support to get off the ground. Moreover, what is demanded here is close in character to the sort of practical, rather than theoretical, judgment that Aristotle calls phronesis (Dunne 1997).
Against this background, it is perhaps worth giving attention to one particular influence on policymakers and practitioners that is frequently caricatured as a negative factor: ideology. This is generally treated as consisting of false views that people adopt so as to serve their own interests, or into which they have been socialized because these views function to preserve some larger form of social organization. However, what the term ‘ideology’ points to is more complex than this, as Alasdair C. MacIntyre (1971:5-6) highlighted in his critique of the claim that the 1950s had witnessed ‘the end of ideology.’ Ideology offers a worldview within which particular events, actions, etc – and information about these – can be understood. As MacIntyre emphasizes, it is impossible to escape reliance upon ideology in this sense, especially if one is engaged in practical action: even a thoroughly pragmatic or expedient orientation expresses a distinctive ideology. Moreover, a key component of ideologies is a particular prioritization among value principles, this providing the basis for judgments about the relative significance of potential benefits or costs associated with particular policies or practices.
This should lead us to ask: Do researchers also necessarily rely upon ideological assumptions? Probably few social scientists today would deny this. It is certainly true that to the extent that researchers seek to present practical evaluations and recommendations as legitimated by their research, they cannot avoid reliance upon ideology in this sense of the term. And, if this is true, then any insistence that the impact of research should be maximized at the expense of ideology is disingenuous. What may be involved here is, in effect, a contest between ideologies.
What this suggests is that ‘impact’ is most likely to occur where there is a consonance between the ideological framework within which research operates, and that relied upon by policymakers and practitioners – or where research offers a new framework that seems appealing or productive. An interesting case from this point of view is the way in which, over the course of the twentieth century, a mode of economic thinking operating within a neoliberal ideological framework effectively colonized public discussion and decision-making. Existing and proposed policies came to be assessed almost entirely according to their costs and benefits in these economic terms. We might reasonably ask whether this was a desirable development.
In conclusion, then, we need to adopt a rather more sophisticated view of the relationship between research and policymaking/practice than that assumed by the injunction to maximize the impact of research. Carol Weiss’s words in 1979 remain true today:
There has been much glib rhetoric about the vast benefits that social science can offer if only policy makers paid attention. Perhaps it is time for social scientists to pay attention to the imperatives of policymaking systems and to consider soberly what they can do, not necessarily to increase the use of research, but to improve the contribution that research makes to the wisdom of social policy.
