Would You Forego Citations for Journal Status?

Guides, ratings and rankings play a pervasive role in eliciting behavior and shaping perceptions. Often, such rankings have obscure origins and change in scope and purpose over time with unintended consequences. However, all rankings and corresponding accolades are only as important as the status and benefits they confer.

In an increasingly competitive academic community these accolades can be considerable. Whilst many critics have pointed out that publications in high-status journals may not automatically be of higher quality, the wide misuse of journal rankings as a one-stop shop to evaluate research continues to be a hot button issue. Something that is reflected by the inroads made by the advocates of the DORA declaration on research assessment into altering the behavior of funding agencies, academic institutions, journals, metric providers, and individual researchers.

Given these developments, it is surprising that there is scant evidence about which academics are subject to the lure of high-rank journal status, or which ones are instead immune to its pull. Our recent work focuses on business and management academics in the UK, who as a body represent a range of social science and humanities scholars. More so than other fields of research, business school academics are also subject to formalized journal ranking systems, such as the Chartered Association of Business Schools ABS Journal Guide. This has led to critiques from within the field, arguing reliance on rankings incentivizes “careerism over creativity” and that, as studies in a US context have indicated, “an A-journal article may often be celebrated as a victory with relatively little conversation about the study’s content, the quality of its methodology and data, and the implications of its findings for theory and practice.” As such, business and management faculty at UK institutions arguably represent ‘canaries in the mine’ for academics in other contexts where such assessment systems are still taking shape.

So, given the choice of publishing in a high-rank journal or a less prestigious one, you might choose to ‘have your cake and eat it’ – that is publish in a high-rank journal over a lower-ranked outlet –, but what if you had something to give up as well? To tackle this issue, we designed a choice-set question that explored individuals’ preferences for publication in ‘elite’ general journals in relation to a hypothetical level of citations in subsequent years in a leading specialized journal. In other words, at what point would you trade journal prestige for the recognition or your peers, as represented through citations?

Our question design was inspired by studies of managerial choice in innovation and research on risk preferences in behavioural economics. In these studies, individuals are offered a set of hypothetical staged choices between two options, which have different degrees of risk. The choice-set fixes the value of one option and then increases the reward aligned to the second option. The purpose being to determine at what stage people shift their choice from the first option to the second. So, we designed a hypothetical choice-set question to assess the preference for a publication in a 4* journal versus publication in a 4-rated journal. We leverage the widespread dissemination of the ABS Academic Journal Guide in the UK. The list ranks journals on a five-point scale (4*, 4, 3, 2, 1), with the highest ranking being ‘Journal of Distinction’ or ‘4*’, which almost exactly overlaps with the club of ‘A journals’ as defined within the US economics and business school scholarly communities. Because citations are commonly used understood by academics, we use them in the hypothetical set as a proxy for the reward associated with each option in a neutral fashion to avoid response bias.

Almost half (44 percent) of the academics we surveyed indicated a preference for publishing in leading specialized journals (4-rated) compared to publishing in high-status ‘elite’ journal (4*), suggesting that for these respondents the appeal of high-status journals was rather modest. However, for the remaining 56 percent of the surveyed academics status was king, with some individuals often being willing to forgo substantial scholarly impact to achieve the benefits of high-status journal publication. This illustrates a tale of two ’worlds’ within the UK business and management academic community. In one ‘world’, academics are indifferent to journal status and are happy to focus on developing their work in 4-rated journals. These individuals may identify their research as being too distant from what gets published in 4* journals, or they may simply prefer to publish in specialized journals. In the other ‘world’, scholars seek opportunities to reinforce their status by publishing in high-rank journals.

The analysis of what distinguished these ‘two worlds’ highlighted three key drivers. We found that it was scholars who had already previously published in ‘elite’ outlets that were most willing to forfeit scholarly impact for ‘elite’ journal status, to reaffirm their affiliation to these high-status outlets. This highlights a self-reinforcing cycle of rankings and professional aspirations. Second, we found that the willingness to pay, in terms of citations for publishing research in 4* journals, was also strong among scholars who have published in 4-rated journals, but might aspire to higher academic rewards. Third, the preference for ‘elite’ journals was also strong at high-ranked institutions. This is perhaps less surprising as these institutions are likely to expect their faculty to engage in ‘elite’ research status competitions, but it does shine a spot-light on how these working environments operate.

These findings point to an urgent need for the use of a more diverse basket of measures in research assessment. In terms of the design of research systems and scientific performance indicators, it may be the case that ‘hybrid’ systems that combine citation based metrics and expert review are better aligned to the range of preferences that exist within the academic community. Indeed, until article-level citations are accorded greater value within the academic community, by institutions, and within ranking systems, it is likely that many academics will continue – not without reason – to prioritize the status of the outlet over the impact of the work reported therein. We also might be seeing a fundamental shift in academic attitudes – arising from ubiquity of rankings – away from the traditional model where reputation among peers was the ‘gold standard’ of academic status, to a mixed model where the status of the journal where research is published is equally (or even more) prized rather than its academic impact.