Many observers have accused our universities, and especially our business schools, of behaving more like corporations – focusing on ROIs that benefit us and ignoring the ecosystems in which we operate. The UK’s Research Excellence Framework (REF) stands out as one of the most influential governmental efforts to provide accountability for public investment in research and to establish reputational yardsticks. In the REF 2021, 25 percent of a university’s evaluation depended on the impact, socioeconomic effects, and benefits of the research beyond academia. For the most part, such demands for external accountability do not exist for universities in other parts of the world. Yet the REF has also been criticized for subjectivity as well as the heavy demands it places on faculty to identify and to describe examples of impact.

In the Academy of Management study we mentioned in the first article of this series, senior scholars showed persistent themes of high concern regarding the measures that institutions use to gauge scholarly impact, effects on career development, management research’s value, and societal benefits. Most of the scholars stated that the present system of faculty evaluation and business school rankings led to overreliance on more traditional techniques and methodologies, and what journal editors find acceptable. Some scholars identified that these developments had led to “junk science,” journals as “incestuous outlets for career-aspiring management academics,” under-reliance on ideas, community and society, and excessive “balkanization” as management scholars became “angels dancing on a pin head.” Some raised concerns about the universal applicability and acceptance abroad of US faculty evaluation standards and research approaches that diminish scholarly impact. One scholar categorized the spread of US research standards globally as amounting to “imperialism” and a form of “colonialism,” with disregard for context.

In the major academic journals for the field of management, we have seen an exponential rise in discussions and mentions of impact. Simultaneously, we have seen an exponential decrease in the influence our research has on regulation and on our ability to catch the attention of external constituencies (see Haley, 2021). Ironically, business schools have incentivized discussing scholarly impact, especially in the major journals; yet, simultaneously, administrators and evaluators have at best ignored, almost never rewarded, and sometimes punished, actually having external impact.

In the medium term, we need to ask: Who does this system of research benefit, and how do we throw a wider net? As the surveys we have identified previously indicate, most business school scholars appear to conduct rigorous research that speaks to just a few people as such research advances their careers. Yet, business schools’ administrators could use promotions, tenure or research support to reward researchers who identify how their projects have generated community outreach or benefited certain audiences.

Second, journal publications could also ask for broader impact statements rather than an “implications” section that few read except as afterthoughts. In the US, researchers who have applied for National Science Foundation grants know that they have to identify the broader impacts of their research or show that it has the potential to benefit society and contribute to the achievement of desired societal outcomes. The impact statement assumes such importance that a poor one can sink a proposal, even if it displays good science.

Third, as so many scholars (see Aguinis et al., 2014) and the Association to Advance Collegiate Schools of Business have suggested, we may also expand measures of scholarly impact to correspond with universities’ and academic associations’ missions. Simultaneously, we may increase awareness of the strengths and weaknesses of alternate metrics.

Fourth, researchers could partner with outside technical specialists in academic fields, as well as people working in businesses and government for broader and interdisciplinary views. As Altmetric points out in this series, currently, the business fields display excessive balkanization and specialization.

Fifth, many experienced scholars have proposed investing resources in disseminating the knowledge we create to make it less abstruse and more accessible. When academic vocabularies change to include external stakeholders, when we acknowledge their aspirations and needs, we extend the reach of our ideas.

Finally, the AOM experts recommended that business schools pursue alliances with other academic associations to create interdisciplinary collaborations and achieve wider impact. Many other disciplines, including political science and psychology, have echoed similar concerns of silos erupting in academia. Business schools are part of communities and ecosystems, and we cannot go it alone. To solve the issue of social and external impact, we need to rise above the level of individual academic disciplines.

Despite these medium-term recommendations, lengthy time scales, difficulties in attribution, subjectivity in judgment, complexity in the variety of potential impacts and the intensity of resources for evaluating impact may endanger the generation of truly universal societal-impact measures.

Consequently, short-term, less-complex metrics can move the needle on acquiring more complete data to measure scholarly impact. We suggest that such metrics may include:

• Standardized, more broadly adopted, open-access classification systems for individual journal articles (DOI), other forms of academic outputs (books, chapters, patents, grants), authors (Orcid), and business-school level institutions (RoR) that would permit greater analysis including through platforms such as OpenAlex;

• Refinement of citation data to differentiate positive and negative citations to research, self citations, and if possible, coerced citations from editors and journals;

• Greater emphasis by academics, institutions and publications on non-scholarly outputs such as blogs, video summaries, contributions to Harvard Business Review/Sloan Management Review/Stanford Social Innovation Review and similar, op-eds in the general media, and articles in The Conversation with links to the underpinning peer-reviewed research;

• Greater recognition and reward by administrators to incentivize impactful activities;

• Greater involvement of practitioners alongside academics from the conception of research projects to ensure design for maximum relevance;

• More exploration of scaled up crowdsourcing of practitioner networks to signal the most useful and powerful publications produced;

• Greater investment in ecosystems of citations by encouraging practitioners (and the academics they contact privately) to signal more openly when they draw on academic research to foster audit trails of attribution;

• Greater documentation by faculties and publishers of visits to and downloads from websites detailing their research to measure the types of organizations that are seeking and using these data;

• Greater focus on intermediate outputs such as Altmetric and Overton which identify uptake of rigorous outputs by credible disseminators/influencers bridging academia with the world of practice; and,

• Exploration of text-mining tools to interpret full-text content and impact in articles mapped to relevant societal goals such as the United Nations Sustainable Development Goals.

Upcoming Articles in This Series

“A Quick Examination of Existing Academic Impact Metrics and Concerns in Business Education” | Usha Haley and Andrew Jack

“How Might Societal Impact be Recognized within an FT Top 50 Journal?” | Renate E. Meyer

“A Decades-Long Journey of Marketing and Public Policy Research to Support the Greater Good” | Maura L. Scott

“Why Don’t Business Schools Publish More Impactful Research?” | Ben McLeish and Mike Taylor

“Some Opportunities for Future Business & Management Research: Employee Health and Well-Being” | Sir Cary L. Cooper

“Efforts to Turn the Tide” | Usha Haley and Andrew Jack

“Medium And Short-term Recommendations to Move Forward” | Usha Haley and Andrew Jack