Impact

Charting the Road From Attention to Citation

October 30, 2014 1254

alternatives_optCitation has been the widely accepted metric of impact of a scientific publication for decades. Besides citations, the impact of an article could be reflected and quantified by some alternative metrics, including article views, Mendeley readerships and Altmetric scores, etc. There were lots of studies on the relationship between citations and other metrics to examine the reliability of different metrics. Significant correlation between views and citations is confirmed by previous studies, but some researchers also found that the correlation between altmetrics and citations is low.

LSE Impact logo

This article by Xianwen Wang originally appeared on the LSE Impact of Social Sciences blog as “From Attention to Citation: What are altmetrics and how do they work?” and is reposted under the Creative Commons license (CC BY 3.0).

Strong/weak correlation between citation and article views/Altmetric scores

Usually, academic publishers provide two main browsing ways to read articles, which are browser HTML views and PDF views, each of them has its individual characteristics. Our questions are that, do different types of view patterns make same impact on citations? Furthermore, does distinguishing different types of view patterns offer a more reasonable description for measuring the impact of an article? Although previous studies show that there is low correlation between altmetrics and citation, how is the relationship between altmetrics and other metrics, e.g., article views?

In order to answer these questions, we selected 64,305 PLOS (Public Library of Science) research articles to analyze the relationship among different metrics. Articles are divided by the publication years in our study. Then, the annual Spearman correlation coefficients between citation and other metrics are calculated. We find that the citation correlates with PDF views much better (correlation coefficients range from 0.52 to 0.77) than with HTML views (range from 0.44 to 0.68). It seems that PDF views have the greater potentiality to lead to academic citations. And the correlation between citations and Altmetric scores is very weak, lower than 0.25.

Note: All correlation coefficients are statistically significant at the 1% level. To better demonstrate the result, here we use color scales and column sparklines to show the show the general distribution of values. For color scales, cells are shaded with gradations of colors that correspond to minimum (green), midpoint (yellow), and maximum values (red). Besides, the column sparklines in the last column show the performance of the values, and highlight the highest value.

Note: All correlation coefficients are statistically significant at the 1% level. To better demonstrate the result, here we use color scales and column sparklines to show the show the general distribution of values. For color scales, cells are shaded with gradations of colors that correspond to minimum (green), midpoint (yellow), and maximum values (red). Besides, the column sparklines in the last column show the performance of the values, and highlight the highest value.

Relatively strong correlation between altmetrics and article views

Different viewing patterns do make different impact on citations. But does the weak correlation between Altmetric scores and citations certainly indicates altmetrics have little or no effect on citations?

With this question, the correlation coefficients between Altmetric scores and other three metrics of view (HTML and PDF) are calculated too. We find that the correlation between Altmetric scores and HTML views is the highest (range from 0.35 to 0.40), when the correlation between Altmetric scores and PDF views is lowest (range from 0.23 to 0.30). The results reveal that although the direct correlation between Altmetric scores and citations is weak, the correlation between Altmetric scores and article views, especially HTML views, is relatively high.

Note: All correlation coefficients are statistically significant at the 1 percent level.

Note: All correlation coefficients are statistically significant at the 1 percent level.

Exploring the relationship among different metrics

In our study, a conceptual model is proposed to interpret the relationship among social attention, article view and academic citation. Social attention and academic citation are two ends of the relationship chain. There is little possibility for researchers to cite an article without even reading it. So, there is no link from the end of social attention to another end of citation. On the contrary, the relatively strong correlation between Altmetric scores and HTML views indicates that social attention may bring network traffic to browser HTML view, and a part of the HTML view may generate citation directly, when some would lead to PDF downloads and a part of them turn into citations, as the orange route shows.

Note: Different routes are visualized in different colors, the orange arrows indicate the route originated from social attention and ended citation finally, when the purple arrows are the route originated from browser HTML view. The decreased arrow size of the same color indicates the conversions rates from one status to the next.

Note: Different routes are visualized in different colors, the orange arrows indicate the route originated from social attention and ended citation finally, when the purple arrows are the route originated from browser HTML view. The decreased arrow size of the same color indicates the conversions rates from one status to the next.

Although correlation between social attention and citation is rather weak, social attention doesn’t have much influence on the citation directly, however, it may cause article views and lead to citations subsequently. In other words, more social attention bring more downloads, and more downloads bring more citations. Social attentions do have the potentiality to contribute some extra citations to the paper.

Further details of the study can be found here.


Xianwen Wang is an associate professor at WISE Lab, Dalian University of Technology in China. His research interests include scientometrics, altmetrics (especially article usage study), and scholarly communication. His personal website is xianwenwang.com.

View all posts by Xianwen Wang

Related Articles

Canada’s Storytellers Challenge Seeks Compelling Narratives About Student Research
Communication
November 21, 2024

Canada’s Storytellers Challenge Seeks Compelling Narratives About Student Research

Read Now
Tom Burns, 1959-2024: A Pioneer in Learning Development 
Impact
November 5, 2024

Tom Burns, 1959-2024: A Pioneer in Learning Development 

Read Now
Alondra Nelson Named to U.S. National Science Board
Announcements
October 18, 2024

Alondra Nelson Named to U.S. National Science Board

Read Now
Lee Miller: Ethics, photography and ethnography
News
September 30, 2024

Lee Miller: Ethics, photography and ethnography

Read Now
‘Settler Colonialism’ and the Promised Land

‘Settler Colonialism’ and the Promised Land

The term ‘settler colonialism’ was coined by an Australian historian in the 1960s to describe the occupation of a territory with a […]

Read Now
Research Assessment, Scientometrics, and Qualitative v. Quantitative Measures

Research Assessment, Scientometrics, and Qualitative v. Quantitative Measures

The creation of the Coalition for Advancing Research Assessment (CoARA) has led to a heated debate on the balance between peer review and evaluative metrics in research assessment regimes. Luciana Balboa, Elizabeth Gadd, Eva Mendez, Janne Pölönen, Karen Stroobants, Erzsebet Toth Cithra and the CoARA Steering Board address these arguments and state CoARA’s commitment to finding ways in which peer review and bibliometrics can be used together responsibly.

Read Now
Paper to Advance Debate on Dual-Process Theories Genuinely Advanced Debate

Paper to Advance Debate on Dual-Process Theories Genuinely Advanced Debate

Sage 998 Impact

Psychologists Jonathan St. B. T. Evans and Keith E. Stanovich have a history of publishing important research papers that resonate for years.

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments