Charting the Road From Attention to Citation

Citation has been the widely accepted metric of impact of a scientific publication for decades. Besides citations, the impact of an article could be reflected and quantified by some alternative metrics, including article views, Mendeley readerships and Altmetric scores, etc. There were lots of studies on the relationship between citations and other metrics to examine the reliability of different metrics. Significant correlation between views and citations is confirmed by previous studies, but some researchers also found that the correlation between altmetrics and citations is low.

This article by Xianwen Wang originally appeared on the LSE Impact of Social Sciences blog as “From Attention to Citation: What are altmetrics and how do they work?” and is reposted under the Creative Commons license (CC BY 3.0).

Usually, academic publishers provide two main browsing ways to read articles, which are browser HTML views and PDF views, each of them has its individual characteristics. Our questions are that, do different types of view patterns make same impact on citations? Furthermore, does distinguishing different types of view patterns offer a more reasonable description for measuring the impact of an article? Although previous studies show that there is low correlation between altmetrics and citation, how is the relationship between altmetrics and other metrics, e.g., article views?

In order to answer these questions, we selected 64,305 PLOS (Public Library of Science) research articles to analyze the relationship among different metrics. Articles are divided by the publication years in our study. Then, the annual Spearman correlation coefficients between citation and other metrics are calculated. We find that the citation correlates with PDF views much better (correlation coefficients range from 0.52 to 0.77) than with HTML views (range from 0.44 to 0.68). It seems that PDF views have the greater potentiality to lead to academic citations. And the correlation between citations and Altmetric scores is very weak, lower than 0.25.

Note: All correlation coefficients are statistically significant at the 1% level. To better demonstrate the result, here we use color scales and column sparklines to show the show the general distribution of values. For color scales, cells are shaded with gradations of colors that correspond to minimum (green), midpoint (yellow), and maximum values (red). Besides, the column sparklines in the last column show the performance of the values, and highlight the highest value.

Relatively strong correlation between altmetrics and article views

Different viewing patterns do make different impact on citations. But does the weak correlation between Altmetric scores and citations certainly indicates altmetrics have little or no effect on citations?

With this question, the correlation coefficients between Altmetric scores and other three metrics of view (HTML and PDF) are calculated too. We find that the correlation between Altmetric scores and HTML views is the highest (range from 0.35 to 0.40), when the correlation between Altmetric scores and PDF views is lowest (range from 0.23 to 0.30). The results reveal that although the direct correlation between Altmetric scores and citations is weak, the correlation between Altmetric scores and article views, especially HTML views, is relatively high.

Note: All correlation coefficients are statistically significant at the 1 percent level.

Exploring the relationship among different metrics

In our study, a conceptual model is proposed to interpret the relationship among social attention, article view and academic citation. Social attention and academic citation are two ends of the relationship chain. There is little possibility for researchers to cite an article without even reading it. So, there is no link from the end of social attention to another end of citation. On the contrary, the relatively strong correlation between Altmetric scores and HTML views indicates that social attention may bring network traffic to browser HTML view, and a part of the HTML view may generate citation directly, when some would lead to PDF downloads and a part of them turn into citations, as the orange route shows.

Note: Different routes are visualized in different colors, the orange arrows indicate the route originated from social attention and ended citation finally, when the purple arrows are the route originated from browser HTML view. The decreased arrow size of the same color indicates the conversions rates from one status to the next.

Although correlation between social attention and citation is rather weak, social attention doesn’t have much influence on the citation directly, however, it may cause article views and lead to citations subsequently. In other words, more social attention bring more downloads, and more downloads bring more citations. Social attentions do have the potentiality to contribute some extra citations to the paper.

Further details of the study can be found here.