Feature article: New Developments in Research Metrics

In scholarly publishing, metrics such as the impact factor and h-index are powerful tools for assessing the impact of research outputs. However, these metrics have limitations, a major one being that they are unable to track publications’ impact outside academia. Also, as measures of citation counting, they do not reveal the context and reasons for a publication being cited.

To supplement these conventional metrics, Altmetrics (alternative metrics) were proposed in 2010 for measuring the broader impact of scholarly outputs (Priem, Taraborelli, Groth, & Neylon, 2010). They include a broad range of indicators, e.g.:

a record of attention, indicating how many people are exposed to a scholarly publication. Examples are HTML page views and PDF downloads.
a measure of dissemination, showing how a publication is being discussed and shared. Examples are sharing in social networking sites and coverage in mainstream media.
an indicator of influence and impact, representing how a piece of research is having an impact on a field of study or the larger society. Examples include commentary from working professionals or references in public policy documents (altmetric.com, n.d.a).

Altmetrics supplement conventional metrics in various ways. First, Altmetrics can help researchers showcase the broader influence of their scholarly work, such as revealing the number of readers of a given paper on online reference managers (e.g. Mendeley). They can also show the citation of a paper by major news outlets. These data can demonstrate a variety of impacts of researchers’ work in their CVs or for grant application.

Researchers can also track early engagement with their work, as an article might be assessed by thousands of conversations and bookmarks in just a week (altmetric.com, n.d.b). This comes much quicker than the release of citation data. Scholars can receive useful feedback that leads to improvements in their research.

Altmetrics can be used for discovery purposes (altmetric.com, n.d.b). Researchers can look at what types of publication prompt discussion, which helps to identify research trends in an area. They can also contribute to the conversation around the work of others.

Altmetrics are not without limitations, however. A notable weakness is that altmetric data such as social media metrics and usage statistics are vulnerable to manipulation (Ronald & Fred, 2013). For example, page views and downloads can be inflated through automated download bots. Facebook Likes, blog posts or other social media mentions can also be bought. Meanwhile, even their proponents have emphasized that Altmetrics are about showing engagement, instead of the impact of a scholarly work (Lin & Fenner, 2013). One can easily imagine that a much discussed paper on the Web can merely be controversial because of a scientific hoax or obvious mistakes in the paper.

Given these limitations, it is important that academics use and interpret Altmetrics carefully. Altmetrics should be regarded as a complement to, not a replacement for, conventional metrics, for understanding the full impact of a scholarly work. In light of the potential for gaming of Altmetrics, academics should also look at the underlying qualitative information when interpreting altmetric data.

Further details and resources about Altmetrics are available in Research Resources section of this issue of the Research Bulletin.

References

Almetric.com (n.d.a). Altmetric for researchers. Retrieved from https://www.altmetric.com/audience/researchers/

Almetric.com (n.d.b). What are Altmetrics. Retrieved from https://www.altmetric.com/about-altmetrics/what-are-altmetrics/

Lin, J., & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quaterly, 25(2), 20.

Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved from http://altmetrics.org/manifesto/

Ronald, R., & Fred, Y. Y. (2013). A multi-metric approach for research evaluation. Chinese Science Bulletin, 295(1), 90–93.

    
 

 

© Copyright 2016. The Open University of Hong Kong. All Rights Reserved.