In scholarly and scientific publishing, altmetrics are non-traditional bibliometrics proposed as an alternative or complement to more traditional citation impact metrics, such as impact factor and h-index. The term altmetrics was proposed in 2010, as a generalization of article level metrics, and has its roots in the #altmetrics hashtag. Although altmetrics are often thought of as metrics about articles, they can be applied to people, journals, books, data sets, presentations, videos, source code repositories, web pages, etc. Altmetrics use public APIs across platforms to gather data with open scripts and algorithms. Altmetrics did not originally cover citation counts, but calculate scholar impact based on diverse online research output, such as social media, online news media, online reference managers and so on. It demonstrates both the impact and the detailed composition of the impact. Altmetrics could be applied to research filter, promotion and tenure dossiers, grant applications and for ranking newly-published articles in academic search engines.
The development of web 2.0 has changed the research publication seeking and sharing within or outside the academy, but also provides new innovative constructs to measure the broad scientific impact of scholar work. Although the traditional metrics are useful, they might be insufficient to measure immediate and uncited impacts, especially outside the peer-review realm.
Projects such as ImpactStory, and various companies, including Altmetric, and Plum Analytics are calculating altmetrics. Several publishers have started providing such information to readers, including BioMed Central, Public Library of Science (PLOS), Frontiers, Nature Publishing Group, and Elsevier.
In 2008, the Journal of Medical Internet Research started to systematically collect tweets about its articles. Starting in March 2009, the Public Library of Science also introduced article-level metrics for all articles. Funders have started showing interest in alternative metrics, including the UK Medical Research Council. Altmetrics have been used in applications for promotion review by researchers. Furthermore, several universities, including the University of Pittsburgh are experimenting with altmetrics at an institute level.
However, it is also observed that an article needs little attention to jump to the upper quartile of ranked papers, suggesting that not enough sources of altmetrics are currently available to give a balanced picture of impact for the majority of papers.
Important in determining the relative impact of a paper, a service that calculates altmetrics statistics needs a considerably sized knowledge base. The following table shows the number of papers covered by services (as of 2016):
|Website||Number of papers|
|Plum Analytics||~ 29.7 million|
|Altmetric.com||> 5 Million|
|ImpactStory||~ 1 Million|
Altmetrics are a very broad group of metrics, capturing various parts of impact a paper or work can have. A classification of altmetrics was proposed by ImpactStory in September 2012, and a very similar classification is used by the Public Library of Science:
One of the first alternative metrics to be used was the number of views of a paper. Traditionally, an author would wish to publish in a journal with a high subscription rate, so many people would have access to the research. With the introduction of web technologies it became possible to actually count how often a single paper was looked at. Typically, publishers count the number of HTML views and PDF views. As early as 2004, the BMJ published the number of views for its articles, which was found to be somewhat correlated to citations.
The discussion of a paper can be seen as a metric that captures the potential impact of a paper. Typical sources of data to calculate this metric include Facebook, Google+, Twitter, Science Blogs, and Wikipedia pages. Some researchers regard the mentions on social media as citations. For example, citations on a social media platform could be divided into two categories: internal and external. For instance, the former includes retweets, the latter refers to tweets containing links to outside documents. The correlation between the mentions and likes and citation by primary scientific literature has been studied, and a slight correlation at best was found, e.g. for articles in PubMed. In 2008 the Journal of Medical Internet Research began publishing views and tweets. These "tweetations" proved to be a good indicator of highly cited articles, leading the author to propose a "Twimpact factor", which is the number of Tweets it receives in the first seven days of publication, as well as a Twindex, which is the rank percentile of an article's Twimpact factor. However, if implementing use of the Twimpact factor, research shows scores to be highly subject specific, and as a result, comparisons of Twimpact factors should be made between papers of the same subject area. It is necessary to note that while past research in the literature has demonstrated a correlation between tweetations and citations, it is not a causative relationship. At this point in time, it is unclear whether higher citations occur as a result of greater media attention via twitter and other platforms, or is simply reflective of the quality of the article itself.
Recent research conducted at the individual level, rather than the article level, supports the use of twitter and social media platforms as a mechanism for increasing impact value. Results indicate that researchers whose work is mentioned on twitter have significantly higher h-indices than those of researchers whose work was not mentioned on twitter. The study highlights the role of using discussion based platforms, such as twitter, in order to increase the value of traditional impact metrics.
Besides Twitter and other streams, blogging has shown to be a powerful platform to discuss literature. Various platforms exist that keep track of which papers are being blogged about. Altmetric.com uses this information for calculating metrics, while other tools just report where discussion is happening, such as ResearchBlogging and Chemical blogspace.
It is also informative to quantify the number of times a page has been saved, or bookmarked. It is thought that individuals typically choose to bookmark pages that have a high relevance to their own work, and as a result, bookmarks may be an additional indicator of impact for a specific study. Providers of such information include science specific social bookmarking services such as CiteULike and Mendeley.
The cited category is a narrowed definition, different from the discussion. Besides the traditional metrics based on citations in scientific literature, such as those obtained from Google Scholar, CrossRef, PubMed Central, and Scopus, altmetrics also adopt citations in secondary knowledge sources. For example, ImpactStory counts the number of times a paper has been referenced by Wikipedia. Plum Analytics also provides metrics for various academic publications, seeking to track research productivity. PLOS is also a tool that may be used to utilize information on engagement.
While there is less consensus on the validity and consistency of altmetrics, the interpretation of altmetrics in particular is discussed. Proponents of altmetrics make clear that many of the metrics show attention or engagement, rather than the quality of impacts on the progress of science. It should be noted that even citation-based metrics do not indicate if a high score implies a positive impact on science; that is, papers are also cited in papers that disagree with the cited paper, an issue for example addressed by the Citation Typing Ontology project.
Altmetrics could be more appropriately interpreted by providing detailed context and qualitative data. For example, in order to evaluate the scientific contribution of a scholar work to policy making by altmetrics, qualitative data, such as who's citing online and to what extent the online citation is relevant to the policymaking, should be provided as evidence.
Regarding the relatively low correlation between traditional metrics and altmetrics, altmetrics might measure complementary perspectives of the scholar impact. It is reasonable to combine and compare the two types of metrics in interpreting the societal and scientific impacts. Researchers built a 2*2 framework based on the interactions between altmetrics and traditional citations. Further explanations should be provided for the two groups with high altmetrics/low citations and low altmetrics/high citations. Thus, altmetrics provide convenient approaches for researchers and institutions to monitor the impact of their work and avoid inappropriate interpretations.
The usefulness of metrics for estimating scientific impact is controversial. Research has found that online buzz could amplify the effect of other forms of outreach on researchers' scientific impact. For the nano-scientists that are mentioned on Twitter, their interactions with reporters and non-scientists positively and significantly predicted higher h-index, whereas the non-mentioned group failed. Altmetrics expands the measurement of scholar impact for containing a rapid uptake, a broader range of audiences and diverse research outputs. In addition, the community shows a clear need: funders demand measurables on the impact of their spending, such as public engagement.
However, there are limitations that affect the usefulness due to technique problems and systematic bias of construct, such as data quality, heterogeneity and particular dependencies. In terms of technique problems, the data might be incomplete, because it is difficult to collect those online research outputs without direct links to their mentions (i.e. videos) and identify different versions of one research work. Additionally, whether the API leads to any missing data is unsolved.
As for systematic bias, like other metrics, altmetrics are prone to self-citation, gaming, and other mechanisms to boost one's apparent impact. Altmetrics can be gamed: for example, likes and mentions can be bought. Altmetrics can be more difficult to standardize than citations. One example is the number of tweets linking to a paper where the number can vary widely depending on how the tweets are collected. Besides, online popularity may not equal to scientific values. Some popular online citations might be far from the value of generating further research discoveries, while some theoretical-driven or minority-targeted research of great science-related importance might be marginalized online. For example, the top tweeted articles in biomedicine in 2011 were relevant to curious or funny content, potential health applications, and catastrophe.
Altmetrics for more recent articles may be higher because of the increasing uptake of the social web and because articles may be mentioned mainly when they are published. As a result, it might not be fair to compare the altmetrics scores of articles unless they have been published at a similar time. Researchers has developed a sign test to avoid the usage uptake bias by comparing the metrics of an article with the two articles published immediately before and after it.
It should be kept in mind that the metrics are only one of the outcomes of tracking how research is disseminated and used. Altmetrics should be carefully interpreted to overcome the bias. Even more informative than knowing how often a paper is cited, is which papers are citing it. That information allows researchers to see how their work is impacting the field (or not). Providers of metrics also typically provide access to the information from which the metrics were calculated. For example, Web of Science shows which are the citing papers, ImpactStory shows which Wikipedia pages are referencing the paper, and CitedIn shows which databases extracted data from the paper.
Another concern of altmetrics, or any metrics, is how universities or institutions are using metrics to rank their employees make promotion or funding decisions, and the aim should be limited to measure engagement.
The overall online research output is very little and varied among different disciplines. The phenomenon might be consistent with the social media use among scientists. Surveys has shown that nearly half of their respondents held ambivalent attitudes of social media's influence on academic impact and never announced their research work on social media. With the changing shift in open science and social media use, the consistent altmetrics across disciplines and institutions will more likely be adopted.
The specific use cases and characteristics is an active research field in bibliometrics, providing much needed data to measure the impact of altmetrics itself. Public Library of Science has an Altmetrics Collection and both the Information Standards Quarterly and the Aslib Journal of Information Management recently published special issues on altmetrics. A series of articles that extensively reviews altmetrics was published in late 2015.
There is other research examining the validity of one altmetrics or make comparisons across different platforms. Researchers examine the correlation between altmetrics and traditional citations as the validity test. They assume that the positive and significant correlation reveals the accuracy of altmetrics to measure scientific impact as citations. The low correlation (less than 0.30) leads to the conclusion that altmetrics serves a complementary role in scholar impact measurement. However, it remains unsolved that what altmetrics are most valuable and what degree of correlation between two metrics generates a stronger impact on the measurement. Additionally, the validity test itself faces some technical problems as well. For example, replication of the data collection is impossible because of the instant changing algorithms of data providers.
Altmetrics is the study and use of non-traditional scholarly impact measures that are based on activity in web-based environments
Altmetrics and article-level metrics are sometimes used interchangeably, but there are important differences: article-level metrics also include citations and usage data; ...
Academic Medicine is the monthly peer-reviewed medical journal, published by the Association of American Medical Colleges.Academic journal
An academic or scholarly journal is a periodical publication in which scholarship relating to a particular academic discipline is published. Academic journals serve as permanent and transparent forums for the presentation, scrutiny, and discussion of research. They are usually peer-reviewed or refereed. Content typically takes the form of articles presenting original research, review articles, and book reviews. The purpose of an academic journal, according to Henry Oldenburg (the first editor of Philosophical Transactions of the Royal Society), is to give researchers a venue to "impart their knowledge to one another, and contribute what they can to the Grand design of improving natural knowledge, and perfecting all Philosophical Arts, and Sciences."The term academic journal applies to scholarly publications in all fields; this article discusses the aspects common to all academic field journals. Scientific journals and journals of the quantitative social sciences vary in form and function from journals of the humanities and qualitative social sciences; their specific aspects are separately discussed.
The first academic journal was Journal des sçavans (January 1665), followed soon after by Philosophical Transactions of the Royal Society (March 1665), and Mémoires de l'Académie des Sciences (1666). The first fully peer-reviewed journal was Medical Essays and Observations (1733).Altmetric
Altmetric, or altmetric.com, is a data science company that tracks where published research is mentioned online, and provides tools and services to institutions, publishers, researchers, funders and other organisations to monitor this activity, commonly referred to as altmetrics. Altmetric was recognized by European Commissioner Máire Geoghegan-Quinn in 2014 as a company challenging the traditional reputation systems.Altmetric is a portfolio company of Digital Science, which is owned by Holtzbrinck Publishing Group.Article-level metrics
Article-level metrics are citation metrics which measure the usage and impact of individual scholarly articles.Bibliometrics
Bibliometrics is statistical analysis of written publications, such as books or articles. Bibliometric methods are frequently used in the field of library and information science, including scientometrics. For instance, bibliometrics are used to provide quantitative analysis of academic literature or for evaluating budgetary spending. Citation analysis is a commonly used bibliometric method which is based on constructing the citation graph, a network or graph representation of the citations between documents. Many research fields use bibliometric methods to explore the impact of their field, the impact of a set of researchers, the impact of a particular paper, or to identify particularly impactful papers within a specific field of research. Bibliometrics also has a wide range of other applications, such as in descriptive linguistics, the development of thesauri, and evaluation of reader usage.Figshare
Figshare is an online open access repository where researchers can preserve and share their research outputs, including figures, datasets, images, and videos. It is free to upload content and free to access, in adherence to the principle of open data. Figshare is one of a number of portfolio businesses supported by Digital Science.ImpactStory
ImpactStory is an open source, web-based tool that provides altmetrics to help researchers measure the impacts of their research outputs including journal articles, blog posts, datasets, and software. It aims to change the focus of the scholarly reward system to value and encourage web-native scholarship. ImpactStory is a nonprofit organisation funded by the Alfred P. Sloan Foundation and the National Science Foundation.Impact factor
The impact factor (IF) or journal impact factor (JIF) of an academic journal is a measure reflecting the yearly average number of citations to recent articles published in that journal. It is frequently used as a proxy for the relative importance of a journal within its field; journals with higher impact factors are often deemed to be more important than those with lower ones. The impact factor was devised by Eugene Garfield, the founder of the Institute for Scientific Information. Impact factors are calculated yearly starting from 1975 for journals listed in the Journal Citation Reports.Journal ranking
Journal ranking is widely used in academic circles in the evaluation of an academic journal's impact and quality. Journal rankings are intended to reflect the place of a journal within its field, the relative difficulty of being published in that journal, and the prestige associated with it. They have been introduced as official research evaluation tools in several countries.List of open-access projects
Some of the most important open-access publishing projects or lists of such projects are listed below.Multiple Sclerosis Discovery Forum
Multiple Sclerosis Discovery Forum (MSDF) is a non-profit online resource created to speed progress toward a cure for multiple sclerosis (MS) and other demyelinating diseases by enabling faster sharing of information and free discussion among MS researchers in academia, industry, and the clinic.
Launched in April 2012, MSDF deploys science journalism as a primary tool in fostering communication and collaboration among MS researchers from all corners of the scientific enterprise. The site combines news and features with technical resources, such as a weekly editor-curated index of MS-related papers from PubMed and a database with the latest scientific and regulatory information about drugs being marketed or in the pipeline for treatment of MS. Other resources include interactive data visualizations, meetings and events, and discussion forums.
MSDF is modeled after online scientific community Alzheimer Research Forum (AlzForum). Since 1996, AlzForum has become an location for information and interaction for investigators working on age-related neurodegeneration. More recently, similar independent neutral Web-based neurology disease forums have followed, including Schizophrenia Research Forum and Pain Research Forum.As with its sister forums, all content on MSDF is provided free of charge to the research community, and editorial independence from sponsors and donors is strictly maintained. MSDF articles have unique digital object identifiers (DOI) to provide stable linking over time and to facilitate discussion and altmetrics tracking of scientific articles in social media forums, s. MSDF articles are indexed by Google News.
MSDF covers the plausible but unproven questions of whether the dozen new anti-inflammatory therapies can be deployed more effectively against disease progression, as well as the upsurge in research to understand the pathological mechanisms and treatments for progressive MS. Related demyelinating conditions include neuromyelitis optica (NMO), transverse myelitis, acute disseminated encephalomyelitis, and optic neuritis. Their misdiagnosis as MS can lead to inappropriate and even harmful therapeutic choices, such as was discovered with NMO, now clearly understand to be a different disease.
MSDF is a joint activity of Accelerated Cure Project for Multiple Sclerosis (ACP) and the MassGeneral Institute for Neurodegenerative Disease (MIND).Open science
Open science is the movement to make scientific research (including publications, data, physical samples, and software) and its dissemination accessible to all levels of an inquiring society, amateur or professional. Open science is transparent and accessible knowledge that is shared and developed through collaborative networks. It encompasses practices such as publishing open research, campaigning for open access, encouraging scientists to practice open notebook science, and generally making it easier to publish and communicate scientific knowledge.
Open Science can be seen as a continuation of, rather than a revolution in, practices begun in the 17th century with the advent of the academic journal, when the societal demand for access to scientific knowledge reached a point at which it became necessary for groups of scientists to share resources with each other so that they could collectively do their work. In modern times there is debate about the extent to which scientific information should be shared. The conflict that led to the Open Science movement is between the desire of scientists to have access to shared resources versus the desire of individual entities to profit when other entities partake of their resources. Additionally, the status of open access and resources that are available for its promotion are likely to differ from one field of academic inquiry to another.Plum Analytics
Plum Analytics is a Philadelphia, Pennsylvania-based altmetrics company dedicated to measuring the influence of scientific research. It was founded in 2011 by Andrea Michalek, who is its current president, and Mike Buschman. It was acquired by Elsevier in February 2017, which purchased it from EBSCO Information Services for an undisclosed amount. Its metrics were immediately incorporated into Elsevier's existing products, including Mendeley and Scopus.ScienceOpen
ScienceOpen is an interactive discovery environment for scholarly research across all disciplines. It is freely accessible for all and offers hosting and promotional services within the platform for publishers and institutes. The organization is based in Berlin and has a technical office in Boston. It is a member of CrossRef, ORCID, the Open Access Scholarly Publishers Association, STM Association and the Directory of Open Access Journals. The company was designated as one of “10 to Watch” by research advisory firm Outsell in its report “Open Access 2015: Market Size, Share, Forecast, and Trends.”Semantometrics
Semantometrics is a tool for evaluating research. It is functionally an extension of tools such as bibliometrics, webometrics, and altmetrics, but instead of just evaluating citations – which entails relying on outside evidence – it uses a semantic evaluation of the full text of the research paper being evaluated.Steve Pettifer
Stephen Robert Pettifer (born September 21, 1970) is a Professor in the School of Computer Science at the University of Manchester, UK.Webometrics
The science of webometrics (also cybermetrics) tries to measure the World Wide Web to get knowledge about the number and types of hyperlinks, structure of the World Wide Web and usage patterns. According to Björneborn and Ingwersen (2004), the definition of webometrics is "the study of the quantitative aspects of the construction and use of information resources, structures and technologies on the Web drawing on bibliometric and informetric approaches." The term webometrics was first coined by Almind and Ingwersen (1997). A second definition of webometrics has also been introduced, "the study of web-based content with primarily quantitative methods for social science research goals using techniques that are not specific to one field of study" (Thelwall, 2009), which emphasizes the development of applied methods for use in the wider social sciences. The purpose of this alternative definition was to help publicize appropriate methods outside of the information science discipline rather than to replace the original definition within information science.
Similar scientific fields are Bibliometrics, Informetrics, Scientometrics, Virtual ethnography, and Web mining.
One relatively straightforward measure is the "Web Impact Factor" (WIF) introduced by Ingwersen (1998). The WIF measure may be defined as the number of web pages in a web site receiving links from other web sites, divided by the number of web pages published in the site that are accessible to the crawler. However the use of WIF has been disregarded due to the mathematical artifacts derived from power law distributions of these variables. Other similar indicators using size of the institution instead of number of webpages have been proved more useful.