This collaborative study aims to identify the impact of social media, if any, on academic recognition by peers, and to see if the pattern of social media impact and scientific impact of scholarly publications was different in three institutes located in two different countries.
This study was carried out to assess whether a correlation exists between the social media visibility (altmetrics) and scientific impact (citations) of published refereed articles in astronomy. We have compiled citations and altmetrics of refereed articles between 2016-2019 at Gemini Observatory/NSF’s NOIRLab (Gemini), W.M. Keck Observatory (Keck) and the Physical Research Laboratory (PRL). The study was aimed to identify the impact of social media, if any, on academic recognition by peers. This collaborative study identifies the existence of a positive correlation between the citations and altmetrics and also explores the characteristics of the correlation for the publications from the three institutions.
We all know that research results get disseminated through scholarly communication and that scholarly publishing is a cost intensive process. In recent years, electronic publishing and the Internet have facilitated easy access to scholarly content. This has led to the popularity of the Open Access model of publishing.
Researchers used to find relevant articles for their research by browsing journals, attending meetings, and communicating with peers. In today’s digital world they rely on keyword searches or online browsing. Google Scholar browsing yields articles in decreasing order of citations. In any search result, a publication with more citations is seen at the top and gradual scrolling shows the decreasing number of citations. This is probably because of the assumption that more highly cited articles are more useful . However, citations need time to be added/collected hence they are not the best indicator of important recent work.
The development of Web 2.0 has further changed the research publication scenario within, as well as outside, of academics. It has also provided new innovative tools to measure the broad scientific impact of scholarly work. Altmetrics is a term coined in 2010 by ImpactStory co-founder Jason Priem and refers to metrics and data complementary to citations. Many altmetric services have come into existence in the past few years. They assess the impact not only with citations but also with digital use and sharing of data such as tweets, likes on Facebook, coverage by media or blogs, and numbers of downloads or bookmarks online .
Altmetrics provide quantitative and qualitative data that can tell a lot about how often journal articles and other scholarly outputs like datasets are discussed and used around the world. They can include discussions on research blogs, bookmarks on reference managers like Mendeley, and mentions on social networks such as Twitter and Facebook.
In other words, it is an alternative measure of academic impact and it is gaining in global importance. As a fairly nascent discipline, altmetrics is quickly changing the dynamic study of scholarly communication .
Blogs, Twitter, and Mendeley are the platforms most used to share and disseminate information about the scholarly articles published. Generally, an article may be most tweeted on its publication day and most blogged within a month of publication.
According to Priem & Hemminger , Altmetrics or alternative metrics can also be described as collection and measure of online events relating to the consumption and dissemination of scientific documents. They had proposed that the first step in scientometrics 2.0 is to access article related data from as many of these sources (Twitter, blogs, Mendeley readership) as possible, compiling descriptive statistics and examining the quality of the data. According to Priem & Hemminger, one of the strengths of web-based metrics is the ability to examine impacts of other forms of scholarly communication as well.
A study carried out by Mohamadi and Thelwal  suggests that Mendeley readership data could be used to help capture knowledge transfer across disciplines, especially for people that read but do not write articles. Additionally, it can give evidence of the impact at an earlier stage than is possible with citation counts. It is fairly clear from this study that Mendeley may be universally useful to gauge the research impact throughout all areas of scholarship as it covers broader types of users such as undergraduates, postgraduates, and practitioners while citation data comes only from authors. Mendeley data can also be extracted earlier than citations because of the lack of publication delays of citing papers.
Didegah, et al.  carried out a study to identify the factors affecting the citation counts and whether these also affect the altmetric counts. They found that journal impact factor (JIF), research funding, and subject discipline significantly affect the citation counts. A very interesting find of this study was that the JIF effect is even higher with Twitter posts than with citations, indicating that high impact journals are also popular on social media platforms. It is important to note that journals like Nature, Science, PLoS, etc. have Twitter accounts and tweet their new articles regularly. They also found that the more collaborators on a publication, the more tweets, blog posts, and news posts it solicits.
Another study carried out by O’Connor, et al.  tried to assess whether a correlation exists between newsworthiness (through social media) and scientific impact in the field of Urology. They found that top articles based on altmetric scores were not highly cited, suggesting that papers receiving the most media attention may not be the most scientifically rigorous.
A short study carried out by Rajendra Babu and Vysakh  compared citations and altmetric attention scores of highly cited papers in Nature. They found that Mendeley readership has a high impact on determining the altmetric score of a publication. They also found that countries with more Twitter users tweet more about scholarly articles. They proposed that the emergence of altmetrics and its acceptability by funding agencies have prompted the research community to support open access by creating an open metrics environment.
Another recent study done by Azer and Azer  has tried to assess the characteristics of highly cited articles in the medical field and their altmetric scores. They did not find any correlation between citation counts and number of authors, female authorship, or number of grants received. A correlation was however found between citation counts and altmetric scores for the papers published after 2007, not before.
The present study aims to learn whether altmetric measures and citations are related. Do the altmetric counts act as early indicators of the quality of the paper? This collaborative study also seeks to identify whether the pattern of social media impact and scientific impact of scholarly publications are different among the three institutes that are located in two different locations.
To carry out this study, it was important to know some background of each Institute.
Gemini Observatory is an astronomical observatory consisting of two 8.1-metre telescopes, Gemini North and Gemini South, which are located in Hawai’i and Chile.
Keck Observatory is a twin-telescope astronomical observatory at an elevation of 4,145 meters (13,600 ft) near the summit of Mauna Kea in Hawai’i. Both telescopes have 10 m (33 ft) aperture primary mirrors.
The Physical Research Laboratory (PRL), known as the cradle of space sciences in India, was founded by Dr. Vikram Sarabhai in 1947. As a unit of the Department of Space and the Government of India, PRL carries out fundamental research in selected areas of physics, astronomy, astrophysics & solar physics, space & atmospheric sciences, and planetary & geo-sciences.
The period of this study is limited to years 2016-2019.
Citation data were extracted from NASA ADS in June 2021 for the refereed publications published in 2016-2019 for the three institutions – Gemini, Keck, and PRL. The altmetrics for the publications were obtained by using the Altmetric Bookmarklet (available at altmetric.com). The higher the altmetric score an article gets, the more it has been shared on social media such as Twitter, Blogs, Mendeley, and Facebook.
Table 1-3 shows the total and average altmetrics and ADS citations for the refereed papers by year for Gemini, Keck, and PRL respectively.
Number of papers
Altmetrics per paper
Citations per paper
Number of papers
Altmetrics per paper
Citations per paper
Number of papers
Altmetrics per paper
Citations per paper
The above tables indicate that Gemini and Keck publications obtained an altmetric score that is about double the citations, while PRL publications received altmetric and citations numbers that are close, with citations higher than the altmetrics.
The authors then extracted the top 200 papers (top 50 in each year) with the most citations and the top 200 (top 50 in each year) papers with the most altmetrics score to identify which journals attracted more altmetrics or citations. To decrease the noise for the analysis, the following major journals in which most of the sample papers were published were selected: Astronomy and Astrophysics (A&A), Astronomical Journal (AJ), Astrophysical Journal (ApJ), Monthly Notices of the Royal Astronomical Society (MNRAS), Nature, Nature Astronomy, and Science.
A positive correlation can be found from figures 1 between altmetrics and citations for the top cited publications. Pearson’s Correlation Coefficient was calculated for each institute. Values between 0 and 1 indicate a positive linear correlation: 0.96 for PRL, 0.41 for Gemini, and 0.32 for Keck.
Most cited publications published in astronomical journals (A&S, AJ, ApJ, and MNRAS) generally have higher citation totals than altmetric totals, if not very close, while those published in science journals (Nature, Nature Astronomy, and Science) papers tend to have much higher altmetric totals than citation totals.
Again, a positive correlation can be found from figures 2 between altmetrics and citations for publications with the most altmetrics. However, for Gemini and Keck publications, altmetric totals are all higher than citation totals for both astronomical journals and science journals, with ApJ and Nature papers showing the largest gaps. PRL papers display tight correlations between altmetric totals and citation totals for both most cited and most altmetric publications.
Figures 3 and 4 repeat the above analysis for average altmetrics and citations instead of totals. For Gemini and Keck the most cited publications, the average altmetrics and citations per paper are very close for astronomical journals. Again, science journals display much higher average altmetrics than average citations. For Gemini and Keck papers with the most altmetrics, the average altmetrics and citations per paper are also close but with higher average altmetrics for astronomical journals. For PRL papers, the average altmetrics and citations display similar patterns for both papers with most altmetrics and most citations. Unlike Gemini and Keck papers, PRL papers published in AJ generate much higher average altmetrics than average citations compared to other astronomical journals.
Based on the analysis of the top 50 publications most cited and with most altmetrics per year for the three institutions, a positive correlation pattern has been identified between the scientific impact and social media impact for refereed astronomical journal articles. The most cited papers generate more altmetrics and vice versa, with the following characteristics: First, papers published in science journals have much higher social media impact than those published in astronomical journals, especially for Gemini and Keck papers. But those papers usually don’t generate higher citations. Second, Gemini and Keck publications display similar patterns while PRL publications show a very different pattern. One possibility is that Gemini and Keck own 8-10 m telescopes which produce more cutting-edge discoveries and attract more media attention. Third, Nature is a clear winner in generating higher social media impact among journals. For PRL, it’s Nature Astronomy. Although ApJ appears to produce the most total citations, its average citation counts are actually comparable to other journals except for PRL papers. PRL papers published in AJ generate the second highest altmetrics.
Also, the authors that are based in the US share more through social media making their papers more visible. Digital marketing expert Brian Dean told Newsweek: "According to Twitter's SEC filings, the U.S. has the most Twitter users in the world.” A Google search about blogs also yields the same results. Low altmetric scores of PRL papers could also indicate that researchers at PRL have yet to open up to social media and use it to increase the visibility of their papers and thereby their impact.
It would be interesting to know which areas of research generate the most citations or altmetrics, for example, exoplanets or black holes. Other factors such as the number of authors may also be explored.