Abstract
Introduction: Researchers are increasingly interested in appraising the impact of their research work, which eventually drives public perception. The overall impact of a study can only be gauged if we consider both traditional and non-traditional dissemination patterns. Hence, we preferred to study the association between the non-traditional reader engagement metrics and traditional dissemination metrics in relation to coronavirus disease 2019 (COVID-19)-related research published in five high-impact peer-reviewed medical journals.
Method: This observational study was conducted using data sourced from Altmetric, including the Altmetric attention score (AAS), an aggregate score of an article’s dissemination. New England Journal of Medicine (NEJM), Lancet Infectious Diseases, Clinical Infectious Diseases (CID), Chest Journal (CHEST), and Journal of the American Medical Association (JAMA) were included in the study based on the prevalence of COVID-19-related original research published in each of them. The number of citations was framed as the reference for traditional metrics. To avoid artificial variance, data were collected on the same day, November 13, 2022. Correlational analyses were performed using the Pearson correlation coefficient using Minitab 17 (Minitab Inc., State College, PA). The relationship between the variables was considered very weak if r<0.3, weak if r: 0.3 to 0.5, moderate if r: 0.5 to 0.7, and strong for r>0.7.
Results: We found a very weak correlation between citations and AAS for Clinical Infectious Diseases, Lancet Infectious Diseases, and CHEST, whereas the correlation was moderate for NEJM and JAMA. The correlation between citations and Twitter mentions was very weak for Clinical Infectious Disease, Lancet Infectious Disease, and CHEST, but it improved for NEJM and JAMA. There was a very weak correlation between citations and news mentions for Clinical Infectious Diseases, Lancet Infectious Diseases, and CHEST.
Conclusion: Our study highlights that the traditional indicator, i.e., citation has a very weak to moderate correlation with the AAS and it doesn’t capture the entire influence of a research publication. Also, the current method of determining a journal's impact factor doesn’t take this disparity into consideration. Hence, there needs to have a more inclusive strategy to define the impact of scientific research on the general population in real-time.