CHAMPAIGN, Ill. — A small portion of scientific papers are retracted for research that is in error or fraudulent. But those papers can continue to be cited by other scientists in their work, potentially passing along the misinformation from the retracted articles.
Jodi Schneider, a professor of information sciences at the University of Illinois Urbana-Champaign who studies scholarly publications and how information gets used, is considering how scientific journals can better communicate about retracted articles. In a new study published in the journal Scientometrics, she found that a retracted clinical trial report continues to be cited 11 years after its retraction – and that citations actually increased after it was retracted.
Schneider’s team reviewed citations of a 2005 paper in the field of respiratory medicine that found omega-3 fatty acids to be helpful in reducing inflammatory markers in patients with chronic obstructive pulmonary disease. The paper was retracted in 2008 for reporting on falsified clinical trial data.
However, it continues to be cited to support the medical nutrition intervention. Schneider’s analysis covered 148 direct citations of the paper from 2006-2019 and 2,542 second-generation citations. The retraction was not mentioned in 96% of the 112 direct post-retraction citations for which she was able to study the context of the citation. The example demonstrates problems with how the current digital library environment communicates the retraction status of scientific papers, Schneider wrote.
The number of articles that are retracted “is really, really tiny. It’s a regular occurrence that things get retracted, but people publish so much,” Schneider said. Since 2012, there have been about four retractions per 10,000 publications, she wrote in her article.
When a new paper is published, most journals don’t check the bibliographies for citations of retracted research. “It’s really rare for any journal to be scanning reference lists,” Schneider said.
Notices of retractions are often hard to find. Some websites and databases, such as PubMed, make retraction notices easily visible, but many others do not, Schneider said. There may be multiple sites where a copy of a scientific paper can be found, and not all will indicate if it is retracted.
In addition, many databases don’t have complete information about a retraction. Retractions can occur for a number of reasons, including errors made in the research or misconduct such as reporting fraudulent data. About 10% of retraction notices don’t mention the reason for retraction, Schneider said.
“Part of the challenge is understanding the reasons why a particular article was retracted, which often are really vague,” Schneider said.
How an article is cited matters – for example, whether authors seem aware of the retraction, whether they use the retracted paper as a foundation to build on with their own work or whether it is cited for a general concept or the history of research on a particular matter. Analyzing the context as well as the number of citations, Schneider found that more than 41% of the post-retraction citations of the respiratory medicine paper that did not mention the retraction also described the paper in detail.
“We looked at whether they were discussing the methods and results of the retracted paper. The more in-depth they are discussing it, the more they are using it as part of an argument for their work,” she said.
Unlike with most retracted research articles, the respiratory medicine paper’s citations increased substantially after the retraction, Schneider found. One likely factor influencing the number of citations the paper received is that it was the first purported random clinical trial on a research question with very little previous research, she said.
The retraction process can take a long time and involve institutional or governmental investigations, she said. The longer it takes to retract an article that is in error or fraudulent, the more likely it is to be cited in the meantime.
“The current information environment facilitates the spread of research papers, but basic facts about these papers, such as their retraction status, do not spread as swiftly as the PDFs or citations to these papers themselves. Our case study suggests that unknowing and likely unintentional citation of retracted papers could be common, and that post-retraction citation may be correlated with visibility of retraction status,” Schneider wrote.
She has led several workshops for people across scholarly publishing to talk about how journals can better check for citation of retracted articles, and the workshop participants are drafting recommendations. Schneider said best practices for journals should include ensuring every retraction notice is publicly available and establishing a convention for how to flag an article that is retracted. Among the suggestions is developing a standard set of metadata regarding retractions that would be attached to articles and use a taxonomy of statuses, such as retraction or expression of concern.
Information sciences students at Illinois built a prototype tool called ReTracker that would automatically check for retractions among the articles that scholars compile in their online libraries, using data from PubMed, and add the retraction status directly into the library. Zotero, a popular reference-management software system used for managing bibliographic data and research materials, also has developed such a tool, Schneider said.
Ensuring that retracted articles don’t continue to be cited as reliable research is important not just for scientists but also for public confidence in scientific research, Schneider said.
“A scientific paper that is right is like a brick to build walls of evidence we can rely on,” she said.
Schneider's research was supported by the Alfred P. Sloan Foundation.