By July 10th 2022, PubMed, the US National Library of Medicine’s search engine for publications on biomedical topics, had referenced in its data base a total of 274,956 scientific papers of various sorts on the subject of Covid-19. The first of these was published in February 2020, since when the flow of papers has never ceased to grow.
Since the pandemic kicked in, there has been an average daily publication of 307 papers about the virus, an astounding output which demonstrates that no-one can today, even remotely, follow what is being published on Covid. In comparison, during the H1N1 pandemic between June 2009 and May 2010, there was an average of just six new papers per day on the subject.
How could this torrent of new information be canalised by scientific journals? To better appreciate the issue, one must understand how the journals work. When scientists submit a paper to them (most often in English), the journals’ editorial teams, composed of scientists, seek out reviewers who are experts on the subject concerned by the paper, and who will then give their critical appreciation of its worth.
The reviewers, whose identities are most often undisclosed to the authors, can reject a paper, or demand further experiments or analyses, or even that the information be reformulated. The paper will only be published by the journal once several of the reviewers have given the green light, and in the case of disagreement it will be the editor of the journal who decides.
Enlargement : Illustration 1
This process of peer reviewing, upon which the circulation of new scientific knowledge has relied for more than a century, has been doubly thrown into difficulty by the Covid-19 pandemic.
Firstly, because the number of articles that were to be reviewed multiplied enormously in just a few weeks, soon saturating the workload of the reviewers (even when they were in lockdown). Before the Covid-19 pandemic, the subject of coronaviruses was a relatively quite small area of research, and specialists on the topic were rare.
Secondly, because the editors of journals imposed very tight deadlines for peer reviews, in order to accelerate the circulation of new knowledge to address a worldwide emergency.
The result was that in 2020, around 50% of the 31,319 published articles on Covid-19 that year had been peer-reviewed and accepted, on average, in less than eight days following their submission, while another 19.5% were accepted on the same day that they were submitted. In the history of scientific publications, such short timescales have never been seen before. In comparison, the average length of time in 2020 for papers on the subject of flu to have been peer reviewed and accepted for publication was 92 days.
Urgency and haste
To no surprise, the fast-tracking of peer reviews allowed numerous papers of doubtful worth to pass through the net and be published. Again in 2020, there were 82 articles on the subject of Covid that were retracted (either removed from a journal on its own subsequent editorial decision or, more rarely, on the initiative of the paper’s authors). That same year, just one paper on the topic of flu was retracted.
The reason for retracting an article was only made clear in half of the cases. In descending order, the reasons given were plagiarism, errors in the methodology or analysis, ethical problems (for example, the failure to declare a situation of conflict of interest or to give proof of patients’ consent to clinical tests), and lastly, the falsification of results.
Since the beginning of the pandemic, using our own calculations based on information from Retraction Watch, 0,08 % of all papers on the topic of Covid were retracted, which is eight times more than retractions of all papers published since 1982 on the subject of HIV. Furthermore, this clean-up has only just begun, both because it takes time to discover errors and also because a number of articles have not been properly read due to the over-abundance of information published.
There are good reasons to believe that the problems concern a not insignificant fraction of scientific literature on the subject of Covid, but which remain unidentified. This can be due to the fact that they are cited, or because they are included in analyses that present several separate studies in one block.
The most striking example is that out of the 652 citations of a paper published on May 1st 2020 by the New England Journal of Medicine, which was about the increased risk of mortality for Covid patients who have cardiovascular disease, only 18% of them had mentioned one year later that the article had been retracted on June 4th 2020 because of the impossibility of accessing the source data (an example of virtuous retraction).
The change in function of prepublication sites
The urgency for publishing results of studies that are of interest for public health worldwide, and the overloading of the capacity of peer reviewers, led to a quite spectacular growth in pre-publication servers, on which scientific researchers can deposit their manuscripts before they are peer-reviewed.
The first such server, arXiv, emerged among the community of theoretical mathematicians and theoretical physicists in the 1990s. Its vocation was to prompt critical comment in order to improve a paper before it was submitted to a journal. The first prepublication server dedicated to life sciences, bioRxiv, was launched in 2013, followed in June 2019 by another, medRxiv, for medical research. The latter, which at first was largely just ticking over, saw an extraordinary burst of activity when the pandemic struck, its new contents expanding from a few dozen to several hundred papers posted on it every week.
Above all, the function of the prepublication servers for topics of biology and medicine changed with the pandemic. Their usage became less that of prompting critical discussion (which nobody had time to write up given the daily avalanche of new publications), and more that for placing supposedly new knowledge in circulation, notably for the attention of the general public, the media and so-called deciders.
This was knowledge that was supposedly new because nothing certified the reliability of the reported data, which had not been scrutinised. Furthermore, no established procedure was in place to allow for the retraction of these papers in cases of errors or fraud. One of the rare known retractions from a prepublication server was that of an article by a Mexican team which pretended to demonstrate the effectiveness of anti-parasite drug ivermectin in treating Covid, and which had been posted on SocArXiv – a server for sociology topics.
Predatory journals
Another phenomenon, which is also associated with the Covid pandemic, has further destabilised the system of scientific publications, and this is what are called predatory journals.
Up until the 2000s, scientific journals were only available through subscription; one had to pay to read them. But along with the internet boom came the arrival of free-access online journals, which demanded that authors pay to publish their papers. The expansion of these online journals was encouraged by most of the institutions that funded research, in the name of the public’s right to access the results of the research they financed through their taxes.
But some publishers, who were as clever as they were dishonest, then created what are known as predatory journals (just as those behind them are known as predatory publishers). These are journals that resemble serious publications but which employ no critical review of the contents of submitted papers and which serve only to pocket money from researchers seeking to be published. Among these is Covid Journal and Infotext Journal of Infectious Disease and Therapy – whose acronym, IJIDT, was identical to the respected International Journal of Infectious Diseases and Therapy – both meeting all the criteria of a predatory journal as defined by international consensus.
The strategy of such predatory journals was an attempt to gain respectability. Some, in a more or less honest manner, succeeded in becoming referenced in large international bibliographic data bases, lending them the appearance of serious publications.
An edifying report by the InterAcademy Partnership (IAP), a worldwide network of academies of science, medicine and engineering, entitled Combatting Predatory Academic Journals and Conferences and published in March this year, warned: “The distinction between predatory and reputable outlets is growing less apparent (largely as the former make inroads into the latter) and presents a huge challenge for efforts to curb them”. The report underlined that establishing a blacklist, or “watchlist”, of predatory journals would now be futile, and that it would be better instead to list scientific journals in a spectrum ranging from the totally dishonest to those that are of quality, and all those that come between.
A necessary reinvention
The IAP report notes that “there are questionable and unethical practices by some established reputable outlets, such as establishing a second tier of journals that publish rejected papers on payment, which can be harder to both identify and challenge”. For when all is said and done, the globalised industry of scientific publications has seen its practices exposed in broad daylight by the pandemic.
A very lucrative business, paying nothing for its raw material (the scientific papers whose authors are not remunerated), nor for its workforce (the reviewers who work for free), and is organised by a handful of large, international publishing groups, this industry has long been the target of criticism from scientists, many of who regard both the subscription fees and the payment demanded of authors by the freely accessible online journals as exorbitant.
The industry must now reinvent itself in order to manage the turning point that has come about, and which, it is generally agreed, will be difficult to go back upon given the growing number of journals, the increasing number of research bodies which accept citations on prepublication sites, and the posting online on dedicated servers of papers that have not been reviewed.
While those servers were initially run by academic institutions, the publishing industry soon saw the interest in launching their own. MedRxiv, which published a large number of the medical articles that appeared on the subject of Covid-19, is jointly managed by Cold Spring Harbor Laboratory, a private US non-profit institution, Yale University and the company that publishes the prestigious British Medical Journal.
Meanwhile, several publishers, including Springer Nature (the third-largest in the sector), have created their own prepublication servers, which they dip into to find papers they can publish on the pages of their journals.
One thing is certain; the conflict that exists between the appetite for profit of a money-making scientific publication industry and the necessity of circulating the results of scientific research among both the profession and the public, will not be resolved any time soon.
-------------------------
- The original French version of this article can be found here. It is one of a series of articles on the theme of 'Five lessons to be learned from the pandemic', published in French on Mediapart here.
English version by Graham Tearse