Is scientific peer review a scam?
A response to Matt Ridley's recent article
The title of this post is essentially cribbed straight from a recent missive written by science journalist and occasional firebrand Matt Ridley in the Telegraph and picked up by other news sites. In his article Ridley recounts how he and Oxford medical researcher Anton van der Merwe had been requested, toward the beginning of the COVID-19 pandemic, to write an article summarizing evidence indicating that a lab accident at was likely the ultimate source of the pandemic. That article, upon submission, had been summarily rejected by the “prestigious scientific journal” that had solicited it. Further, the article had later also been snubbed by another such journal to which it was subsequently, after some further revision, submitted. It was this state of affairs that lead Ridley to make his bold proclamation about the perpetration of a scam.
Given, in the intervening years since the pandemic’s appearance, the seemingly growing acceptance of data that more or less exonerates these authors’ early findings, readers of Ridely’s telegraph article are invited to share his umbrage at the earlier rejection of his findings in this area and to conclude with him that the system that eventuated in that rejection, namely the peer review process, is fundamentally flawed. Or, to put it in less formal, more sensational terms, that scientific peer review is a scam.
While sweeping statements of the sort Ridley makes in the title of his Telegraph article are often either cheap attempts at sensationalization for personal aggrandizement purposes or are simply patently prima facie ridiculous, this one is certainly worthy of further consideration. The author, who is well-credentialed and has a fairly lengthy history in science journalism is clearly no crank or cheap-shot attention-monger. So we should definitely pay some heed to his assertions.
But beyond his highlighting of some of the ways in which, in this particular case, important data was being deliberately obscured from public view—important as it is that such factors be recounted—there lie some interesting dynamics that I consider at least equally worthy of consideration but that seem seldom to receive what I hold to be adequate notice. Reflect then, on the following.
For example, it might be asked from whence arises this sense of umbrage about the deliberate obfuscation of data from public view? Why do we have the expectation that such data, damaging though it might prove to some, should be openly and widely propagated? Admittedly, the reasons could seem so obvious as to not even require adumbration. But I hold that adumbrating some of them is what is perhaps most sorely needed here. And further, that engaging in that sort of rehashing might prove quite complementary to the sorts of remedies proposed by Mr. Ridley.
I believe one major reason that we have a sense of scandal at the obfuscation of data and views about which Mr. Ridley and his colleague were attempting to publish is due to our assumptions about the phenomenon we and Mr. Ridley label “science.” In the view of so many of us, science is some sort of pure intellectual endeavor whose only aim is to arrive at some sort of ultimate truth about the nature of our world by means of rigorous research and testing. This impression is sometimes further formalized by characterizing the process by which the requisite truth is arrived at as one that involves attempting to subject theories to falsification, as the great Sir Karl Popper maintained. In other words, theories presumably undergo a continuing process of subjugation to testing that either proves or undercuts their veracity. Peer review, as a matter of fact, can be considered an element of the above-mentioned falsification procedure.
In the case of Ridley and van der Merwe’s original article, what should have happened, according to Popper’s model, is the free and open publication of the evidence they were presenting, to be followed by attempts by those critical of their interpretation of the data to undercut their case. In other words, those critics would be attempting to falsify Ridley’s and van der Merwe’s claims indicating a lab accident and presenting evidence proving that it was more likely either that no accident had occurred or that the virus had a natural origin. But instead, as we know by now, the natural origins theory was the only one actually allowed to be aired publicly. Given this view of the enterprise of science, when we see rigorous investigations such as those about which Mr. Ridley and his colleague were attempting to publish being concertedly kept from public view, we are scandalized.
The problem I see here, though, is that the underlying assumption about the nature of science operative here is a faulty one, one that does not adequately account for the reality of the situation. It is inadequate, in my view, for reasons perhaps first voiced by the great philosopher of science and Popper critic Paul Feyerabend: this accounting of the scientific enterprise does not accord with the historical record of scientific discovery and progress. What, according to the Popperian view would be expected to be an ongoing and steady process of testing and displacement of inadequate theories by more legitimate and better-verified ones, is actually, when the historical record is consulted, a quite chaotic and sporadic process that proceeds in fits and starts and that is governed as much by human intuition and happenstance as by steady, methodical progress. In light of such realizations, the notion that falsification is the engine by which the enterprise we call science steadily progresses seems, instead of an accounting that accords with historical realities, to be some sort of idealized depiction of the situation rather than an accurate description of the realities “on the ground,” as it were.
The research of social scientist Thomas Gieryn provides helpful insights into the actual situation with which we are faced in the instance about which Ridley is writing. Gieryn approached the question of what could be called the bounds of science, a matter typically labeled by philosophers of science the “demarcation question,” by pointing out that demarcation is not simply a matter of mapping out metaphysical parameters. He helpfully pointed out that, in actual practice demarcation is accomplished day in and day out by the exclusion of certain activities and ways of thinking on an institutional level. As he has pointed out, “(e)ven as sociologists and philosophers argue over the uniqueness of science among intellectual activities, demarcation is routinely accomplished in practical, everyday settings: education administrators set up curricula that include chemistry but exclude alchemy; the National Science Foundation adopts standards to assure that some physicists but no psychics get funded; journal editors reject some manuscripts as unscientific.” (Boundary Work, 1983, p. 781) In other words, what we call “science” actually has institutional manifestations with conceptual criteria, and it is these criteria that, in actual practice, that ultimately determine which types of research and findings merit classification under the rubric “science.”
As that thinking applies to the current circumstance involving the pandemic, we might further add that the actual human institutions that represent the phenomenon we think of as science—in this case medical science—are far from being the vehicles of pure intellectual inquiry meant to eventuate in steady progress by the known mechanisms our ideals dictate they should be. Instead, they appear to be institutions, being as they are staffed by human beings and existing within a social matrix indifferent to the idealistic depictions of certain analysts, subject to all the flaws and foibles—perhaps writ even larger—of the human beings that staff all such institutions. In short, there is nothing terribly unique about institutions representing science enterprises as over against the background of all other human institutions—whether or not they are representative of endeavors we might consider properly “scientific.”
Owing, then, to standard institutional dynamics where protection of reputations is of ultimate concern, research contravening that aim may be suppressed. Or, perhaps owing to pressure on a given institution from other powerful social institutions, i.e., such as those in the political sphere, information of an otherwise ostensibly scientific interest may be obscured or kept from public view. Something like this must surely have been behind the apparent censure of Mr. Ridley’s work. Taking adequate account of such factors should be chief among the considerations of those seeking some redress for such apparent perversions of intellectual rectitude.
That, alongside Mr. Ridley’s proposed remedy, namely that researchers publish their findings without hindrance far and wide, openly inviting criticism as the test of the verity of their findings, does seem like it could aid in addressing the flaws highlighted in the case he has recounted. But given the ubiquity of such social and institutional dynamics throughout the course of human history, perhaps the best we can hope for is sporadic and fleeting alleviations of the problem.
With realizations such as this in mind then, alongside Mr. Ridley’s remedy of widespread and unfettered publication and dissemination of research, we should perhaps also acknowledge more frequently the fact that what we call “science” has a very real, very mundane, and, in actual fact, fairly obvious manifestation as a human institution—and, as the current case reveals, one with all the imperfections that implies. And it exists within a matrix of other such institutions and those institutions, which may take little to no cognizance of or have any reputational stake in the ideals of some of its representatives, can also exercise their influence on its operations. Consequently, we may be less apt to find ourselves so scandalized when, in the name of science, some perverted or even malicious scheme such as having a pandemic visited upon us, while the true nature of its origins are actively and systematically covered up by publications we had come to think of as uniquely capable of arriving at and conveying ultimate truth.
In summarizing, we go back to the question posed in the title—is peer review a scam?—and to a potential answer. The answer depends on what is expected of peer review. If it is expected that it will unfailingly reveal truth or ruthlessly, without regard for entrenched interests, personalities or institutional loyalties, advance knowledge, instances like that recounted by Ridley do give the impression that something that could be called scamming is involved. On the other hand, if we take the view that the scientific enterprise is inseparable from its institutional manifestations with their vested interests, revered personalities or positions, the web of institutional entities within which they exist in society at large and which they either influence or are influenced by, and finally, per Gieryn, the boundary-establishing practices that are one of their hallmarks, instances like the one encountered by Ridley should be expected. His work violated a boundary and he found himself, at least temporarily, outside the parameters of the scientific enterprise and shading into quackery due to his propagation of an unacceptable theory.
Whether that was due to the unsoundness of his work (seems unlikely), the influences of powerful forces outside the confines of the technical sphere of his research, or perhaps his slighting of some venerable person or individual (Anthony Fauci, perhaps?) is a separate question. As the case may be, this writer will be watching with interest the unfolding of this matter as the effects of the pandemic recede into the historical background.