Masks and COVID: The Mystery of the Missing RCTs

Randomized controlled trials (RCTs) have historically been regarded as the gold standard for evaluating medical interventions. Evidence-based medicine (EBM) developed during the 1970s as a reaction to the dominance of personal opinion and individual experience as the basis of medical education and practice. This would now be placed on a firm scientific footing. While the EBM hierarchy of evidence does not always translate well into social and organizational research on medical practice and health services, it has been the foundation of dramatic improvements in the effectiveness of medical care over the last 50 years. The lack of RCTs on non-pharmaceutical interventions (NPIs) during the COVID pandemic has been particularly lamented. We have been left with arguments over observational studies, with the protagonists accusing each other of cherry-picking or ignoring confounding variables, or laboratory studies from physics and engineering, whose applicability beyond the artificial conditions of the experiments is uncertain.

The two RCTs that have been published, from Denmark and Bangladesh, both report findings that suggests mask use in community settings has nil to minimal impact on infection rates. As critics have pointed out, neither is perfect – but this is not uncommon with RCTs, which are difficult to manage even in highly controlled settings like hospitals. EBM deals with this problem by encouraging further trials and aggregating the results in a meta-analysis that can compensate for the limitations of each of the studies that contribute to the outcomes. The two RCTs conducted so far are consistent with previous trials on the possible contribution of masks to reducing transmission of other respiratory viruses. A critic can, of course, respond simply by claiming that COVID is a novel virus to which no previous work can be applied…

This makes it all the more notable that the findings of two other trials of masks in the time of COVID have not been published – the dogs that have not barked. A feature of contemporary EBM is the development of registries for trials. These are intended to eliminate duplication and wasted resources. They normally also specify the statistical analyses that will be carried out, with the intention of avoiding ‘data-dredging’, running calculations until something significant falls out and then adjusting the research question to fit. Clinical Trials.gov has entries for both trials.

A trial in Guinea-Bissau was comparing the effectiveness of a standardized, locally produced, cloth mask in reducing community transmission compared with controls who were simply given information sheets about other measures thought to reduce this risk. The outcome measure would be self-reports of a COVID-like infection. This was intended to be a large trial, with the recruitment of 40,000 participants, which would have made it comparable to the Bangladesh study. It was registered on 15 July 2020 and expected to complete on 30 October 2021. The Canadian trial was much smaller and modeled on some of the pre-2020 studies of hospital transmission. Nurses would be randomly allocated to wear cloth or N95 masks while providing routine, non-aerosol generating, care to febrile patients. The outcome measure would be the number of nurses contracting COVID, confirmed by PCR testing, in each arm of the trial. This was registered on 5 March 2020 and expected to complete on 1 April 2021.

Together these trials would have provided significant evidence. The Guinea-Bissau trial would have helped to resolve some of the statistical uncertainty around the analysis of the Bangladesh trial, where different choices by different analysts found either a small (around 10 percent) benefit or an uncertain one lost in the noise around zero. The Canadian trial would have helped determine whether the results of the pre-2020 trials with other respiratory infections could simply be carried through, again supporting the case that there were nil to minimal benefits from masks. It would also have contributed to evaluating the claims that no benefits were being seen from community masks because of the types recommended – cloth or surgical rather than N95 (FP2 in the UK).

We are, however, now 12-18 months beyond the expected completion date of both studies and nothing has appeared in the public domain. The dogs have not barked. There is nothing in the registry entries to indicate a reason for this. Indeed the Guinea-Bissau trial posted an update on its planned analyses on 21 September 2021, which implies that the data were to hand and analysis expected to proceed imminently. What is going on here?

The answer to that question requires a consulting detective rather than a consulting sociologist. There are, though, rumors circulating widely within the academic public health community that both trials came up with the ‘wrong’ answer, that masks made little or no difference to transmission in either context. The lack of publications is alleged to be due to threats made against the lead investigators, of blocking future funding, promotions and other forms of professional advancement. In the nature of things, such claims are hard to substantiate. The investigators are hardly likely to go public with the threats and those threatening them are as well-served by silence and gossip that may discourage others.

We can, however, ask about the credibility of the rumors. Science and technology studies, or STS, has long pointed to the gap between the self-representation of the scientific community as a disinterested body of men and women, pursuing knowledge in a meritocratic and value-neutral fashion, and the actual practice of competition for recognition, reward and status through patronage and control of what counts as legitimate knowledge. The COVID pandemic has created an arena for paradigm wars to become visible in ways that are normally concealed from wider publics. The internal conflicts revealed by this are shocking only to those who had taken the self-representation at face value. Science has been insulated from the growing skepticism about professional claims to expertise and higher levels of civic morality. The pandemic has opened the way to the sort of critical scrutiny that has long been applied to medicine or law.

Unless someone chooses to go public, it is impossible to say that the rumors about threats to the investigators are true or false. However, we can say that they are consistent with what has long been known, and widely documented within STS, about the behavior of the scientific community. It must, in the future, expect a degree of accountability for this that reaches beyond a small cadre of STS scholars.