Masks Help, but We May Never Know How Much

On Wednesday, one of the world’s most prestigious medical journals published what many took to be a disheartening result: According to some headlines, a 6,000-person randomized controlled trial in Denmark had found that wearing a mask does not offer any clear protection from being infected with the SARS-CoV-2 coronavirus. This follows months of heated debate, and the assertion in October from White House Coronavirus Task Force member Scott Atlas that, in light of “considerable uncertainty” in the research literature, masks cannot be said to work.

Atlas may indeed be right, albeit only in a technical sense; there is no study which provides direct, strong, and reliable evidence that lets us estimate the precise extent to which masks and mask mandates reduce the spread of the pandemic. But that lack has little to do with how well masks work, and everything to do with the profound difficulty of doing this kind of research. The newly published trial, which was named DANMASK-19, is no exception. Its limitations are severe enough to render it almost completely uninformative.

The impact of masks is not just unknown, it’s unknowable.

To understand why, consider how hard it is to estimate the impact of the mask mandates from last spring. Covid was ramping up around the world, and everything from policies to infectious disease dynamics to social behaviors was changing all around us and all at once. Each of these factors affected the spread of the virus in complicated and interdependent ways, with unknown lags between when the changes occurred and when their effects might be expected to show up in the data. Add in the fact that testing availability and data access have also shifted over time, and it’s clear that isolating the impact of just one intervention—a mandatory masking rule, let’s say—is a heroic, maybe impossible feat.

Still, there are now dozens of studies that have attempted to do just this. One high-profile example, published last June in the Proceedings of the National Academy of Sciences, claimed that early mask mandates like the one in New York City had reduced the number of local infections by many tens of thousands. (The paper was widely covered at the time, and has been cited many times since.) Unfortunately, the findings were based on flawed methods and study design; when the Johns Hopkins University Novel Coronavirus Research Compendium reviewed the study’s strengths and value added, it gave a one-word judgment: “none.” The issues were sufficiently alarming that my colleagues and I led an effort to have the journal editors and authors retract it. Several months later, a representative from the journal’s editorial board told us that they’d “found errors” in the paper but would not be retracting or correcting it, and declined to elaborate further.

Read all of our coronavirus coverage here.

The DANMASK-19 randomized controlled trial isn’t informative either, but for very different reasons. First of all, this wasn’t a trial about mask-wearing; it was a trial about messages to wear masks. Participants were first encouraged to practice social distancing, and then half were randomly selected to receive repeated messages to wear masks whenever they left the house for the next month, as well as a box of 50 disposable face masks. Any protective effect those masks may have had was dampened by the fact that many of the participants didn’t actually use them: In the end, less than half the people in the intervention group reported having worn the masks as recommended. Further, an unknown number of people in the control group may have made their own decision to wear masks anyway, though this was probably limited. The small number of SARS-CoV-2 infections that occurred during the trial—just 95 cases in all—makes it too small to detect anything other than an absurdly large effect. But the most glaring flaw of this research is that it did not (and for practical reasons, could not) measure what is likely to be the most important benefit of masks: reduced transmission to others.

None of this was unexpected. My colleagues and I registered our concerns about the trial design on Sept. 8, and now our predictions have been confirmed. The design was destined for insignificance, diluted effects, and misinterpretation regardless of how effective mask wearing really is. Although the publication itself clearly indicates its limitations, that hasn’t stopped some from saying that this trial shows that masks are ineffective. In fact, it tells us little to nothing about this question. Randomized controlled trials are often considered the gold standard for medical evidence, but some trials offer little more than glitter.

For all its flaws, the DANMASK-19 trial may very well have been the best study on mask-wearing that could have been done, but the best we can do is often not very informative or useful. If the study had been larger, it may have given us a good answer to an important, but less useful, question: What is the personal protective effect of messages that encourage mask use? However, this study gives us no information one way or another on the question that matters most: How much does mask-wearing reduce the spread of SARS-CoV-2? The sort of study that could have answered that—a population-scale trial of government mask mandates, for example—would have been almost impossible to run, requiring dozens of regional decisionmakers to agree on behalf of their constituents to comply with public health policies determined by a roll of the dice, all while in the midst of a pandemic. There are new opportunities and studies in the works, but they remain to be seen. Generating useful evidence on this question is very tricky, however you approach it.

This is not just about masks. An absence of useful evidence in favor of an intervention means little by itself; we need to know why the evidence does not exist. The reason is that strong evidence is hard to generate, but the default interpretation—the way people tend to understand the absence—is that there must be no effect. Even official sources get this wrong, as we saw last March, when some public health authorities suggested that masks provide no benefits. The same mistake also goes the other way: In the hydroxychloroquine debacle, for example, enormous numbers of people were treated due to lack of evidence that a treatment didn’t work.

On masks and mask mandates, there may never be a direct, reliable, and actionable study of their population-level benefits. What we have instead is a collection of evidentiary bits and pieces; and when these are taken all together they say that masks are certainly somewhat effective—maybe even very effective—at slowing the spread of SARS-CoV-2. That’s what we know; no more, no less. If you want to extrapolate from these bits and pieces to potential policies, such as a federal mask mandate, you’ll need to take far more into account—the full complexities of the political, social, and economic context. Coping with the profound uncertainty about this crucial question remains our best, worst, and only option for the foreseeable future.

A fixation on trying to wring false or impossible certainty from uninformative evidence gets in the way of making good decisions. Evidence-based policy should start with a comprehensive understanding of what we know, what we don’t know, and what is unknowable. When we don’t have evidence for a particular decision, it’s usually the case that we also don’t have evidence against it, either. We can fill some of the gaps with indirect evidence and experience, such as what we have learned from previous epidemics or from theory about basic mechanisms. But uncertainty will remain, and sometimes the best we can do is hedge our bets.

In the research and public health expert community, we need to prioritize communicating the uncertainty and limitations of our work above our conclusions. We must also accept that weak research, even if it’s the best we can do, does not resolve uncertainty; it just obscures it. Sometimes, nothing is better than something.

Uncertainty is the default position here, whether we like it or not. It’s long past time for us to embrace the unknowable.

WIRED Opinion publishes articles by outside contributors representing a wide range of viewpoints. Read more opinions here, and see our submission guidelines here. Submit an op-ed at [email protected].

More From WIRED on Covid-19

• 📩 Want the latest on tech, science, and more? Sign up for our newsletters!
• Can you get Covid-19 on an Airplane? Yeah, probably
• The vulnerable can wait. Vaccinate the super-spreaders first
• Winter is coming. Could humidifiers help?
• If you’re thinking about holiday gatherings, look at this map
• Taking that Covid test doesn’t mean you’re safe
• Read all of our coronavirus coverage here