A study on the use of masks that many have not understood

When the Cochrane Association publishes a meta-analysis, or when it updates the results of a meta-analysis in the light of new studies, the result obtained is usually of the utmost importance, considering the highly standardized and transparent method by which the evaluations are conducted. In principle, these meta-analyses constitute the best that can be done to evaluate, as a whole, the quality and quantity of evidence in favor of a certain medical intervention, and for this reason, since they have existed, they have strongly influenced health policy. Now, on 30 January an update of a particular meta-analysis dedicated to physical means to prevent or reduce the spread of respiratory viruses was published.

This update has caused a flood of statements not only from Luddites and various opponents of scientific thinking, but even from prominent members of the academic community, the gist of which could be summarized as follows: face masks, including the N95/P2 grade one, are of no use to combat respiratory viruses, and particularly are not and have not been used to combat the spread of SARS-CoV-2. Therefore – this is the thought of those who see the mask as a muzzle – they were imposed without real scientific evidence, on the basis of a superstitious thought, to be benevolent, or a deliberate will of oppression, for the most extremist among these thinkers. These thinkers, even when they exploded in their statements, hadn’t even stopped to reflect deeply on the data, methodology and level of solidity of the conclusions of the work in question; despite the same authors declaring that “high risk of bias across trials, variability in outcome measurements, and relatively low adherence to interventions across trials hinder drawing firm conclusions”, for the above thinkers the question of principle was to prove one’s own right and everyone else’s wrong, and they saw nothing but an opportunity for revenge where instead it would have been necessary to stop and think carefully.

In fact, the great part of the scientific community, including myself, had already warned not only of all the caveats posed by the authors themselves, but also and above all of the numerous underlying problems in the method and in the merits of the analysis; problems so relevant that for example Richard EllisonIIIprofessor of Medicine, Microbiology and Physiological Systems in the Division of Infectious Diseases and Immunology of the University of Massachusetts Medical School, as well as past editor in chief of the New England Journal of Medicine, had gone so far as to declare that “the Cochrane review could be interpreted at the maximum in the sense that it demonstrates that masks do not provide protection when they are not worn”.

To definitively clarify the matter he thought about it Karla Soares-Weiser, editor in chief of the Cochrane Library, who on behalf of the association released the following statement: “Many commentators have stated that a recently updated Cochrane Review shows that ‘masks don’t work’, which is an inaccurate and misleading interpretation. It would be fair to say that the review looked at whether interventions to promote mask wearing help slow the spread of respiratory viruses, and that the results were inconclusive. Given the limitations of the primary evidence, the review is unable to address whether mask wearing by itself reduces people’s risk of contracting or spreading respiratory viruses. The review authors are clear on the abstract’s limitations: “High risk of bias across trials, variation in outcome measures, and relatively low adherence to interventions across trials hinder drawing firm conclusions“. […] The original plain-language summary for this review stated that “we are uncertain whether wearing N95/P2 masks or respirators helps slow the spread of respiratory viruses based on the studies we evaluated.” This wording was susceptible to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are collaborating with the review authors to update the summary and plain-language abstract to clarify that the review examined whether interventions to promote mask wearing help slow the spread of respiratory viruses.”

In essence, they say: look, the review has assessed at most, without obtaining definitive evidence, whether the policies to induce the adoption of masks have worked in blocking the virus, without however being able to say anything about the functioning of the masks, given that they do not it is possible from the data to estimate adherence to these policies. Furthermore, not even much can be said about the functioning of the interventions to promote the masks, because the studies collected are so different in terms of methods and possible confounders, that the final result of the meta-analysis is not reliable.

Now, to tell the truth, if a mask works or doesn’t work, it is evaluated with specific laboratory tests, certainly not with epidemiological data: in the same way that to know if an umbrella protects from the rain, a population study is not carried out, so to know if the physical characteristics of a device are such as to block a virus, we do not resort to a meta-analysis of epidemiological studies. Those who demand meta-analyses of randomized trials for things like the effectiveness of a physical barrier should realize that this approach doesn’t get very far: it would be like requesting a meta-analysis to find out if a radiator heats up. Meta-analyses – and even earlier randomized trials – should be usefully employed for those cases in which the system under investigation has such a high variability, for intrinsic reasons, that the effectiveness of a procedure cannot be examined except statistically, as is the case as regards the action of a drug on a living system: that is, statistics are indispensable when it is not possible to resort to experimental replication in a laboratory to answer a particular question.

In the present case, the meta-analysis can help us to say, for example, that although masks are an excellent barrier to infection, they are little or very effective due to the population’s adherence to correct use, or in the presence of limits deriving from the reality of human behavior; not whether or not a mask works in filtering a virus. Otherwise, next time he’ll ask for a randomized controlled trial to establish the usefulness of a parachute after jumping from an airplane.