1) We have to consider Jason Abaluck was predisposed to finding a confirmation of his bias. I'm sure he's a smart guy, but he was not neutral, and that would potentially impact how he reads the tea leaves. He authored a widely shared plea to mask up immediately when Covid hysteria appeared in April of 2020 calling upon the precautionary principle. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3567438
2) The ARR is so small that you have to wonder how? If masks did nothing there would still be a 50/50 shot a random study happened to have more infections in the control. I would have expected a bigger ARR swing just by randomness. The fact that the ARR is below even the placebo level suggests a chance of negative efficacy - especially if you control for the increased distancing.
1) Thanks; I wasn't aware. I admit I don't have a great understanding of how exactly masks came to be widely adopted in March and April 2020; I'm more a scientist than a historian. But the history is a really important part of the picture, so I appreciate you sharing this.
I suspect almost all of the bias in this study is a result of the methodology rather than the final analysis of the data. After all, the analysis was mostly preregistered. But your point that even the most intelligent researcher is susceptible to confirmation bias is well taken.
2) That's a good point. I didn't get into this in the review itself, because of how highly technical it is, but the authors did compute p = 0.03 for the primary endpoint. That is, if the intervention really had no effect whatsoever, there would only be a 3% chance of observing an effect on symptomatic seropositivity that's this big or bigger.
This p-value is sensitive to how the situation is modeled. Using a different model, Dr. Chao Wang computed an abysmal p = 0.856 (see the eLetter at the bottom of the page here: https://www.science.org/doi/10.1126/science.abi9069), which would imply that the observed effect could easily be a result of random chance.
I do have a bit more to say about this issue, if I ever get around to actually writing part 1(b).
3) Oh, absolutely. But I suspect that if the authors had decided to use "fake masks" in the control group, and then found a negative result, we'd just be seeing headlines like "Yale researchers show that even a 'fake mask' is as effective as a surgical mask at stopping the coronavirus".
4) I don't understand. Do you mean that I should look into the survey responses of the symptomatic but seronegative people?
5) Thanks. It took me a while to see this, but the method the authors used to determine whether or not someone had COVID is incredibly flawed. I just had to include that example.
6) No, they didn't. Sorry, I see now how that could be misleading.
"4) I don't understand. Do you mean that I should look into the survey responses of the symptomatic but seronegative people?"
No, I didn't mean you should track down the other 7/9th, I was merely reflecting on the absurdity of how without our newfound diagnostic tests we weren't able to discern between the common cold and the super deadly Covid-19.
It's an understated reason for the hysteria, in human history we have never searched so hard for a virus. We do more PCR tests per day then we previously did each year for flu.
We replaced clinical diagnosis with laboratory diagnosis and many of the titans of medicine have warned of this [1] and continue to warn of this [2].
Overdiagnosis is a serious problem in medicine which leading clinicians have been working on reducing for years, it is just crazy to me that we threw all of this out. Just over a decade ago for example we learned the danger of going crazy with testing in South Korea when we created a Thyroid Cancer epidemic out of thin air. [4]
[1] Fantastic article by David Sackett, read this. Along with Archie Cochrane he was the founder of Evidence Based Medicine. This piece is 20 years old but more relevant now than ever. https://www.cmaj.ca/content/cmaj/167/4/363.full.pdf
[4] South Korea Thyroid Epidemic created because we started giving free PSA screening which detected trace amounts of cancerous cells that likely would never had been an issue, subjecting thousands to unnecessary chemo, radiation, etc.
Excellent review. Quick notes/questions:
1) We have to consider Jason Abaluck was predisposed to finding a confirmation of his bias. I'm sure he's a smart guy, but he was not neutral, and that would potentially impact how he reads the tea leaves. He authored a widely shared plea to mask up immediately when Covid hysteria appeared in April of 2020 calling upon the precautionary principle. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3567438
2) The ARR is so small that you have to wonder how? If masks did nothing there would still be a 50/50 shot a random study happened to have more infections in the control. I would have expected a bigger ARR swing just by randomness. The fact that the ARR is below even the placebo level suggests a chance of negative efficacy - especially if you control for the increased distancing.
3) We could have easily blinded it. Give half the villagers these. Never know the difference. https://fakemaskworldwide.com/the-fake-surgical-mask-2-layer/
4) 2/9ths of the sick people weren't sick with Covid.... why not chase down the other 7/9th with such vigor?
5) ML: “Did H-4087427-168-1 actually have COVID?” - << Really impressive you dug that deep!
6) “How could they not do what we say? Is it because we didn’t do enough emotional blackmail?
- Did they really say this?
1) Thanks; I wasn't aware. I admit I don't have a great understanding of how exactly masks came to be widely adopted in March and April 2020; I'm more a scientist than a historian. But the history is a really important part of the picture, so I appreciate you sharing this.
I suspect almost all of the bias in this study is a result of the methodology rather than the final analysis of the data. After all, the analysis was mostly preregistered. But your point that even the most intelligent researcher is susceptible to confirmation bias is well taken.
2) That's a good point. I didn't get into this in the review itself, because of how highly technical it is, but the authors did compute p = 0.03 for the primary endpoint. That is, if the intervention really had no effect whatsoever, there would only be a 3% chance of observing an effect on symptomatic seropositivity that's this big or bigger.
This p-value is sensitive to how the situation is modeled. Using a different model, Dr. Chao Wang computed an abysmal p = 0.856 (see the eLetter at the bottom of the page here: https://www.science.org/doi/10.1126/science.abi9069), which would imply that the observed effect could easily be a result of random chance.
I do have a bit more to say about this issue, if I ever get around to actually writing part 1(b).
3) Oh, absolutely. But I suspect that if the authors had decided to use "fake masks" in the control group, and then found a negative result, we'd just be seeing headlines like "Yale researchers show that even a 'fake mask' is as effective as a surgical mask at stopping the coronavirus".
4) I don't understand. Do you mean that I should look into the survey responses of the symptomatic but seronegative people?
5) Thanks. It took me a while to see this, but the method the authors used to determine whether or not someone had COVID is incredibly flawed. I just had to include that example.
6) No, they didn't. Sorry, I see now how that could be misleading.
"4) I don't understand. Do you mean that I should look into the survey responses of the symptomatic but seronegative people?"
No, I didn't mean you should track down the other 7/9th, I was merely reflecting on the absurdity of how without our newfound diagnostic tests we weren't able to discern between the common cold and the super deadly Covid-19.
It's an understated reason for the hysteria, in human history we have never searched so hard for a virus. We do more PCR tests per day then we previously did each year for flu.
We replaced clinical diagnosis with laboratory diagnosis and many of the titans of medicine have warned of this [1] and continue to warn of this [2].
Overdiagnosis is a serious problem in medicine which leading clinicians have been working on reducing for years, it is just crazy to me that we threw all of this out. Just over a decade ago for example we learned the danger of going crazy with testing in South Korea when we created a Thyroid Cancer epidemic out of thin air. [4]
[1] Fantastic article by David Sackett, read this. Along with Archie Cochrane he was the founder of Evidence Based Medicine. This piece is 20 years old but more relevant now than ever. https://www.cmaj.ca/content/cmaj/167/4/363.full.pdf
[2] Carl Heneghan is a leader in evidence based medicine prior to being ostracized from public health by the Covid hysterics. https://www.spectator.co.uk/article/covid-19-and-the-end-of-clinical-medicine-as-we-know-it
[3] Gerd Gigerenzer describing the problems with innumeracy and lack of statistical background with physicians https://home.cs.colorado.edu/~martin/Csci6402/Papers/gg03.pdf
[4] South Korea Thyroid Epidemic created because we started giving free PSA screening which detected trace amounts of cancerous cells that likely would never had been an issue, subjecting thousands to unnecessary chemo, radiation, etc.
https://www.bmj.com/content/355/bmj.i5745