In February a study was published out of France that the very old malaria drug, hydroxychloroquine (HCQ), could inhibit the infection of cell cultures in the laboratory with the SARS-CoV-2 virus. I did an analysis of the data within 24 hours and posted a note on social media that the concentration needed for it to work in the test tube was not reached by the drug when given in normal doses. I was skeptical of it working.
But the drug developed a life of its own. It eventually received an Emergency Use Authorization from the FDA and it is likely many people are taking HCQ either to prevent the disease or to treat it if they have COVID-19. It became, the first drug in history to become a political pawn. But it is still a drug and I am still interested in the risk-benefit of treatments for important diseases, including COVID-19. So I continue to examine the benefit-risk balance of HCQ.
As a reminder, the risks of HCQ are well known as it is a very old drug. The most important risk is a dangerous, sudden heart pattern that can kill you if you are not defibrillated with the heart paddles in short order. While there is an algorithm for knowing how likely you are to have these arrhythmias (my upcoming book has a detailed discussion of how to calculate your risk) for the most part, anyone over about 65 years with a serious COVID-19 infection will be at moderate to high risk.
Today, four Harvard Medical School medical scientists stepped in to try to settle the matter but their analysis is patently wrong and because I assume they are reasonably smart I am wondering how they came to their conclusion except by the process of starting with the end in mind. That is a great way to plan a trip but a lousy way to study unknown science.
In an article published in the British Journal, The Lancet1, they pooled data from over 96,000 hospitalized patients with COVID-19 located in 671 hospitals in six continents. If I remember my geography lessons correctly this means hospitals from everywhere in the world except Antarctica! They compared the rate of heart arrhythmias and deaths for patients that were taking HCQ or its cousin drug, chloroquine (the Treatment Group) to those who were not (the Control Group).
Their results are shown here in this Text-Table.
The safety findings are very clear and damning for HCQ. The treated group had between two- and five-times more heart rhythm problems than the control group. The range of numbers is because the scientists tried to help HCQ ‘win’ by doing subgroup analyses of HCQ alone or with other drugs, etc. But it didn’t work. Any combination that included HCQ or chloroquine had more arrhythmias. The false result rate for the study was set so that the chance that this result is not correct is one out of twenty times.
As expected, when you have so many more heart arrhythmias you also have increased deaths. Can I make the picture clear? The patient is in the hospital with COVID-19, is very sick, probably on a ventilator, is taking HCQ, and suddenly their heart monitor goes off telling the ‘crash cart staff’ they need to rush to the bedside of the patient because they have about four minutes to get the heart problem corrected or the patient dies. But everyone is wearing PPE and nervous and doing a rescue takes even a little longer than it would under normal circumstances. I think you see where this ends up.
So does that settle it that HCQ doesn’t work? No! The last column above in my Table is why.
Buried in one of the tables in the paper, without any real comment by the authors, is this remarkable data point in the column titled mechanical ventilator: the patients who got HCQ or chloroquine were much more likely to be on ventilators than the control group. With this single observation, their whole paper suddenly heats to 454 Fahrenheit and burns up.
Why? Bad science. Two input variables with one output variable means you can’t say which input variable caused the output.
If you look at an outcome like arrhythmias or death and you have two input variables, one variable is being on a ventilator and the other is taking HCQ, you have no way to know which caused the outcomes. Said simply, the results also could be concluded that ventilation leads to more deaths. But writing a paper that says that patients with COVID-19 who are on mechanical ventilation die more often is hardly worthy of publication in The Lancet. But ignore that confounder and come out and say HCQ kills people and suddenly you get top billing. Strange indeed!
I am actually kind of embarrassed for them because this is a college level experimental design error although I was pleased to see that none of them were from either The University of Michigan or The Massachusetts General Hospital.
Could they have salvaged the study with the data they have? Absolutely.
All they had to do was perform what is called a Case Matched Control Study, where they use preset features to match the controls to the treated group and then redo the analysis. So here, instead of having a control group with 7.7% mechanical ventilation they make a control group with 23-25% mechanical ventilation, the rate in the HCQ treatment group, and then repeat their analysis.
I did a back of the envelope analysis myself and my take is that if you control for ventilation HCQ is either neutral or maybe a little helpful to patients. That might not get as many internet clicks but it would at least be descent science.
I still lean out on whether HCQ is beneficial for COVID-19 (if I were betting I would conclude no). But this study gets us no closer to the truth and mucks up things for true science.
