Why new hydroxichloroquines will come

Versão em português aqui

Hy­drox­y­chloro­quine and COVID-​19

It is not new that hy­drox­y­chloro­quine was once the promised drug to de­feat (or at least to curb) the emerg­ing COVID-​19 pan­demic. Also, every­one knows that the short but con­vo­luted hy­drox­y­chloro­quine (HCQ for short) story is not rosy.

HCQ rose to sci­en­tific and main­stream media at­ten­tion when Chi­nese re­searchers noted that none of 80 pa­tients tak­ing HCQ to treat lupus had be­come in­fected by the Sars-​Cov-​2 virus (which causes COVID-​19). This kind of ev­i­dence is very spec­u­la­tive and, while it might point to­wards a valid hy­poth­e­sis, should not be stretched too far. The sam­ple of 80 pa­tients is tiny and pa­tients with lupus might ad­here to so­cial dis­tanc­ing more as they are more sus­cep­ti­ble to in­fec­tious dis­eases due to their usual treat­ments. Still, stretched it was, es­pe­cially after a French study with many flaws was pub­lished, in which HCQ com­bined with the an­tibi­otic (which pre­sum­ably does noth­ing against viruses, only against bac­te­ria) azithromycin sup­pos­edly led to shock­ingly quicker re­cov­er­ies in hos­pi­tal­ized pa­tients (read here to know more about this study).

As one says: ex­tra­or­di­nary claims re­quire ex­tra­or­di­nary ev­i­dence. But the more-​than-​ordinary and highly ques­tion­able ev­i­dence for HCQ use against COVID-​19 quickly be­came pop­u­lar, with politi­cians and even some health au­thor­i­ties de­fend­ing its use. Since March, var­i­ous stud­ies with greater sci­en­tific rigor in­ves­ti­gated this use of HCQ (read here, here, here, here and here). Un­sur­pris­ingly, all these stud­ies (ran­dom­ized clin­i­cal tri­als) found no ev­i­dence that HCQ presents any ben­e­fit against COVID-​19.

Nonethe­less, some po­lit­i­cal lead­er­ships still de­fend HCQ with great pas­sion and its use is still widely pop­u­lar. In Brazil, not only does the pres­i­dent de­fend its wide­spread use, but health in­sur­ance com­pa­nies are also dis­trib­ut­ing “Covid-​19 kits”, that con­tain HCQ and other drugs, to in­fected pa­tients.

In the mid­dle of this non­sense, one might won­der: how did we get to this point? This is not an easy ques­tion, with mul­ti­ple fac­tors com­ing into play, such as le­git­i­mate hope when no ther­apy ex­ists. But from all the fac­tors, I’d like to focus on one: how we, as sci­en­tists, are fail­ing to ex­plain why well-​controlled stud­ies are nec­es­sary. I won’t even try to ex­plain what ran­dom­ized clin­i­cal tri­als are, there are great re­sources on this topic.

I think this fail­ure is partly due to a lack of in­tu­itive un­der­stand­ing of basic sta­tis­ti­cal and sci­en­tific no­tions, which will be briefly ex­plored here.

Prior prob­a­bil­i­ties and the coin toss mis­take

Let’s sup­pose there is a dis­ease spread­ing out with no avail­able ther­apy, with low death rates (1%). Now let’s also sup­pose that a cer­tain treat­ment, with very weak ev­i­dence in its favor, gains mo­men­tum and pop­u­lar­ity. Let’s call the treat­ment the ‘What If (WI) it works’ drug. Peo­ple start to take the WI drug and re­port that their re­cov­er­ies were extra quick and smooth. As this hap­pens thou­sands of times, it is tempt­ing, al­most ir­re­sistible, to con­clude that WI drug works, right? Not so much.

As­sum­ing that the drug does ab­solutely noth­ing, let’s build a sim­ple table. The columns in­di­cate whether a per­son took WI or not, while the rows in­di­cate whether the per­son died or not (with 1% mor­tal­ity).

In this ex­am­ple, out of 10000 peo­ple with the dis­ease, half took WI (to make it eas­ier to vi­su­al­ize), and 50 peo­ple died in both groups (tak­ing WI or not). When we see this table, it’s very easy to see the drug doesn’t ap­pear to be work­ing at all. How­ever, con­sider that 4950 peo­ple that took the drug are per­fectly well, and many if not most of them will at­tribute their sur­vival to the drug, while the 50 who died will not be heard.

Let’s sim­u­late a new sce­nario. This time, the drug made things worse. Again, look­ing at the table, it’s easy to see that WI lead to a 100% in­crease in mor­tal­ity! But there are still 4900 peo­ple (that is, 98% of those who took the drug) that sur­vived and may de­fend WI with all their pas­sion.

One last ex­am­ple, this time a more re­al­is­tic one. The drug has no ef­fect, but imag­ine get­ting to know about cases only through in­di­vid­ual re­ports and media cov­er­age. The num­bers get cloudy, and it’s easy to draw wrong con­clu­sions.

These ex­am­ples il­lus­trate that in a real-​world sce­nario, it’s hard to grasp the real di­men­sion of things at this level. Anec­do­tal ev­i­dence — that is, when some­one tells you he or she got bet­ter after tak­ing WI — should not be taken at face value. Even if this table showed a pos­i­tive ef­fect of WI, that would not be enough. Peo­ple who choose to take WI may be dif­fer­ent from those who don’t take it in var­i­ous as­pects that in­flu­ence their chance of sur­vival. These bi­ases can only be ad­dressed with ran­dom­ized clin­i­cal tri­als.

Dis­ease is not a coin toss

In­tu­itively, our sense of prob­a­bil­ity is close to a coin toss rea­son­ing. That is, we think: I got this dis­ease and may die or not (a coin toss). ‘I may very well try this What If drug every­one is talk­ing about, it may help me’ (50/50 chance). ‘If I sur­vive, the drug prob­a­bly helped. Bet­ter yet, if many peo­ple sur­vive after tak­ing WI, the drug must work!’

The mis­take should be ob­vi­ous by now. While al­most every­one knows that their chance of dying is not 50% (in this case, it’s 1%), it’s hard to in­tu­itively take these prob­a­bil­i­ties into ac­count, and we grav­i­tate to­wards the 50/50 sce­nario. Under the in­flu­ence of the coin toss bias, when thou­sands of peo­ple sur­vive after tak­ing WI, we tend to think that the drug must be work­ing ex­actly be­cause we ig­nore the known low mor­tal­ity. What is the prob­a­bil­ity that the drug ac­tu­ally works? If ev­i­dence is too weak, it might as well be the prob­a­bil­ity that any ran­dom drug would work, which is of course very close to zero.

These prob­a­bil­i­ties, 1% mor­tal­ity and near-​zero chance of a ran­dom drug to work are called prior (or base) prob­a­bil­i­ties. If ev­i­dence about a par­tic­u­lar case is weak (for ex­am­ple, ‘will I sur­vive this dis­ease?’), we should al­ways use the base prob­a­bil­ity as a start­ing point (1% chance of dying). From that prob­a­bil­ity, we can cau­tiously walk in ei­ther di­rec­tion with ev­i­dence about par­tic­u­lar cases (e.g., ‘I have a heart con­di­tion, which in­creases my risk of dying’, or, ‘I’m very young, which de­creases my over­all risk’). The weaker the ev­i­dence, the less we should stray away from the base prob­a­bil­ity.

Re­gard­ing the drug, if the ev­i­dence is very weak, we should con­sider the prior prob­a­bil­ity as the chance that any ran­dom drug would work against the dis­ease, or the chance that other drugs of the same class (some might al­ready be tested) are ef­fec­tive.

How­ever, in­tu­itively we tend to largely over­es­ti­mate the weak pieces of ev­i­dence of par­tic­u­lar cases and grav­i­tate to­wards a 50/50 sce­nario. ‘My neigh­bor was al­most dying and re­cov­ered after tak­ing WI’, for ex­am­ple. This may hap­pen also with high mor­tal­ity dis­eases (e.g., can­cer) be­cause only those who sur­vived can talk about their ther­a­pies, and only a hand­ful of the ac­quain­tances of those who passed will crit­i­cize any drug or ther­apy.

This alone is enough to as­sure new HCQs will come, as new or old dis­eases con­tinue to exist, and many peo­ple sur­vive them after tak­ing var­i­ous drugs or treat­ments that might as well do noth­ing. In the real world, it’s very hard to tell the % of peo­ple who sur­vived after tak­ing the drug ver­sus the % among those who didn’t take it. This makes proper rea­son­ing al­most im­pos­si­ble with­out ad­e­quate stud­ies.

No pos­i­tive ev­i­dence

The sec­ond rea­son why new HCQs will come is a huge mis­com­mu­ni­ca­tion issue be­tween the sci­en­tific com­mu­nity and the gen­eral pub­lic: in sci­en­tific terms, we can­not for­mally prove a neg­a­tive state­ment. That is, sci­en­tists never claim that drug X or Y were proven to be in­ef­fec­tive against COVID-​19 or any other con­di­tion. That is due to how stud­ies are de­signed: they seek dif­fer­ences be­tween groups un­der­tak­ing dif­fer­ent treat­ments. Thus, the study can state that (1) there was a dif­fer­ence among groups (e.g., peo­ple tak­ing drug X died less) or (2) that there was no dif­fer­ence. No dif­fer­ence be­tween groups is not for­mally a proof that the drug is in­ef­fec­tive, but it’s sure a good piece of ev­i­dence in this di­rec­tion.

Through­out the COVID-​19 pan­demic, many stud­ies found no dif­fer­ences be­tween HCQ and placebo groups both in terms of death and hos­pi­tal­iza­tion out­comes. With var­i­ous in­de­pen­dent stud­ies ar­riv­ing at the same con­clu­sions, it’s safe to say with good con­fi­dence that HCQ is not a good treat­ment for Covid-​19.

Still, au­thor­i­ties ad­here to the tech­ni­cal terms and state that there is no sci­en­tific ev­i­dence that HCQ works against COVID-​19. How­ever, that is mis­lead­ing. It is true, of course, but it un­der­mines the fact that there is sim­ply no good ev­i­dence in favor of HCQ. This is a huge mis­com­mu­ni­ca­tion prob­lem that needs to be ad­dressed. We will never prove, in sci­en­tific terms, that HCQ is in­ef­fec­tive against COVID-​19, just like we’ll never prove that eat­ing choco­late is also in­ef­fec­tive against COVID-​19 or any other dis­ease!

Nev­er­the­less, the fact that neg­a­tive state­ments are never proven in sci­en­tific terms is not an ex­cuse to stick to in­ef­fec­tive ther­a­pies nor to study them for­ever, hop­ing that one mirac­u­lous study will prove its ef­fi­cacy. Even if one study does so, the ev­i­dence must be taken as a whole, and one pos­i­tive study among many neg­a­tive ones can hap­pen by pure chance. The more stud­ies we do, the higher the chance that a false-​positive re­sult occur.

Con­se­quences of “hy­drox­y­chloro­quine”

Sci­ence has an im­per­fect but the best-​to-​date avail­able method to avoid un­nec­es­sary and poorly-​conducted re­search: peer re­view. Every re­search fund­ing re­quest is eval­u­ated by other mem­bers of the sci­en­tific com­mu­nity. It’s very likely that with­out all the pub­lic at­ten­tion and pres­sure HCQ would have never been tested as much as it has been. Sim­ply put: there was never enough ev­i­dence to con­sider it a vi­able ther­apy. Thus, if tra­di­tional peer re­view process were fol­lowed, a huge amount of fi­nan­cial and human re­sources could have been bet­ter em­ployed else­where, in other and more promis­ing tri­als to tackle the COVID-​19 pan­demic, for in­stance.

This also hap­pened in Brazil with phos­pho­ethanolamine. A Pro­fes­sor started mak­ing un­sup­ported claims that this drug was the “cure” for can­cer. It quickly gained pop­u­lar­ity and mo­men­tum, press­ing politi­cians and reg­u­la­tors to test it or even allow its com­mer­cial­iza­tion with­out any proven ef­fi­cacy. In­deed, a bill that le­gal­ized its med­ical use was ap­proved at the time and later re­voked by the ju­di­ciary. Even human stud­ies were con­ducted, fail­ing to prove its ef­fi­cacy against can­cer. Again, a huge amount of re­sources was spent on use­less re­search be­cause of pres­sure to test a ther­apy with­out good ev­i­dence from in vitro nor an­i­mal stud­ies.

Con­clu­sion

Our ne­glect­ful­ness to­wards prior prob­a­bil­i­ties, com­bined with huge amounts of anec­do­tal ev­i­dence from peers on so­cial media, neigh­bor­hood or other spaces pro­vide a pow­er­ful booster to the pop­u­lar­ity of in­ef­fec­tive ther­a­pies. This, com­bined with the mis­com­mu­ni­ca­tion prob­lem we ex­plored ear­lier, is the per­fect com­bi­na­tion to allow pseu­do­science and even pure bull­shit to strive.

This, along with other so­cial and cul­tural fac­tors, is one of the main rea­sons why com­pletely in­ef­fec­tive ther­a­pies such as home­opa­thy are still so pop­u­lar to this day. We, as sci­en­tists, should pay more at­ten­tion to the pop­u­lar­ity of use­less treat­ments. Even if they have no side ef­fects (and no ef­fect at all, such as home­opa­thy), they pro­vide fer­tile ground for harm­ful mis­con­cep­tions, such as the case with HCQ, to flour­ish.