The RECOVERY trial is the first large scale trial to assess the efficacy of different treatments for COVID-19 patients hospitalized in the United Kingdom. In The New England Journal of Medicine, S. James Adelstein Professor of Health Care Policy (Biostatistics) Sharon-Lise T. Normand, PhD, evaluates the methods and success of this trial.
Patients were randomly assigned to one of four treatments- dexamethasone, hydroxychloroquine, lopinavir–ritonavir, or azithromycin- or to usual care using a platform design. Out of the 11,303 patients in the trial, 6,425 were randomized to dexamethasone or to usual care. The patients who received dexamethasone had a 2.8% absolute 28-day mortality benefit compared to those who received usual care. This benefit increased to 12.1% for those patients on mechanical ventilation at the time of randomization.
Normand applauds the investigators for their quick execution of this trial in a time of crisis. However, she raises several study weaknesses that may limit the interpretation of the findings. The RECOVERY trial did not have any pre-specification rules of when therapies should be added or removed as is common in platform trials. The investigators also did not adjust for multiplicity of testing – this is the process of conducting multiple hypothesis tests without an adjustment in the overall error rate. Such adjustments are required in order to minimize the risk of declaring a difference in treatments when no difference exists.
A third design decision was the authors’ lack of randomization within hospitals. While the study was unblinded, the authors may have been concerned about study investigators guessing what the next randomized treatment would be. However, within-hospital practice tendencies may have impacted the effectiveness of the treatments; hospitals may differ in who they select when patients are placed on mechanical ventilation for instance. Accounting for within-hospital effects is thus critical in the RECOVERY trial. Normand emphasizes the need for adherence to long-held scientific design and analytical principals, even under the pressure of a pandemic.