Covariate Adjustment in Randomized Trials
Chapter 1 Covariate Adjustment: Opportunities and Challenges
Randomized trials are an important tool for generating evidence to inform practice in medicine, public health, science, and industry. Planning ethical, informative, and cost-effective studies requires many careful considerations during the development of the study design and statistical analysis plan. One of these considerations should be how to utilize the information about study participants that is known before randomization takes place. Variables observed prior to randomization, known as baseline covariates, can provide information about the outcomes that are observed during the study. Leveraging the information in baseline covariates could improve our inference about the potential benefits or harms of an intervention. Utilizing this information effectively in the study design and analysis plan has the potential to reduce the number of participants required or the duration of the study without sacrificing the precision of our inference. Such methods can make evaluating interventions more ethical and cost effective, and is generally supported by regulatory agencies.
This book is about the practice of applying covariate adjustment in randomized trials. While there are many approaches for covariate adjustment, our focus will be on estimating the average outcome in a population under different potential treatment assignments, known as marginal treatment effects. Investigators may be interested in comparing the average outcome within strata of a population under different potential treatment assignments, which are known as conditional treatment effects. While conditional treatment effects will arise in our discussion along the way, they will not be our primary focus.
Since randomized trials serve such an important role in policy and practice, stakeholders may be understandably cautious about changing how such research is carried out. It is important that investigators, regulators, reviewers, and readers trust the methods chosen for an application and understand how they compare to potential alternative approaches. It is natural to ask what assumptions are required for the validity of a covariate adjusted analysis, how such results should be interpreted, and how these differ from an unadjusted approach.
The methods that we consider should meet certain acceptability criteria. Firstly, an acceptable approach must not change the statistical and scientific focus of investigation. In other words, our covariate-adjusted approach should infer about the same quantities of interest, or estimands, as an unadjusted approach. Secondly, an acceptable approach must not require stronger, more restrictive assumptions: the assumptions for the validity of a covariate-adjusted approach should be the same or less stringent than an unadjusted approach. Finally, a covariate-adjusted approach should have the same or better precision than the unadjusted estimator.
Statisticians rely on regression models in many contexts to relate covariate information to an outcome of interest. Before interpreting regression coefficients, there is often a great deal of consideration put into choosing an appropriate specification for the model: which variables should be included in the model, how they should be included, and how to link the covariates to the scale of the outcome. Appropriate interpretation and application of models requires some assumptions about how closely the fitted model is to the true data generating process, which is unknown in practice.
Since regression models are used to perform covariate adjustment, it is natural to ask whether using such models also requires assumptions about how closely the specified model is to the true data generating process. What may be surprising to some is that covariate adjusted analyses can be valid under arbitrary model misspecification. Even though covariate adjustment may use regression models to construct an estimator, the validity of the resulting estimator does not require that the model is correctly specified. This is particularly important if we are to have covariate-adjusted methods that do not require stronger assumptions than an unadjusted analysis. Use of regression models allow us to do covariate-adjusted analyses for continuous, binary, ordinal, or time-to-event outcomes, allowing them to be applied to many research areas and questions.
While covariate adjustment has the potential to improve precision and make research more ethical and cost effective, there are still challenges to be addressed in practice. Firstly, the amount of precision gained by using covariate adjustment is not known a priori. We need study designs and analysis plans that guarantee the desired level of statistical power and control of error rates irrespective of the amount of precision gained by using covariate adjustment. Secondly, we need a theoretical framework which includes a broad class of estimators in a single framework. The methods employed must address missing data due to dropout, and incomplete data from participants whose outcomes have yet to be collected. Investigators need the information to effectively advocate for the use of these methods with stakeholders, regulators, and reviewers. Practitioners need validated, freely available software to make applying these methods straightforward. Finally, general guidance and best practices need to be established so as to make covariate adjusted analyses as well understood and trusted as their unadjusted counterparts. We will address each of these challenges in the material ahead.
1.1 Using this Book
This book is meant to provide worked examples to help practitioners apply covariate adjustment in practice. The material will be most accessible to a reader who has a good understanding of probability and statistics (confidence intervals, hypothesis testing), generalized linear models, and survival analysis. Some familiarity with concepts in randomized trials and causal inference is helpful, but not required. In order to make these methods broadly available, we provide worked examples using the R environment for statistical computing, which is free and open source software. While not all users may be familiar with R, we provide example code and links to additional resources to help users understand how to apply it in practice. The data in our examples are simulated from actual randomized trial data, meant to mimic key features such as missingness patterns and the distribution of outcomes and covariates.
We will start with an overview of key ideas and findings from research in randomized trial methodology. From there, we will discuss different targets of inference, or estimands, that may be of interest to investigators. Afterwards, we provide a brief overview of using R, and how to install the necessary software to perform covariate adjustment. With the necessary background in place, we will begin with the most simple and commonly used approach for covariate adjustment, the analysis of covariance (ANCOVA), show how this approach can be generalized to binary and other types of outcomes. From there, we will discuss how to address the issues of missing data in baseline covariates and outcomes using doubly robust methods.
Since randomized trials are often designed with incomplete and imprecise information, the study design should incorporate pre-planned analyses to determine if the study should continue or be stopped for either success or futility. We discuss how covariate adjustment can be integrated into such analysis plans. Finally, we discuss some recommendations for applying these methods in practice.
1.2 Case Studies
The datasets used in our examples are simulated data based on actual randomized trials, with considerable effort spent making the data as realistic as possible. The original study data was used to create regression models for the outcomes of interest and missingness patterns. Next, simulated covariate data were created by resampling the original covariate data, and perturbing the resampled data. Simulated outcome data and missingness patterns were generated using predictions from the outcome regression models, using the simulated covariates as input.
1.2.1 Buprenorphine tapering schedule and illicit opioid use: CTN-03
CTN-03 (NCT00078117) was two-arm a phase III trial to compare two potential tapering schedules of the drug buprenorphine, a pharmacotherapy for opioid dependence (Ling et al. 2009). At the time of the study design, there was considerable variation in tapering schedules in practice, and a knowledge gap in terms of the best way to administer buprenorphine to control withdrawal symptoms and give the greatest chance of abstinence at the end of treatment. It was hypothesized that a longer taper schedule would result in greater likelihood of a participant being retained on study and providing opioid-free urine samples at the end of the drug taper schedule.
Participants were randomized 1:1 to a 7-day or 28-day taper using stratified block randomization across 11 sites in 10 US cities. Randomization was stratified by the maintenance dose of buprenorphine at stabilization: 8, 16, or 24 mg. The structure of the CTN-03 simulated data is as follows:
- Baseline Covariates
age: Participant age at baselinesex: Participant sexrace: Participant raceethnic: Participant ethnicitymarital: Participant marital status
- Randomization Information
arm: Treatment Armstability_dose: Stratification Factor
- Baseline (
_bl) & End-Of-Taper (_eot) Outcomes:arsw_score: Adjective Rating Scale for Withdrawal (ARSW) Score at baselinecows_score: Clinical Opiate Withdrawal Scale (COWS) Score at baselinecows_category: COWS Severity Category - Ordinalvas_crave_opiates: Visual Analog Scale (VAS) - Self report of opiate cravingsvas_current_withdrawal: Visual Analog Scale (VAS) - Current withdrawal symptomsvas_study_tx_help: Visual Analog Scale (VAS) - Study treatment helping symptomsuds_opioids: Urine Drug Screen Result - Opioidsuds_oxycodone: Urine Drug Screen Result - Oxycodoneuds_any_positive: Urine Drug Screen - Any positive result
1.2.2 Functional Outcome in Hemorrhagic Stroke: MISTIE III
Hemorrhagic stroke occurs when a blood vessel in the brain ruptures, causing a bleed inside the skull. The bleeding from an ICH can occur in the brain tissue itself (an intracerebral hemorrhage, or ICH), or in the fluid-filled channels in the brain (an intraventricular hemorrhage, or IVH). The MISTIE III trial (NCT01827046) was a phase III study comparing a minimally invasive surgical intervention to conventional medical management for the treatment of spontaneous, non-traumatic ICH (Hanley et al. 2019).
In this study, participants were randomized 1:1 to receive either standard-of-care medical management or a minimal invasive surgery with Alteplase for ICH removal. Outcomes were measured at 30, 180, and 365-days post-randomization using the Modified Rankin Scale (MRS), which measures functional outcome on a scale ranging from 0 (no residual symptoms) to 6 (death). The MRS was collapsed into a binary variable, representing a score of 0-3 (no symptoms to moderate disability but able to walk without assistance) or 4-6 (unable to walk or attend to daily activities without assistance to death). Survival was also assessed, with patients administratively censored on the date of their final MRS assessment.
The data from MISTIE III was used to create a synthetic dataset for educational purposes. Baseline covariates include demographics, medications and comorbidities, characteristics of the stroke (location of the ICH lesion, the size of the ICH and IVH lesions on CT scans), and neurological status on presentation to the hospital (the Glasgow Coma Scale, or GCS).
In addition to the longitudinal measures of the MRS at 30-, 180-, and 365-days post randomization, mortality data are included. The structure of the data is as follows:
sim_participant_id: Patient id- Baseline Covariates
age: Age in yearsmale: male sexhx_cvd: cardiovascular disease historyhx_hyperlipidemia: hyperlipidemia historyon_anticoagulants: on anticoagulant medicationon_antiplatelets: on antiplatelet medicationich_location: intracerebral hemorrhage location: (Lobar,Deep)ich_s_volume: intracerebral hemorrhage volume on stability scanivh_s_volume: intraventricular hemorrhage volume on stability scangcs_category: presenting Glasgow Coma Score (GCS)
- Treatment:
arm: treatment armich_eot_volume: intracerebral hemorrhage volume on end-of-treatment scan
- Outcome:
mrs_30d: MRS at 30 days (0-3,4,5,6)mrs_30d_complete: MRS at 30 days if no data were missingmrs_180d: MRS at 180 days (0-2,3,4,5,6)mrs_180d_complete: MRS at 180 days if no data were missingmrs_365d: MRS at 365 days (0-1,2,3,4,5,6)mrs_365d_complete: MRS at 365 days if no data were missingdays_on_study: days until death or administrative censoringdied_on_study: participant died (1) or is censored (0)