Background: Multiple sclerosis (MS) is a chronic progressive disorder that affects approximately 2.3 million people worldwide. There is a growing demand for real-world evidence on MS treatments, yet, beyond the well-known concerns about exposure-related bias, the analysis of non-randomized routine care data may be prone to bias due to informative missingness of relevant patient outcomes.

Objectives: To evaluate existing and develop new methods for estimating comparative treatment effects from routine care registry data when outcomes are assessed at irregular visit schedules.

Methods: We conducted an extensive simulation study where we generated individual MS disease trajectories for MS patients from multiple practices. Patient outcomes were expressed as Expanded Disability Status Scale (EDSS), a standard reference scale to assess progression of MS disease, and generated for distinct months using multilevel normal distributions correlated over time. To mimic the irregular visit times in clinical practice, we censored patient outcomes according to an informative missingness procedure. For each simulated dataset, we estimated the treatment effect (with respect to no treatment) in terms of time to confirmed EDSS progression at 6 months, as defined in clinical practice. Setting: Simulation study mimicking routine clinical practice Exposures: MS disease-modifying therapies Main outcome measures: Time to confirmed EDSS progression at 6 months. Statistical analysis: For each simulated dataset, missing EDSS scores were imputed using (i) last observation carried forward (LOCF), (ii) rounding to the closer regular visit schedule (RND), and (iii) advanced mixed effects models accounting for patient clustering and autocorrelation. Subsequently, time to confirmed EDSS progression at 6 months treatment effects were estimated using Cox regression.

Results: Imputation using LOCF or RND leads to substantial bias in confirmed EDSS progression and consequently in treatment effect estimates, particularly when patient visits are scarce and highly dependent on patient characteristics. EDSS scores imputed using multilevel models were much more accurate to estimate confirmed EDSS progression and improved the bias and precision of treatment effect estimates.

Conclusions: Studies analyzing real world effectiveness of medical interventions should account for informativeness of patient visits. Methods such as LOCF and RND should be avoided when patient visits are infrequent; instead, advanced statistical models should be used to impute missing patient outcomes and account for their uncertainty.