Advertisement

Alternatives to Randomised Controlled Trials for the Poor, the Impatient, and When Evaluating Emerging Technologies

Open ArchivePublished:November 30, 2018DOI:https://doi.org/10.1016/j.ejvs.2018.10.026

      Pros and Cons of RCTS

      The best scientific methodology to assess a medical intervention is to perform a randomised controlled trial (RCT), comparing the new intervention with placebo or an alternative treatment. A well designed and well performed RCT has the benefit of controlling all other variables that may affect the outcome through randomisation. Randomised trials do have limitations, however. Multiple inclusion and exclusion criteria may result in selection bias, limiting the generalisability of the results. An example is that octogenarians often are excluded from RCTs, but they constitute a considerable proportion of the patients in everyday vascular surgical practice. Randomised trials may be underpowered, or may be difficult to perform in the setting of rare pathologies.
      • Mascoli C.
      • Vezzosi M.
      • Koutsoumpelis A.
      • Iafrancesco M.
      • Ranasinghe A.
      • Clift P.
      • et al.
      Endovascular repair of acute thoraco-abdominal aortic aneurysms.
      • Sorelius K.
      • Wanhainen A.
      • Furebring M.
      • Bjorck M.
      • Gillgren P.
      • Mani K.
      • et al.
      Nationwide study of the treatment of mycotic abdominal aortic aneurysms comparing open and endovascular repair.
      Exceedingly high study costs requiring important external funding is an additional limiting factor, which may be mitigated when RCTs are performed within a registry setting. An important limitation of RCTs, especially when studying emerging technologies, is that they can often be outdated when published several years after study design.
      • Patel R.
      • Sweeting M.J.
      • Powell J.T.
      • Greenhalgh R.M.
      Endovascular versus open repair of abdominal aortic aneurysm in 15-years' follow-up of the UK endovascular aneurysm repair trial 1 (EVAR trial 1): a randomised controlled trial.
      For the reasons mentioned above, although RCTs remain the highest level of evidence in medicine, they are not without problems.

      How to Achieve Unbiased Comparisons in Observational Studies

      Observational studies are often used to assess and compare medical and surgical interventions. However they carry the risk of comparing apples with pears, if adequate adjustments for confounders are not performed. The standard statistical methodology to correct for confounders includes the use of regression analyses, for binary outcomes, for example 30 day mortality (logistic regression), or continuous outcome, for example aneurysm expansion (linear regression). Time dependent outcome, for example survival, is assessed with Cox regression. Univariable regression analysis assesses the outcome correcting for one confounding variable at a time, whereas multivariable regression allows for correction for several variables. Although regression analyses are easily performed with statistical software, they involve complex mathematical methodology with several important limitations, and often require input from statistical expertise to ensure adequate use. A common mistake is to include more variables than the power of the study permits. A rule of thumb is to have at least 10 events of the main outcome (such as death, stroke, etc.) for each included variable.

      Propensity Score Matching

      Another possibility to correct for confounders in observational studies is to compare outcomes in matched subgroups selected from the total observational cohort. Propensity score matched analysis offers the possibility for such an analysis. To perform propensity score matching, a set of known variables is selected and scored. Subjects with identical scores are included in the final analysis when comparing outcome. This technique is excellent when large cohorts of data are available for comparison, for example to assess outcome of two treatment methods in an observational cohort based on registry data.
      • Huang Y.
      • Gloviczki P.
      • Oderich G.S.
      • Duncan A.A.
      • Kalra M.
      • Fleming M.D.
      • et al.
      Outcome after open and endovascular repairs of abdominal aortic aneurysms in matched cohorts using propensity score modeling.
      • Grip O.
      • Wanhainen A.
      • Michaëlsson K.
      • Lindhagen L.
      • Björck M.
      Open versus endovascular revascularisation for the treatment of acute lower limb ischaemia: a nationwide cohort study.
      A propensity score matched analysis mimics an RCT setting. It is important to acknowledge, however, that residual confounders may remain that affect the comparison of the two matched cohorts. Matching can only be performed for variables that are available in the dataset, and other variables (known or unknown) may differ between the matched groups, affecting the comparison. Examples of this are assessment of outcome after open and endovascular aortic repair in propensity score matched observational groups,
      • Huang Y.
      • Gloviczki P.
      • Oderich G.S.
      • Duncan A.A.
      • Kalra M.
      • Fleming M.D.
      • et al.
      Outcome after open and endovascular repairs of abdominal aortic aneurysms in matched cohorts using propensity score modeling.
      and open vs. endovascular treatment of acute limb ischaemia.
      • Grip O.
      • Wanhainen A.
      • Michaëlsson K.
      • Lindhagen L.
      • Björck M.
      Open versus endovascular revascularisation for the treatment of acute lower limb ischaemia: a nationwide cohort study.
      Such comparisons, which can be performed based on large administrative registries with several thousands of patients, seldom include matching for the anatomical complexity of the operation, which thus remains a residual confounder.

      Step Wedge Cluster Randomisation

      When assessing outcome of healthcare intervention, a step wedged cluster randomised trial design offers the possibility for assessment of the effect of the intervention. This technique was first used to assess the effect of hepatitis B vaccination to prevent chronic liver disease in a nationwide study in Gambia.
      The Gambia Hepatitis Intervention Study
      The Gambia hepatitis study group.
      To assess this outcome, hepatitis B vaccination was rolled out sequentially and randomly in different regions, and the effect of the intervention was assessed by comparing the outcome before and after rollout of the intervention in each region. In vascular surgery, a step wedged cluster randomised design has been used for assessment of the effect of abdominal aortic aneurysm screening on aneurysm related mortality in Sweden.
      • Wanhainen A.
      • Hultgren R.
      • Linne A.
      • Holst J.
      • Gottsater A.
      • Langenskiold M.
      • et al.
      Outcome of the Swedish nationwide abdominal aortic aneurysm screening program.
      In conclusion, unbiased comparisons can be achieved in observational studies through adequate use of available statistical methods. All study designs have their pros and cons, and even RCTs can be affected by bias and faulty study design. Meticulously performed observational studies with sound statistical methodology offer a great supplement to RCTs. In areas where RCTs are lacking, such analyses may be the only source of evidence on which clinical practice can be based. They are also well suited to study the rapidly emerging technologies in contemporary vascular surgery, where RCTs often carry the risk of becoming hopelessly outdated when published.

      References

        • Mascoli C.
        • Vezzosi M.
        • Koutsoumpelis A.
        • Iafrancesco M.
        • Ranasinghe A.
        • Clift P.
        • et al.
        Endovascular repair of acute thoraco-abdominal aortic aneurysms.
        Eur J Vasc Endovasc Surg. 2018; 55: 92-100
        • Sorelius K.
        • Wanhainen A.
        • Furebring M.
        • Bjorck M.
        • Gillgren P.
        • Mani K.
        • et al.
        Nationwide study of the treatment of mycotic abdominal aortic aneurysms comparing open and endovascular repair.
        Circulation. 2016; 134: 1822-1832
        • Patel R.
        • Sweeting M.J.
        • Powell J.T.
        • Greenhalgh R.M.
        Endovascular versus open repair of abdominal aortic aneurysm in 15-years' follow-up of the UK endovascular aneurysm repair trial 1 (EVAR trial 1): a randomised controlled trial.
        Lancet. 2016; 388: 2366-2374
        • Huang Y.
        • Gloviczki P.
        • Oderich G.S.
        • Duncan A.A.
        • Kalra M.
        • Fleming M.D.
        • et al.
        Outcome after open and endovascular repairs of abdominal aortic aneurysms in matched cohorts using propensity score modeling.
        J Vasc Surg. 2015; 62: 304-311
        • Grip O.
        • Wanhainen A.
        • Michaëlsson K.
        • Lindhagen L.
        • Björck M.
        Open versus endovascular revascularisation for the treatment of acute lower limb ischaemia: a nationwide cohort study.
        Br J Surg. 2018; 105: 1598-1606
        • The Gambia Hepatitis Intervention Study
        The Gambia hepatitis study group.
        Cancer Res. 1987; 47: 5782-5787
        • Wanhainen A.
        • Hultgren R.
        • Linne A.
        • Holst J.
        • Gottsater A.
        • Langenskiold M.
        • et al.
        Outcome of the Swedish nationwide abdominal aortic aneurysm screening program.
        Circulation. 2016; 134: 1141-1148

      Comments

      Commenting Guidelines

      To submit a comment for a journal article, please use the space above and note the following:

      • We will review submitted comments as soon as possible, striving for within two business days.
      • This forum is intended for constructive dialogue. Comments that are commercial or promotional in nature, pertain to specific medical cases, are not relevant to the article for which they have been submitted, or are otherwise inappropriate will not be posted.
      • We require that commenters identify themselves with names and affiliations.
      • Comments must be in compliance with our Terms & Conditions.
      • Comments are not peer-reviewed.