Volume 33, Issue 2 , Pages 172-176, February 2007
Impact of Study Design on Outcome after Endovascular Abdominal Aortic Aneurysm Repair. A Comparison between the Randomized Controlled DREAM-trial and the Observational EUROSTAR-registry
Article Outline
Background
Patients with abdominal aortic aneurysm (AAA) can be treated by transfemoral endovascular intervention and by conventional open surgery. Level-one evidence of the safety and efficacy of one treatment mode over the other is only provided by a randomised controlled trial (RCT). Results reported by voluntary registries are considered less valid than data from RCTs. On the other hand the outcome of a RCT may not be generalisable to the common practice because of vigorous selection of patients and institutions.
Purpose
The outcomes reported by the Dutch Randomized Endovascular Aneurysm Management (DREAM) trial were compared with the results of the EURopean collaborators on Stent-graft techniques for AAA Repair (EUROSTAR) registry.
Methods
To obtain comparable study groups with regard to risk factors equal proportions of ASA I, II and III patients as observed in the endovascular arm of the DREAMtrial were selected at random from the EUROSTAR-registry. All patients had an aneurysm of at least 50
mm. Only patients, who had been enrolled into the registry from 1999, were selected to avoid the influence of first generation endografts which are not longer in use. Patient characteristics and outcomes of endovascular AAA repair (EVAR) of EUROSTAR and DREAM-trial participants were compared. Differences in early findings between study groups were assessed by Chi-Square tests for discrete variables and by Wilcoxon rank sum tests for continuous variables. Follow-up variables were analysed by Kaplan-Meier and Cox proportional hazard models.
Results
Data of 177 patients of the DREAM trial with randomization to EVAR and 856 patients selected in the EUROSTAR-registry were compared. Baseline characteristics were comparable between the EUROSTAR-cohort and EVAR-arm of the DREAM-trial. The 36-month survival-rate was 87.6% for EVAR-arm in the DREAM-trial similar to the 86.8% found in this EUROSTAR-study population. The freedom of secondary procedures reached after 3 years 85.7%, and 86.9% in the DREAM and EUROSTAR-cohort, respectively.
Conclusion
We found comparable characteristics and outcomes between patients of comparable risk class of the EUROSTAR-registry and the EVAR-cohort of the DREAM-trial. This demonstrates the following: first, the EUROSTAR-data provide reliable information, and further comparisons of registry data with patients treated by conventional AAA surgery may be justified. Secondly, the various outcomes of the randomised DREAM trial appear generalisable, as it agrees with observations in a broad common practice derived database.
Keywords: Study design, Randomized clinical trial, Observational registry, Abdominal aortic aneurysm, Endovascular repair, DREAM, EUROSTAR
Introduction
A randomised controlled trials (RCT) is generally accepted as the best possible evidence to indicate the value of an intervention. A prospective definition of methods and outcome measures, blind assessment of outcomes and unbiased selection of subjects and controls are indispensable for scientifically sound comparison. RCTs are carefully crafted experiments that provide evidence on the benefit of a specific health intervention only under strictly controlled study conditions. Because of differences in conditions, a health benefit observed in a RCT may not necessarily be maintained when the intervention is introduced into routine clinical practice.1, 2, 3
For the current analysis we used data from the EUROpean collaborators on Stent-graft Techniques for Abdominal aortic aneurysm Repair (EUROSTAR) registry and the Dutch Randomized Endovascular Aneurysm Management (DREAM) trial. A cohort from the EUROSTAR registry was stratified with a comparable ASA physical status classification distribution as the endovascularly treated cohort recruited in the DREAM-trial. The characteristics and outcomes of this EUROSTAR-cohort were compared to these of the EVAR-cohort of the DREAM-trial to analyse any differences or similarities. To avoid bias from smaller series or incomplete reporting of follow-up only centers with the best compliance to the registry recruitment and follow-up schedule were selected for this comparison.
Methods
The randomized clinical DREAM-trial commenced in 2000. Patients referred to clinics departments of vascular surgery at 24 centers in the Netherlands and 4 centers in Belgium participated in the trial (Appendix). All patients were diagnosed with an AAA of at least 5cm, were considered suitable candidates for either endovascular or open repair, and were randomly assigned to receive one of these treatments.4
The EUROSTAR-project is a voluntary multicenter registry, which was established in 1996.5 Of a total of 2696 patients enrolled in the registry by the “top centers” (Appendix). The centers in this analysis were selected on the basis of their follow-up compliance and enrolled patient volume. The selected centers were still active participating in the EUROSTAR registry, enrolled at least 50 patients treated with modern devices, and the follow-up compliance had to be minimal 80%. We stratified a patient group from this selected EUROSTAR-cohort with a comparable distribution of ASA physical status classification as the patient group included in EVAR arm of the DREAM-trial (21% ASA I, 71% ASA II, and 8% ASA III).6 All patients with ASA I enrolled in the EUROSTAR registry were selected (N=180). Subsequently a random sample of 605 patients with ASA II and 71 with ASA III were selected. No patients with ASA IV were selected as this category was excluded for participation in the DREAM-trial. All patients in the present study group had an aneurysm of at least 50mm. Only patients, who had been enrolled into the registry from 1999, were selected to avoid the influence of first generation withdrawn types of devices. Completeness of follow-up in this selected cohort at 1, 3, 5 year was 97.6%, 98.6% and 99.9%, respectively.
The 856 selected patients of the EUROSTAR-cohort where compared to the 177 patients of the EVAR-arm of the DREAM-trial. Differences in findings between study groups were assessed by Chi-Square tests for discrete variables and by Wilcoxon rank sum tests for continuous variables. Kaplan-Meier analysis was used for survival analysis. Multivariate Cox models were used to determine whether baseline and follow-up variables were independently associated with adverse outcomes. A p-value of 0.05 or less was considered as statistical significant. Data was analysed with SAS statistical software, version 8.02 (SAS Institute).
Results
There were 165 men and 12 women in the EVAR-arm of the DREAM-trial, with a median age of 70.6 years. The selected EUROSTAR-cohort consisted of 856 patients of which 793 men with a mean age of 71.6 years. There were no major differences in demographic, morphologic and operative characteristics at baseline between the EUROSTAR-cohort and the EVAR-arm of the DREAM-cohort (Table 1, Table 2, Table 3).
Table 1. Baseline characteristics of the patients
| EUROSTAR N=856 | DREAM N=177 | P-value | |
|---|---|---|---|
| Age (years) | 71.60±7.67 | 70.61±6.51 | NS |
| Male sex | 793 (92.6) | 165 (93.2) | NS |
| Moderate/Severe SVS/AAVS risk score† | |||
| Diabetes | 88 (10.3) | 19 (10.7) | NS |
| Smoking | 414 (48.4) | 113 (63.9) | .0003 |
| Hypertension | 488 (57.0) | 104 (58.8) | NS |
| Hyperlipemia | 331 (38.7) | 81 (45.8) | NS |
| Carotid disease | 75 (11.6) | 27 (15.3) | NS |
| Cardiac disease | 390 (45.6) | 75 (42.4) | NS |
| Renal disease | 89 (10.4) | 13 (7.3) | NS |
| Pulmonary disease | 211 (24.7) | 48 (27.1) | NS |
| Sum of SVS/AAVS risk-factor score‡ | 3.18±2.58 | 3.51±2.65 | NS |
| ASA-class# | |||
| I | 180 (21.0) | 38 (21.5) | NS |
| II | 605 (70.7) | 125 (70.6) | NS |
| III | 71 (8.2) | 14 (7.9) | NS |
| Previous Laparotomy | 206 (24.3) | 43 (25.1) | NS |
| Maximal diameter of aneurysm (mm) | 60.37±10.25 | 60.56±8.93 | NS |
| Aneurysm morphology class∗ | |||
| A | 135 (15.9) | 12 (7.0) | .0025 |
| B | 484 (57.0) | 114 (66.7) | .0193 |
| C | 125 (14.7) | 16 (9.4) | NS |
| D | 62 (7.3) | 14 (8.2) | NS |
| E | 43 (5.1) | 15 (8.8) | NS |
†Risk score reporting adhered to the guidelines of the Society for Vascular Surgery/American Association for Vascular Surgery (SVS/AAVS).9 Moderate/Severe SVS/AAVS score was defined as a score |
‡Sum of all eight recorded SVS risk scores. Each risk score represented as a value between 0 and 3, according to increasing severity level. |
#ASA: American Society of Anesthesiologists. |
∗N |
Table 2. Characteristics of aneurysm-repair procedure
| EUROSTAR N=856 | DREAM N=177 | P-value | |
|---|---|---|---|
| N (%) | N (%) | ||
| Type of anaesthesia | |||
| General | 524 (61.2) | 95 (53.7) | NS |
| Regional | 248 (29.0) | 72 (40.7) | .0022 |
| Local | 84 (9.8) | 10 (5.6) | NS |
| Type of stent-graft | |||
| Bifurcated | 832 (92.9) | 167 (94.4) | NS |
| Aorto-uni-iliac | 46 (5.2) | 7 (4.0) | NS |
| Straight tube | 10 (1.2) | 1 (0.6) | NS |
| Unknown | 6 (0.7) | 2 (1.1) | NS |
| Brand of endograft | |||
| Zenith | 310 (36.2) | 59 (33.3) | NS |
| Talent | 333 (38.9) | 49 (27.7) | .0049 |
| Excluder | 77 (9.0) | 37 (20.9) | <.0001 |
| AneuRx | 77 (9.0) | 12 (6.8) | NS |
| Lifepath | 3 (0.4) | 4 (2.3) | .0048 |
| Endologix | 7 (0.8) | 1 (0.6) | NS |
| Fortron | 23 (2.7) | 8 (4.5) | NS |
| EVT | 5 (0.6) | 5 (2.8) | .0056 |
| Anaconda | 21 (2.4) | – | .0046 |
| Unknown | – | 2 (1.1) | .0008 |
Table 3. Surgical and postoperative data
| EUROSTAR N=856 | DREAM N=177 | P-value | |
|---|---|---|---|
| Duration of surgery - min | |||
| Mean±SD | 123.65±50.02 | 194.46±65.70 | <.0001 |
| Median | 120 | 180 | |
| Interquartile range | 90–150 | 155–220 | |
| Duration of hospitalisation | |||
| Mean±SD | 5.38±7.84 | 5.76±5.64 | NS |
| Median | 4 | 4 | |
| Interquartile range | 3–5 | 3–6 | |
| Replaced blood volume (ml) | |||
| Mean±SD | 62.51±299.29 | 99.52±488.29 | NS |
| Median | 0 | 0 | |
| Interquartile range | 0–0 | 0–0 | |
| Operative Mortality N (%) | 17 (2.0) | 2 (1.2) | NS |
The most frequently reported risk factors were hypertension and a history of cardiac disease. In the EUROSTAR-cohort fewer patients were current smokers or had smoked in the past 10 years (48.4% vs. 63.9%, p=.0003). The mean maximal aneurysm diameter was approximately 60mm, in both groups. However, the distribution of aneurysm morphology classes regarding aneurysmal involvement of the iliac arteries differed between the study populations. The most frequently occurring aneurysm morphology class was B in both study groups. In the EUROSTAR-cohort a larger proportion class A aneurysms were included (15.9% vs. 7.0%, p=.0025). Most patients were treated with a bifurcated stent-graft (93-94%). Regional anaesthesia was used in a higher proportion in the DREAM-trial (40.7% vs. 29.0%, p=.0022).
The operative mortality was 2.0% in the EUROSTAR-cohort versus 1.2% in the EVAR arm of the DREAM-trial (p=NS) (Table 3). Moderate and severe systemic complications were reported in 9.3% vs. 11.7%, and local complications in 10.6% vs. 16.4%, in the EUROSTAR and DREAM-cohort, respectively (Table 4).
Table 4. Complications
| EUROSTAR N=856 N (%) | DREAM N=177 N (%) | P-value | |
|---|---|---|---|
| Endoleak | 1876 (20.6) | 27 (15.6) | NS |
| Type 1 | 31 (3.6) | 5 (2.8) | NS |
| Type 2 | 131 (15.3) | 19 (10.7) | NS |
| Type 3 | 15 (1.8) | 2 (1.1) | NS |
| Early Moderate/Severe complications | |||
| Local – Vascular implant related | 91 (10.6) | 28 (16.4) | .0490 |
| Systemic | 820 (9.3) | 20 (11.7) | NS |
Overall 84 patients died in the EUROSTAR-cohort and 31 in the DREAM-cohort. Within a follow-up period of 3 years the survival-rate in the DREAM-trial (87.6%) was comparable with the EUROSTAR-cohort (86.8%). Secondary interventions were required in 85 and 21 patients of the EUROSTAR and DREAM-cohort, respectively. The freedom of secondary procedures reached after 3 years 85.7% and 86.9% in the DREAM-and EUROSTAR-cohort, respectively. Again we noticed the comparability between the EVAR-arm of the DREAM-trial and the EUROSTAR-cohort. In the EUROSTAR-cohort, with longer follow-up time, a cumulative survival rate of 83.2% and 82.5% was documented after 4 and 5 years of follow-up, respectively. The freedom from secondary interventions drops to 83.9% after 4 years of follow-up.
Multivariate analysis of demonstrated an association between a higher mortality and advanced aged (HR=1.06, 95%CI=1.03–1.09, p<.0001), baseline pulmonary impairment (HR=1.74, 95%CI=1.19–2.54, p=.0046), and larger aneurysm diameter at baseline (HR=1.02, 95%CI=1.01–1.04, p=.0091). The only independent variable influencing the secondary intervention rate was advanced age (HR=1.03, 95%CI=1.00–1.07, p=.0363). These hazard rates are mainly influenced by the larger number patients enrolled in the EUROSTAR registry. When analysing the two study groups separately we noticed that previously observed associations were confirmed in the EUROSTAR subgroup. In the DREAM-cohort survival was associated with advanced age (HR=1.14, 95%CI=1.07–1.23, p=.0002) and diabetes mellitus (HR=4.46, 95%CI=1.41–14.05, p=.0107). For secondary intervention rate no association could be found.
Discussion
For epidemiologic assessments of the effects of treatments on outcomes different study designs can be used. Studies can be organised as a randomized clinical trial (RCT) or as an observational study. In the hierarchy of research designs, RCTs are considered the best quality evidence in medical research. However, an RCT requires an enormous commitment and input from the investigators, participants, and patients. These trials are also costly enterprises, frequently paid out of public resources. For these reasons, investigators should always make a balanced decision of the type and conduct of their research setup. Observational studies are often viewed as having less validity because they reportedly overestimate treatment effects. An observational study usually includes patients with a large variety of coexisting illness and a wider spectrum of disease severity. In contrast, a RCT studies a distinct group of patients as a result of strictly defined inclusion and exclusion criteria regarding coexisting pathology and severity of disease. An experimental protocol may not be representative of the wider spectrum of comorbid conditions typically for the current clinical practice.1, 2, 3
In the current report we analysed characteristics and outcomes of endovascular AAA treatment of patients in the DREAM-trial and patients enrolled in the EUROSTAR-registry with comparable inclusion criteria.6, 7 It should be noted that the registry as a whole includes a much broader spectrum of patients than was used in this analysis. In the overall registry 50% of the patients are at high medical and operative risk with an ASA-classification of III or IV.
We found similar baseline and surgical characteristic between the selected EUROSTAR-cohort and the DREAM-trial-patients. Operative mortality was identical in both study groups. During the first three years of follow-up also similar survival rates were observed in the EUROSTAR-cohort and EVAR-arm of the DREAM trial. Mortality in the EUROSTAR-cohort was associated independently with advanced age, pulmonary status and aneurysm size. While in the DREAM-cohort death was associated with advanced aged and diabetes mellitus.
It has been suggested that registries tend to overestimate the better outcome of the new introduced treatment. This view is arguable as recently was emphasized in two reports.1, 2 These reports concluded on the basis of several comparisons that results of well-designed observational studies do not systematically overestimate the magnitude of the effects of treatment, as compared to RCTs assessing the same research question and with similar criteria to select study subjects. With regard to patient characteristics, mortality and secondary procedure data we can support this more recent opinion. Blankensteijn et al. concluded on the basis of a large overview of the literature on treatment outcomes of AAA, that registries are accurate in documenting death, but unreliable with respect to postoperative morbidity.8 Our analysis of the complications as recorded by the EUROSTAR and DREAM revealed a similar frequency of secondary interventions, early endoleaks and complications in the registry than in the trial. In the EUROSTAR-cohort the secondary intervention rate was associated with advanced age, while no association was observed in the DREAM-cohort.
In conclusion, we found similar characteristics and outcomes in patients with comparable risk class profiles enrolled in the EUROSTAR and the EVAR-arm of the DREAM-trial. This demonstrated in the first place the accuracy of the EUROSTAR-data. Comparisons with conventionally treated patients and outcomes of the EUROSTAR registry appeared valid. Secondly, and equally important, the outcome of the DREAM trial appeared generalisable, as it agrees with findings in a broad common practice derived database.
Appendix.
DREAM Participating centers:
BELGIUM: St Jozef Hospital Turnhout; St. Trudo Hospital St. Truiden; University Hospital Antwerpen; University Medical Center Gent.
THE NETHERLANDS: Catharina Hospital Eindhoven; University Medical Center Utrecht; Academic Medical Center Amsterdam; Erasmus Medical Center Rotterdam; University Hospital Groningen; St. Franciscus Gasthuis Rotterdam; Rijnstate Hospital Arnhem; Leyenburg Hospital's Gravenhage; Albert Schweitzer Hospital Dordrecht; Atrium MedicalCenter Heerlen; MC Rijnmond Zuid Rotterdam; Jeroen Bosch Hospital den Bosch; St. Elisabeth Hospital Tilburg; Maxima Medical Center Veldhoven; OLVG, Amsterdam; Meander Medical Center Amersfoort; Vlietland Hospital Schiedam; University Medical Center Nijmegen; Martini Hospital Groningen; MC Haaglanden's Gravenhage; Hospital Bernhoven Oss; Oosterschelde Hospital Goes. VU Medical Center Amsterdam; Leiden University Medical Center; University Medical Center Maastricht; Bronovo Hospital's Gravenhage.
EUROSTAR Participating centers in current study:
BELGIUM: Onze Lieve Vrouwe Ziekenhuis Aalst; St. Augustinus Hospital Antwerpen; A.Z.Sint Blasius Dendermonde; Universitair Ziekenhuis Gent; Universitair Ziekenhuis Leuven;AZ Vesalius Tongeren.
GERMANY: Stadtischen Kliniken Frankfurt.
IRELAND: St. James Hospital Dublin.
NORWAY: Aker Hospital Oslo; University Hospital of Trondheim.
SWITZERLAND: Clinic for Cardiovascular Surgery Bern.
THE NETHERLANDS: Rijnstate Ziekenhuis Arnhem; Catharina Ziekenhuis Eindhoven; Medisch Spectrum Twente Enschede; Universitair Medisch Centrum Groningen; Canisus-Wilhelmina Ziekenhuis Nijmegen; Isala Klinieken Zwolle.
UNITED KINGDOM: Royal University Hospital Liverpool; Freeman Hospital NorthernVascular Centre New Castle-Upon-Tyne.
References
- . Randomized, controlled trials, observational studies, and hierarchy of research designs. N Engl J Med. 2000;342:1887–1892
- . A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000;342:1878–1886
- . Surgery and the randomised controlled trial: past, present and future. Med J Aust. 1998;169:380–383
- . The Dutch Randomised Endovascular Aneurysm Management (DREAM) trial: background, design and methods. J Cardiovasc Surg (Torino). 2002;43:379–384
- . The need for clinical trials of endovascular abdominal aortic aneurysm stent-graft repair: The EUROSTAR project. J Endovasc Surg. 1997;4:72–77
- A Randomized Trial Comparing Conventional and Endovascular Repair of Abdominal Aortic Aneurysms. N Engl J Med. 2004;351(16):1607–1618
- Two-year outcomes after conventional or endovascular repair of abdominal aortic aneurysms. N Engl J Med. 2005;352:2398–2405
- . Influence of study design on reported mortality and morbidity rates after abdominal aortic aneurysm repair. Br J Surg. 1998;85:1624–1630
- Reporting standards for endovascular aortic aneurysm repair. J Vasc Surg. 2002;35:1048–1060
PII: S1078-5884(06)00510-7
doi:10.1016/j.ejvs.2006.09.011
© 2006 Published by Elsevier Inc.
Volume 33, Issue 2 , Pages 172-176, February 2007
