Publication of Vascular Surgical Registry Data: Strengths and Limitations

The diversity of vascular surgery, treating different pathologies in various locations of the vascular tree, is a core value. However, it turns into an obstacle when it comes to research and when creating an evidence base for clinical decision making.

Vascular surgical registry data have the following advantages: they are prospectively collected, readily available, contemporary, and reflect everyday unselected practice. Internal and external validity are of paramount importance to ensure robust interpretation of data. Internal validity refers to the accurateness of the registered data, for example when compared with patient records. If the registry has a good external validity (i.e., has few missing patients/operations), selection bias is minimal. The Swedish vascular registry, Swedvasc, was the first population based vascular surgical registry, and has been validated continuously, resulting in a more than 30 year process of quality improvement. In an independent international validation, the external validity was 100% for carotid endarterectomy and 98.8% for abdominal aortic aneurysm (AAA) repair. When planning a research project based on registry data, it is recommended to always include validation, by comparing data with other registries (examining external validity) and/or by scrutinising a sample of complete case records (investigating internal validity). Although many registries are not yet validated, such a process is ongoing within Vascunet.

There are limitations when working with registry data. Although an external validity of 98% may seem excellent, it can still seriously affect the investigation if a large proportion of non-registered patients had a negative outcome, for example death or stroke. A targeted validation of missing cases, and/or of those most likely to have suffered adverse events, strengthens the investigation considerably. It should also be recognised that most registries do not collect data on patients not operated on (e.g., patients with a ruptured AAA who are not treated), resulting in selection bias.

In clinical practice, patients are selected to a treatment based on various factors, including disease extent and comorbidities. Therefore, when treatment methods and outcomes are compared in registries, correction for confounders is necessary. Correction for confounders can either be performed with statistical methods (e.g., multivariate analysis), or by selection of matched patient cohorts within a registry (e.g., propensity score matching). A limitation is the fact that important confounders may not be available among the variables in the registry, resulting in residual confounding affecting the results.

One way of addressing the issue of residual confounding is to perform a case–control study nested in the cohort of the registry. Controls are randomly selected within the registry cohort, with complete data extraction for the selected patients from case records. An example of this study design compared patients who developed colonic ischaemia (cases) with those who did not (controls), among all patients undergoing open aorto-iliac surgery. The Achilles' heel of a case control study is the selection of controls, which can be biased. However, if the entire cohort is controlled for, the study becomes scientifically robust.

A great advantage of registry data is the fact that it is possible to merge data from many countries for international comparisons.^,  The Vascunet (including 12 registries in Europe, Australia, and New Zealand) started in 1987. The variables of the different registries were harmonised over time, facilitating the merging of data. In 2015 a further collaboration was established with the North American SVS-VQI, creating the International Consortium of Vascular Registries. In a recent publication, 51,153 patients with AAA treated on three continents were analysed. This study showed a great variability in clinical practice of AAA repair, both between and at centre level within countries. In most clinical studies in vascular surgery, the greatest risk of statistical error is of type II, where there are not enough patients to show the differences. When working with “big data” from multiple registries, however, the risk of type I error becomes significant. The problems with missing data also increase, and may have to be addressed with multiple imputation rather than exclusion. The fact that different registries may have different definitions of their variables creates problems, although the 20 years of collaboration between the registries in the Vascunet has been instrumental in harmonising variables and definitions. It is important that every registry contributing data also participates in the intellectual process when the data is merged, analysed, and interpreted.

In summary, the strength of registry data is obvious, but limitations and pitfalls also exist. With such great potential for creating new knowledge, and being a driver of quality improvement, the authors believe that vascular registries have a bright future. If we do not create them ourselves, administrators will, and we lose control, and ultimately patients will suffer.