Advertisement

A Composite Measure for Quality of Care in Patients with Symptomatic Carotid Stenosis Using Textbook Outcome

Open ArchivePublished:July 27, 2020DOI:https://doi.org/10.1016/j.ejvs.2020.06.012

      Objective

      Composite measures may better objectify hospital performance than individual outcome measures (IOM). Textbook outcome (TO) is an outcome measure achieved for an individual patient when all undesirable outcomes are absent. The aim of this study was to assess TO as an additional outcome measure to evaluate quality of care in symptomatic patients treated by carotid endarterectomy (CEA).

      Methods

      All symptomatic patients treated by CEA in 2018, registered in the Dutch Audit for Carotid Interventions, were included. TO was defined as a composite of the absence of 30 day mortality, neurological events (any stroke or transient ischaemic attack [TIA]), cranial nerve deficit, haemorrhage, 30 day readmission, prolonged length of stay (LOS; > 5 days) and any other surgical complication. Multivariable logistic regression was used to identify covariables associated with achieving TO, which were used for casemix adjustment for hospital comparison. For each hospital, an observed vs. expected number of events ratio (O/E ratio) was calculated and plotted in a funnel plot with 95% control limits.

      Results

      In total, 70.7% of patients had a desired outcome within 30 days after CEA and therefore achieved TO. Prolonged LOS was the most common parameter (85%) and mortality the least common (1.1%) for not achieving TO. Covariates associated with achieving TO were younger age, the absence of pulmonary comorbidity, higher haemoglobin levels, and TIA as index event. In the case mix adjusted funnel plot, the O/E ratios between hospitals ranged between 0.63 and 1.27, with two hospitals revealing a statistically significantly lower rate of TO (with O/E ratios of 0.63 and 0.66).

      Conclusion

      In the Netherlands, most patients treated by CEA achieve TO. Variation between hospitals in achieving TO might imply differences in performance. TO may be used as an additive to the pre-existing IOM, especially in surgical care with low baseline risk such as CEA.

      Keywords

      The peri-operative outcome of carotid endarterectomy is still mostly reported as a composite end point of combined ipsilateral stroke and death rate, both at individual patient level and at hospital level. This paper shows that textbook outcome, a composite measure achieved for an individual patient when all undesirable outcomes are absent, could be added to individual outcome measures to better evaluate hospital performance, especially in surgical interventions with low baseline risk such as carotid interventions.

      Introduction

      The main focus when evaluating the quality of surgical care is outcome measures.
      • Cooperberg M.R.
      • Birkmeyer J.D.
      • Litwin M.S.
      Defining high quality health care.
      Simply measuring these outcome measures could lead to improvement in quality of care.
      • Khuri S.F.
      • Daley J.
      • Henderson W.G.
      The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs.
      These individual outcome measures (IOM) are undisputedly meaningful, as they refer directly to the focus of improvement opportunity. However, they do not reflect the whole process of surgical care and therefore do not assess overall hospital performance. Additionally, the incidence of IOM in low risk surgical care, such as the treatment of patients with carotid artery stenosis by carotid endarterectomy (CEA), is low with procedural stroke and death rates of <5%. Therefore, IOM are unsuitable to assess significant changes over time or differences in complication rates between hospitals. Composite measures, consisting of multiple IOMs, for instance stroke and/or death, are therefore widely used tools to assess improvement in outcome in carotid surgery trials.
      • Naylor A.R.
      • Ricco J.B.
      • de Borst G.J.
      • Debus S.
      • de Haro J.
      • Halliday A.
      • et al.
      Editor's choice – management of atherosclerotic carotid and vertebral artery disease: 2017 clinical practice guidelines of the European Society for Vascular Surgery (ESVS).
      In the Netherlands, carotid surgery care quality is evaluated using a mandatory, nationwide, population based registry, the Dutch Audit for Carotid Interventions (DACI), in which all consecutive patients undergoing carotid revascularisastion are registered. The first DACI analysis did not reveal much variation between hospitals for IOM, implying that the quality of carotid surgery care in the Netherlands is generally good.
      • Karthaus E.G.
      • Vahl A.
      • Kuhrij L.S.
      • Elsman B.H.P.
      • Geelkerken R.H.
      • Wouters M.W.J.M.
      • et al.
      The Dutch Audit of Carotid Interventions: transparency in quality of carotid endarterectomy in symptomatic patients in The Netherlands.
      Theoretically, composite measures can better explain the variation between hospitals in overall performance.
      • Dimick J.B.
      • Staiger D.O.
      • Osborne N.H.
      • Nicholas L.H.
      • Birkmeyer J.D.
      Composite measures for rating hospital quality of major surgery.
      One of these composite measures as described in other surgical fields is textbook outcome (TO).
      • Busweiler L.A.D.
      • Schouwenburg M.G.
      • van Berge Henegouwen M.I.
      • Kolfschoten N.E.
      • de Jong P.C.
      • Rozema T.
      • et al.
      Textbook outcome as a composite measure in oesophagogastic cancer surgery.
      • Karthaus E.G.
      • Lijftogt N.
      • Busweiler L.A.D.
      • Elsman B.H.P.
      • Wouters M.W.J.M.
      • Vahl A.C.
      • et al.
      Textbook outcome: a composite measure for quality of elective aneurysm surgery.
      • Poelemeijer Y.Q.M.
      • Marang-van de Mheen P.J.
      • Wouters M.W.J.M.
      • Nienhuijs S.W.
      • Liem R.S.L.
      Textbook outcome: an ordered composite measure for quality of bariatric surgery.
      • Van Roessel S.
      • Mackay T.M.
      • van Dieren S.
      • van der Schelling G.P.
      • Nieuwenhuijs V.B.
      • Bosscha K.
      • et al.
      Textbook outcome: nationwide analysis of a novel quality measure in pancreatic surgery.
      • Kolfschoten N.E.
      • Kievit J.
      • Gooiker G.A.
      • van Leersum N.J.
      • Snijders H.S.
      • Eddes E.H.
      • et al.
      Focusing on desired outcomes of care after colon cancer resections: hospital variations in ‘textbook outcome’.
      TO is achieved for an individual patient when all undesirable outcomes, such as complications, are absent. A difference in TO between hospitals implies a variation in quality of care provided.
      The aim of this study was to assess the value of TO as an additional outcome measure to evaluate quality of care in symptomatic patients treated by CEA.

      Materials and Methods

      Patient selection

      All patients with symptomatic moderate to severe carotid stenosis undergoing CEA in 2018 that were registered in the DACI were included in this study.
      • Karthaus E.G.
      • Vahl A.
      • Kuhrij L.S.
      • Elsman B.H.P.
      • Geelkerken R.H.
      • Wouters M.W.J.M.
      • et al.
      The Dutch Audit of Carotid Interventions: transparency in quality of carotid endarterectomy in symptomatic patients in The Netherlands.
      The DACI is managed by the Dutch Society for Vascular Surgery (DSVS), a professional national association for vascular surgeons, and facilitated by the Dutch Institute for Clinical Auditing. Patients undergoing carotid artery stenting or asymptomatic patients were excluded from this analysis as these differ in outcome. As treatment of asymptomatic patients and stenting in general rarely occur in the Netherlands, these subgroups are very small. The minimum data requirements were type of surgery, presence of index event, and the date of CEA (used to determine the year of inclusion). No ethical approval or informed consent was required for this study under Dutch law.

      Definition of textbook outcome and its parameters

      The definition of TO was determined by the scientific committee of the DACI, which is mandated by the DSVS. It comprises the absence of all the following IOM, as registered by the DACI: no mortality during admission and/or within 30 days after CEA, no post-operative neurological events within 30 days, no post-operative cranial nerve deficit, no post-operative haemorrhage, no readmission within 30 days after discharge, no prolonged length of hospital stay (LOS) and no other post-operative surgical complications. TO was calculated as the number of patients who achieve TO (as a binary outcome) divided by the total number of patients included. If data were missing for one of the parameters of the TO, TO could not be achieved.
      Post-operative neurological events were defined as either transient ischaemic attack (TIA), cerebral haemorrhage and/or ischaemic stroke, both ipsilateral and contralateral. Prolonged LOS was specified as a hospital admission >5 days, counting from the day of CEA. This cutoff point was chosen by the scientific committee of the DACI, consisting of vascular surgeons mandated by the DSVS. Non-surgical complications (e.g., urinary tract infections or pneumonias) were not part of the definition of the TO as non-surgical complications are potentially interpreted differently by hospitals and at risk of over or under reporting. Additionally, it is believed that in the absence of surgical complications, prolonged LOS is a better proxy for non-surgical complications. As severe non-surgical complications will lead to a prolonged LOS, TO will not be achieved in these patients.

      Other definitions

      Pulmonary comorbidity was scored if there was shortness of breath at rest or while moving due to pulmonary disease, and/or visible consolidations and/or signs of fibrosis on chest imaging in the medical history. Cardiac comorbidity was defined as presence of hypertension, angina, cardiomegaly, cardiomyopathy, or abnormalities on ECG (e.g., atrial fibrillation) in the medical history.

      Statistical analysis

      All statistical analyses were performed with R software (version 1.1.383). Continuous values were expressed as mean ± standard deviation (SD) and nominal variables as count and percentages. The chi square test was used to compare categorical data for (not) achieving TO. The Mann–Whitney U test was used for comparison of the normally distributed continuous variables.
      Logistic regression analyses were performed to identify patient characteristics that are predictors of achieving TO. The covariables used were age (in years, continuous), sex, pulmonary comorbidity, cardiac comorbidity, any previous CEA, pre-operative haemoglobin levels (mmol/L, continuous), creatinine levels (μmol/L, continuous), systolic blood pressure (mmHg, continuous), and type of index event (TIA or stroke). After univariable analysis, covariables with a p value of < .10 were included for the multivariable regression model. The discrimination of the regression model was tested using a receiver operator curve (ROC) to estimate the area under the ROC (AUC). Covariables independently associated with TO in multivariable analysis were considered relevant case mix factors and were therefore used for hospital comparison.
      To explore the association between volume and achieving TO, hospitals were categorised by volume into tertiles: hospitals with <35 CEAs were classified as low volume, hospitals with 35–53 CEAs were classified as medium volume, and hospitals with >53 CEAs as high volume. The chi square test was used to compare achieving TO at patient level in the different volume categories.
      Additionally, hospitals achieving TO were compared using funnel plot with 95% confidence intervals (CIs). The funnel plot was adjusted for the case mix factors identified by the multivariable analysis. On the y axis, the actual observed number of events are divided by the expected number of events (O/E ratio), estimated using the case mix factors, resulting in an O/E ratio for each hospital. On the x axis, the expected number of events are displayed. If the ratio is > 1 (illustrated by the dotted line), the hospital has more patients with TO than would be expected. If the ratio is < 1, the hospital has fewer patients with TO than would be expected and therefore performs worse.
      Lastly, as the cutoff point of the prolonged LOS was chosen by the scientific committee of the DACI, a sensitivity analysis was performed to investigate whether changes in the cutoff point would result in different outliers in TO.

      Results

      In total, 2 528 patients were registered in 2018 in the DACI. After the exclusion of asymptomatic patients (n = 80, 3.2%) and patients treated by CAS (n = 79; 3.1%), 2 374 patients from 52 hospitals were included in the study (Fig. 1).
      Figure 1
      Figure 1Flowchart of inclusion and exclusion criteria of patients treated surgically for carotid artery stenosis to assess textbook outcome, determined by the Dutch Institute for Clinical Auditing, as an additional outcome measure to evaluate quality of care in symptomatic patients treated by carotid endarterectomy.

      Rates of complications and textbook outcome

      The complication rate per IOM and for stepwise addition of parameters for TO are shown in Table 1. TO was achieved in 1 679 patients (70.7%). Mortality within 30 days after CEA was the least frequent parameter (1.1%). Prolonged LOS was the most frequent parameter (15%), followed by readmission (10.4%). However, readmission had the highest number of missing values (6.3%) with missing rates ranging from 0% up to 91% per hospital. All other parameters had missing values below 2%.
      Table 1Complication rates after 2 374 carotid endarterectomies (CEAs) per indicator and stepwise addition of each parameter within textbook outcome (TO), determined by Dutch Institute for Clinical Auditing, as an additional outcome measure to evaluate quality of care in symptomatic patients for carotid artery stenosis
      IndicatorPatients treated by CEA (n = 2 374)
      Individual complicationsBuild up TO
      No mortality2 347 (98.9)2 347 (98.9)
      No post-operative neurological event2 313 (97.4)2 294 (96.6)
      No post-operative cranial nerve deficit2 332 (98.2)2 254 (94.9)
      No post-operative haemorrhage2 290 (96.5)2 177 (91.7)
      No readmission2 128 (89.6)1 967 (82.9)
      No prolonged LOS2 018 (85.0)1 697 (71.5)
      No other post-operative surgical complication2 330 (98.1)1 679 (70.7)
      TO1 679 (70.7)
      Data are n (%). LOS = length of stay; TO = textbook outcome.

      Textbook outcome on the patient level

      Patient characteristics are shown in Table 2. Mean age was 73 ± 9 years and 70% were male. Most patients had cardiac comorbidity (75%) and 22% had a pulmonary comorbidity. One hundred and ten patients had previously undergone CEA (either ipsilateral, contralateral, or both). Patients more often had a TIA as index event than an ischaemic stroke.
      Table 2Characteristics of 2 374 carotid endarterectomy (CEA) patients, categorised by achieving (n = 1 679; 70.7%) or not (n = 695; 29.3%) textbook outcome (TO), determined by the Dutch Institute for Clinical Auditing, as an additional outcome measure to evaluate quality of care after CEA for symptomatic carotid artery stenosis. Odds ratios (OR) with 95% confidence intervals (CIs) are reported for achieving TO from the multivariable analysis
      Patient characteristicsTotal (n = 2374)Patients without TO (n = 695)Patients with TO (n = 1679)OR (95% CI)p value
      p < .05 considered statistically significant.
      Age — years73 ± 9.074 ± 9.072 ± 8.90.97 (0.96–0.99)<.001
      Male sex1 657 (69.8)478 (68.8)1 179 (70.2).55
      Pulmonary comorbidity0.76 (0.60–0.95).017
       No1 772 (74.6)491 (70.6)1 281 (76.3)
       Yes513 (21.6)173 (24.9)340 (20.2)
       Unknown89 (3.7)31 (4.5)58 (3.5)
      Cardiac comorbidity0.94 (0.73–1.21).64
       No504 (21.2)129 (18.6)375 (22.3)
       Yes1 825 (76.9)552 (79.4)1 273 (75.0)
       Unknown45 (1.9)14 (2.0)31 (1.8)
      Previous CEA110 (4.6)35 (5.0)75 (4.5).62
      Haemoglobin — mmol/L8.6 ± 1.08.5 ± 1.18.7 ± 1.01.13 (1.02–1.25).017
      Creatinine — μmol/L93 ± 3694 ± 3692 ± 36.44
      Systolic blood pressure — mmHg146 ± 21147 ± 22146 ± 21.64
      Type of index event0.74 (0.61–0.90).003
       TIA1 261 (53.1)326 (46.9)935 (55.7)
       Stroke903 (38.0)284 (40.9)619 (36.9)
       Unknown210 (8.8)85 (12.2)125 (7.4)
      Data are presented as n (%) or mean ± standard deviation. TIA = transient ischaemic attack; TO = textbook outcome.
      p < .05 considered statistically significant.
      Younger age (odds ratio [OR] 0.97 per year, 95% CI 0.96–0.98), the absence of pulmonary comorbidity (OR 0.75, 95% CI 0.62–0.93), the absence of cardiac comorbidity (OR 0.79, 95% CI 0.63–0.99), higher haemoglobin levels (OR 1.19, 95% CI 1.09–1.29), and TIA as the index event (OR 0.75, 95% CI 0.63–0.92) were associated with achieving TO. After multivariable analysis, younger age (OR 0.97, 95% CI 0.96–0.99), absence of pulmonary comorbidity (OR 0.76, 95% CI 0.60–0.95), higher haemoglobin levels (OR 1.13, 95% CI 1.02–1.25), and TIA as the index event (OR 0.74, 95% CI 0.61–0.90) remained statistically significantly associated with achieving TO (Table 2). The AUC of the model was 0.60.

      Volume and textbook outcome

      The association between hospital volume where the patient was treated by CEA and achieving TO was also examined. Patients treated in middle volume hospitals (35–53 CEAs) had a higher odds of achieving TO than low volume hospitals (OR 1.37, 95% CI 1.08–1.76; p = .020). Patients treated at a high volume hospital had an even higher odds of achieving TO (OR 1.70, 95% CI 1.34–2.16; p < .001).

      Hospital level textbook outcome

      The case mix adjusted funnel plot for the comparison of TO between hospitals is shown in Fig. 2, using the case mix factors identified by the multivariable logistic regression analysis. The O/E ratios ranged between 0.63 to 1.27 between hospitals. None of the hospitals had a significantly higher rate of patients with TO. Two hospitals had significantly fewer patients with TO with an O/E ratio of 0.63 (hospital A) and 0.66 (hospital B), treating 44 and 53 patients, respectively. Individual hospital scores for these two hospitals on single outcome indicators and TO are shown in Table 3. For both hospitals, re-admission was rated highest, subsequently leading to a large decrease in the number of patients achieving TO. Of these re-admission rates, 61% and 91%, respectively, were caused by missing values and therefore TO was not achieved in these patients.
      Figure 2
      Figure 2Casemix adjusted funnel plot of between hospital variation in textbook outcome (TO), determined by Dutch Institute for Clinical Auditing, as an additional outcome measure to evaluate quality of care of patients after carotid endarterectomy (CEA) for symptomatic carotid artery stenosis. The dotted line indicates equal number of observed and expected events. The other two lines are the upper and lower control limits. Each diamond shape is a hospital. Hospitals A and B (red) are negative outliers.
      Table 3Complication rate per indicator and stepwise addition of each parameter within the textbook outcome (TO), determined by Dutch Institute for Clinical Auditing, as an additional outcome measure to evaluate quality of care of patients after carotid endarterectomy (CEA) for symptomatic carotid artery stenosis in the two hospitals A and B with significantly lower rates of patients with TO, as shown in Fig. 2
      IndicatorComplication rate after CEA — %
      Hospital AHospital B
      IndividualTOIndividualTO
      No mortality97.997.998.198.1
      No post-operative neurological event97.795.592.590.6
      No post-operative cranial nerve deficit97.795.510090.6
      No post-operative haemorrhage90.988.698.188.7
      No readmission59.152.258.554.7
      No prolonged LOS75.040.984.949.1
      No other post-operative surgical complication10040.998.149.1
      TO40.949.1
      LOS = length of stay; TO = textbook outcome.

      Prolonged length of stay and textbook outcome

      A sensitivity analysis was performed for the prolonged LOS for three, four, and six days instead of the five days chosen by the scientific committee of the DACI. When changing the cutoff point from five to three, four, and six days, TO was achieved in 56.3%, 65.8%, and 74.7% of all patients, respectively. At hospital level, the change in cutoff point from five to three days did change the distribution of hospitals in the funnel plot. However, the two hospitals that were statistically worse performers were still weak performers. Three other hospitals now seemed to be worse performers. In the sensitivity analysis for four and six days, the distribution of the hospitals in the funnel plot did not change from the original funnel plot.

      Discussion

      This study shows that TO as a composite outcome measure can provide additional information on the overall care provided in symptomatic patients with carotid stenosis treated by CEA. Additionally, when using TO more variation is shown between hospitals than IOM as the sample size in each hospital is too low and the baseline risk is too low to detect differences between hospitals when comparing on IOM.
      • Karthaus E.G.
      • Vahl A.
      • Kuhrij L.S.
      • Elsman B.H.P.
      • Geelkerken R.H.
      • Wouters M.W.J.M.
      • et al.
      The Dutch Audit of Carotid Interventions: transparency in quality of carotid endarterectomy in symptomatic patients in The Netherlands.
      ,
      • Dimick J.B.
      • Welch H.G.
      • Birkmeyer J.D.
      Surgical mortality as an indicator of hospital quality: the problem with small sample size.
      The percentage TO varied between hospitals with O/E ratios from 0.63 to 1.27, with two hospitals being negative outliers. Therefore, it can be of added value when evaluating hospital performance.
      In this study, using data from a mandatory nationwide registry, the TO was 70.7%. A prolonged LOS was the highest rating single parameter (85%). Prolonged LOS can also be caused by undesired outcomes other than those included in the TO definitions, such as infection, delirium, or hyperperfusion syndrome, which is why it was chosen to be added to the TO. Another study showed that patients with a prolonged LOS have higher one year mortality and therefore prolonged LOS is an important outcome measure.
      • Mehaffe J.H.
      • LaPar D.J.
      • Tracci M.C.
      • Cherry K.J.
      • Kern J.A.
      • Kron I.
      • et al.
      Modifiable factors leading to increased length of stay after carotid endarterectomy.
      As the cutoff point for a prolonged LOS was set at five days based on expert opinion, a sensitivity analysis was performed using a different cutoff point, i.e., three, four, and six days, to assess the impact on hospital comparison of TO. When choosing the cutoff point at four or six days, the distribution of the funnel plot was similar to the cutoff of five days used in this study. When using a cutoff point at three days, there was a change in outlier hospitals. This observation could be explained by the fact that in these hospitals more patients are treated for post-operative hypertension (which is not registered in the DACI), which can lead to hyperperfusion syndrome if not managed correctly or more patients who were on oral anticoagulants pre-operatively had to be reinstated to therapeutic levels. In these examples, the prolonged LOS is to be considered good quality of care. When having a LOS of four days or more, no such difference in distribution is seen and is therefore preferred as a cutoff point.
      Additionally, post-operative haemorrhage was one of the parameters in TO. However, this could also be a consequence of following the guidelines on post-operative dual antiplatelet therapy correctly.
      Another parameter with a high rating was re-admission (89.6%). Two hospitals had a high re-admission rate of 40.9% and 41.5%, with high missing rates. In the authors’ definition of TO, if one of the parameters of TO is missing, TO is not achieved. Therefore, high rates of missing values could imply that hospitals seem to perform significantly worse than they might do. A recent study on TO, combining multi-institutional international databases,
      • Merath K.
      • Chen Q.
      • Bagante F.
      • Alexandrescu S.
      • Marques H.P.
      • Aldrighetti L.
      • et al.
      A multi-institutional international analysis of textbook outcomes among patients undergoing curative-intent resection of intrahepatic cholangiocarcinoma.
      excluded patients with one or more missing parameters. However, it is believed that this could initiate reporting bias to a point that it leads to selection of the better patients. This would misinterpret the TO by neutralisation of the differences between the centres. To demonstrate this, the data were analysed while excluding patients with one or more missing values on the parameters of TO. In this case, the TO rate would be 75.6%. The corresponding funnel plot is shown in Fig. 3, revealing that all hospitals are within control limits, which could be due the neutralising effect as explained earlier. Moreover, the definition of TO, which involves treating missing values as if the parameter of TO has occurred, encourages hospitals to better register their complications.
      Figure 3
      Figure 3Casemix adjusted funnel plot of between hospital variation in textbook outcome (TO), determined by Dutch Institute for Clinical Auditing, as an additional outcome measure to evaluate quality of care of patients after carotid endarterectomy (CEA) for symptomatic carotid artery stenosis, when excluding patients with missing values in one or more parameters. The dotted line indicates equal number of observed and expected events. The other two lines are the upper and lower control limits. Each diamond shape is a hospital. Hospital A and B are the hospitals that are negative outliers in the case mix adjusted funnel plot in which missing values were noted as not achieving TO (as shown in ).
      The two hospitals that were negative outliers on the TO funnel plot were relatively lower to middle volume hospitals. At patient level, higher volume hospitals were associated with achieving TO. A recent systematic review showed that high volume hospitals have a lower risk of post-operative complications after CEA.
      • Poorthuis M.H.F.
      • Brand E.C.
      • Halliday A.
      • Bulbulia R.
      • Bots M.L.
      • de Borst G.J.
      High operator and hospital volume are associated with a decreased risk of death and stroke after carotid revascularization: a systematic review and meta-analysis.
      More research is needed to investigate whether centralisation can lead to fewer post-operative complications.
      Additionally, this study has identified case mix factors for fair hospital comparison: age, pulmonary comorbidity, haemoglobin level, and type of index event. The latter can be explained by the fact that patients with stroke are more prone to have a prolonged LOS than TIA patients as they have more severe disabilities. It was expected that previous CEA would be associated with not achieving TO, as these patients have a higher risk of re-intervention and/or wound complications (with resulting prolonged LOS). A possible explanation for this could be that the number of patients is too small to cause significant effect.
      While it has been shown that TO is a more reliable outcome measure to evaluate overall surgical process, IOM are still essential to identify which parts of the process are necessary to improve. Therefore, it must be seen as an addition to rather than a replacement of IOM.
      There are limitations to this study. Firstly, there were high numbers of missing values in the parameter “re-admission”. Because these missing values were not equally distributed between all hospitals, but were allocated to a specific number of centres, it had a large impact on the hospital TO comparison. The discriminatory power of the model, calculated as an AUC of the ROC, was weak. However, as this is a composite measure it is impractical to fit a model very well as different outcome measures have different risk factors.
      • Lane-Fall M.B.
      • Neumand M.D.
      Outcomes measures and risk adjustment.
      Another limitation is that each parameter is weighed similarly for TO. For example, mortality could be high for a particular hospital while other parameters are low; therefore, that hospital could have a good TO. It is advised that TO be used as an additional outcome measure and not a substitute for each IOM.
      Lastly, only patient characteristics and outcome measures that are registered in the DACI could be selected; therefore, selection bias on the potential case mix factors and the TO parameters chosen in this study is possible. For instance, diabetes is a risk factor for post-operative complications,
      • Ahari A.
      • Bergqvist D.
      • Troëng T.
      • Elfström J.
      • Hedberg B.
      • Ljungström K.G.
      • et al.
      Diabetes mellitus as a risk factor for early outcome after carotid endarterectomy - a population-based study.
      and post-operative myocardial infarction is an important complication to register.
      • Hobeanu C.
      • Lavallée P.C.
      • Rothwell P.M.
      • Sissani L.
      • Albers G.W.
      • Bornstein N.M.
      • et al.
      Symptomatic patients remain at substantial risk of arterial disease complications before and after endarterectomy or stenting.
      Both have been added to the DACI in 2019 and can therefore not be added to this current definition of TO. However, other potential confounding factors could have contributed to the results.

      Conclusion

      In the DACI, a nationwide mandatory prospective registry, the TO rate was 70.7% with a broad variation between hospitals. TO as a composite outcome measure for carotid surgery could be added to the IOM to better evaluate the overall process of quality of care among hospitals. To be of maximum value, registration of parameters, especially on re-admission should be optimised.

      Conflicts of Interest

      None.

      Funding

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      Acknowledgements

      The authors would like to thank all surgeons, registrars, physician assistants, and administrative nurses who registered all the patients in the Dutch Audit for Carotid Interventions (DACI), as well as the Dutch Audit for Carotid Interventions Group.
      COLLABORATORS
      The collaborators are the Steering Committee of the Dutch Audit for Carotid Interventions: Gert J. de Borst, Bernard H.P. Elsman, Jan-Willem Elshof, Robert H. Geelkerken, Jaap F. Hamming, Anco Vahl, and Martine C. Willems, as well as the Dutch Society of Vascular Surgery (see Appendix 1).

      Appendix 1.

      Van den Akker PJ, Akkersdijk GJ, Akkersdijk GP, Akkersdijk WL, van Andringa de Kempenaer MG, Arts CH, Avontuur JA, Bakker OJ, Balm R, Barendregt WB, Bekken JA, Bender MH, Bendermacher BL, van den Berg M, Berger P, Beuk RJ, Blankensteijn JD, Bleker RJ, Blok JJ, Bode AS, Bodegom ME, van der Bogt KE, Boll AP, Booster MH, Borger van der Burg BL, de Borst GJ, Bos-van Rossum WT, Bosma J, Botman JM, Bouwman LH, Brehm V, de Bruijn MT, de Bruin JL, Brummel P, van Brussel JP, Buijk SE, Buijs MA, Buimer MG, Burger DH, Buscher HC, Cancrinus E, Castenmiller PH, Cazander G, Coester AM, Cuypers PH, Daemen JH, Dawson I, Dierikx JE, Dijkstra ML, Diks J, Dinkelman MK, Dirven M, Dolmans DE, van Doorn RC, van Dortmont LM, Drouven JW, van der Eb MM, Eefting D, van Eijck GJ, Elshof JW, Elsman BH, van der Elst A, van Engeland MI, van Eps RG, Faber MJ, de Fijter WM, Fioole B, Fokkema TM, Frans FA, Fritschy WM, Fung Kon Jin PH, Geelkerken RH, van Gent WB, Glade GJ, Govaert B, Groenendijk RP, de Groot HG, van den Haak RF, de Haan EF, Hajer GF, Hamming JF, van Hattum ES, Hazenberg CE, Hedeman Joosten PP, Helleman JN, van der Hem LG, Hendriks JM, van Herwaarden JA, Heyligers JM, Hinnen JW, Hissink RJ, Ho GH, den Hoed PT, Hoedt MT, van Hoek F, Hoencamp R, Hoffmann WH, Hogendoorn W, Hoksbergen AW, Hollander EJ, Hommes M, Hopmans CJ, Huisman LC, Hulsebos RG, Huntjens KM, Idu MM, Jacobs MJ, van der Jagt MF, Jansbeken JR, Janssen RJ, Jiang HH, de Jong SC, Jongbloed-Winkel TA, Jongkind V, Kapma MR, Keller BP, Khodadade Jahrome A, Kievit JK, Klemm PL, Klinkert P, Koedam NA, Koelemaij MJ, Kolkert JL, Koning GG, Koning OH, Konings R, Krasznai AG, Krol RM, Kropman RH, Kruse RR, van der Laan L, van der Laan MJ, van Laanen JH, van Lammeren GW, Lamprou DA, Lardenoye JH, Lauret GJ, Leenders BJ, Legemate DA, Leijdekkers VJ, Lemson MS, Lensvelt MM, Lijkwan MA, Lind RC, van der Linden FT, Liqui Lung PF, Loos MJ, Loubert MC, van de Luijtgaarden KM, Mahmoud DE, Manshanden CG, Mattens EC, Meerwaldt R, Mees BM, von Meijenfeldt GC, Menting TP, Metz R, Minnee RC, de Mol van Otterloo JC, Molegraaf MJ, Montauban van Swijndregt YC, Morak MJ, van de Mortel RH, Mulder W, Nagesser SK, Naves CC, Nederhoed JH, Nevenzel-Putters AM, de Nie AJ, Nieuwenhuis DH, Nieuwenhuizen J, van Nieuwenhuizen RC, Nio D, Noyez VJ, Oomen AP, Oranen BI, Oskam J, Palamba HW, Peppelenbosch AG, van Petersen AS, Petri BJ, Pierie ME, Ploeg AJ, Pol RA, Ponfoort ED, Post IC, Poyck PP, Prent A, ten Raa S, Raymakers JT, Reichart M, Reichmann BL, Reijnen MM, de Ridder JA, Rijbroek A, van Rijn MJ, de Roo RA, Rouwet EV, Saleem BR, Salemans PB, van Sambeek MR, Samyn MG, van ’t Sant HP, van Schaik J, van Schaik PM, Scharn DM, Scheltinga MR, Schepers A, Schlejen PM, Schlosser FJ, Schol FP, Scholtes VP, Schouten O, Schreve MA, Schurink GW, Sikkink CJ, te Slaa A, Smeets HJ, Smeets L, Smeets RR, de Smet AA, Smit PC, Smits TM, Snoeijs MG, Sondakh AO, Speijers MJ, van der Steenhoven TJ, van Sterkenburg SM, Stigter DA, Stokmans RA, Strating RP, Stultiëns GN, Sybrandy JE, Teijink JA, Telgenkamp BJ, Teraa M, Testroote MJ, Tha-In T, The RM, Thijsse WJ, Thomassen I, Tielliu IF, van Tongeren RB, Toorop RJ, Tournoij E, Truijers M, Türkcan K, Tutein Nolthenius RP, Ünlü Ç, Vaes RH, Vafi AA, Vahl AC, Veen EJ, Veger HT, Veldman MG, Velthuis S, Verhagen HJ, Verhoeven BA, Vermeulen CF, Vermeulen EG, Vierhout BP, van der Vijver-Coppen RJ, Visser MJ, van der Vliet JA, Vlijmen - van Keulen CJ, Voorhoeve R, van der Vorst JR, Vos AW, de Vos B, Vos CG, Vos GA, Voute MT, Vriens BH, Vriens PW, de Vries AC, de Vries DK, de Vries JP, de Vries M, van der Waal C, Waasdorp EJ, Wallis de Vries BM, van Walraven LA, van Wanroij JL, Warlé MC, van de Water W, van Weel V, van Well AM, Welten GM, Welten RJ, Wever JJ, Wiersema AM, Wikkeling OR, Willaert WI, Wille J, Willems MC, Willigendael EM, Wilschut ED, Wisselink W, Witte ME, Wittens CH, Wong CY, Wouda R, Yazar O, Yeung KK, Zeebregts CJ, van Zeeland ML.

      References

        • Cooperberg M.R.
        • Birkmeyer J.D.
        • Litwin M.S.
        Defining high quality health care.
        Urol Oncol. 2009; 27: 411-416
        • Khuri S.F.
        • Daley J.
        • Henderson W.G.
        The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs.
        Arch Surg. 2002; 137: 20-27
        • Naylor A.R.
        • Ricco J.B.
        • de Borst G.J.
        • Debus S.
        • de Haro J.
        • Halliday A.
        • et al.
        Editor's choice – management of atherosclerotic carotid and vertebral artery disease: 2017 clinical practice guidelines of the European Society for Vascular Surgery (ESVS).
        Eur J Vasc Endovasc Surg. 2018; 55: 3-81
        • Karthaus E.G.
        • Vahl A.
        • Kuhrij L.S.
        • Elsman B.H.P.
        • Geelkerken R.H.
        • Wouters M.W.J.M.
        • et al.
        The Dutch Audit of Carotid Interventions: transparency in quality of carotid endarterectomy in symptomatic patients in The Netherlands.
        Eur J Vasc Endovasc Surg. 2018; 56: 476-485
        • Dimick J.B.
        • Staiger D.O.
        • Osborne N.H.
        • Nicholas L.H.
        • Birkmeyer J.D.
        Composite measures for rating hospital quality of major surgery.
        Health Serv Res. 2012; 47: 1861-1879
        • Busweiler L.A.D.
        • Schouwenburg M.G.
        • van Berge Henegouwen M.I.
        • Kolfschoten N.E.
        • de Jong P.C.
        • Rozema T.
        • et al.
        Textbook outcome as a composite measure in oesophagogastic cancer surgery.
        Eur J Surg Oncol. 2014; 40: S55
        • Karthaus E.G.
        • Lijftogt N.
        • Busweiler L.A.D.
        • Elsman B.H.P.
        • Wouters M.W.J.M.
        • Vahl A.C.
        • et al.
        Textbook outcome: a composite measure for quality of elective aneurysm surgery.
        Ann Surg. 2017; 266: 898-904
        • Poelemeijer Y.Q.M.
        • Marang-van de Mheen P.J.
        • Wouters M.W.J.M.
        • Nienhuijs S.W.
        • Liem R.S.L.
        Textbook outcome: an ordered composite measure for quality of bariatric surgery.
        Obes Surg. 2019; 29: 1287-1294
        • Van Roessel S.
        • Mackay T.M.
        • van Dieren S.
        • van der Schelling G.P.
        • Nieuwenhuijs V.B.
        • Bosscha K.
        • et al.
        Textbook outcome: nationwide analysis of a novel quality measure in pancreatic surgery.
        Ann Surg. 2020; 271: 155-162
        • Kolfschoten N.E.
        • Kievit J.
        • Gooiker G.A.
        • van Leersum N.J.
        • Snijders H.S.
        • Eddes E.H.
        • et al.
        Focusing on desired outcomes of care after colon cancer resections: hospital variations in ‘textbook outcome’.
        Eur J Surg Oncol. 2013; 39: 156-163
        • Dimick J.B.
        • Welch H.G.
        • Birkmeyer J.D.
        Surgical mortality as an indicator of hospital quality: the problem with small sample size.
        JAMA. 2004; 292: 847-851
        • Mehaffe J.H.
        • LaPar D.J.
        • Tracci M.C.
        • Cherry K.J.
        • Kern J.A.
        • Kron I.
        • et al.
        Modifiable factors leading to increased length of stay after carotid endarterectomy.
        Ann Vasc Surg. 2017; 39: 195-203
        • Merath K.
        • Chen Q.
        • Bagante F.
        • Alexandrescu S.
        • Marques H.P.
        • Aldrighetti L.
        • et al.
        A multi-institutional international analysis of textbook outcomes among patients undergoing curative-intent resection of intrahepatic cholangiocarcinoma.
        JAMA Surg. 2019; 154e190571
        • Poorthuis M.H.F.
        • Brand E.C.
        • Halliday A.
        • Bulbulia R.
        • Bots M.L.
        • de Borst G.J.
        High operator and hospital volume are associated with a decreased risk of death and stroke after carotid revascularization: a systematic review and meta-analysis.
        Ann Surg. 2019; 269: 631-641
        • Lane-Fall M.B.
        • Neumand M.D.
        Outcomes measures and risk adjustment.
        Int Anesthesiol Clin. 2013; 51: 10-21
        • Ahari A.
        • Bergqvist D.
        • Troëng T.
        • Elfström J.
        • Hedberg B.
        • Ljungström K.G.
        • et al.
        Diabetes mellitus as a risk factor for early outcome after carotid endarterectomy - a population-based study.
        Eur J Vasc Endovasc Surg. 1999; 18: 122-126
        • Hobeanu C.
        • Lavallée P.C.
        • Rothwell P.M.
        • Sissani L.
        • Albers G.W.
        • Bornstein N.M.
        • et al.
        Symptomatic patients remain at substantial risk of arterial disease complications before and after endarterectomy or stenting.
        Stroke. 2017; 48: 1005-1010

      Comments

      Commenting Guidelines

      To submit a comment for a journal article, please use the space above and note the following:

      • We will review submitted comments as soon as possible, striving for within two business days.
      • This forum is intended for constructive dialogue. Comments that are commercial or promotional in nature, pertain to specific medical cases, are not relevant to the article for which they have been submitted, or are otherwise inappropriate will not be posted.
      • We require that commenters identify themselves with names and affiliations.
      • Comments must be in compliance with our Terms & Conditions.
      • Comments are not peer-reviewed.