European Journal of Vascular & Endovascular Surgery
Volume 35, Issue 6 , Pages 633-636, June 2008

The Role of Simulation in Training Endovascular Interventions

  • D.A. Gould

      Affiliations

    • Department of Interventional Radiology, Royal Liverpool University Hospital, Prescot Street, Liverpool L7 8XP, England, United Kingdom
    • ,
  • J.A. Reekers

      Affiliations

    • Department of Radiology, AMC, University of Amsterdam, The Netherlands
    • Corresponding Author InformationCorresponding author. Prof. J.A. Reekers, Department of Radiology, AMC, University of Amsterdam, Meibergdreef 9, NL 1105 AZ Amsterdam-Zuidoost, The Netherlands.

Accepted 4 January 2008. published online 04 March 2008.

Article Outline

 

Psychomotor skills for tasks with an element of risk, such as those used to ride a bicycle, are best acquired in a low risk or risk free environment which allows the ascent of the learning curve to be conducted in safety. In this way, the essential skills to remain upright while pedalling and steering can be acquired through deliberate practice, becoming automated to allow attention to be directed to more complex tasks, such as the complexities of riding in traffic. Equally, if we have not cycled for a few years, we might wish to reacquaint ourselves with the essential fine motor skills required in a controlled environment. Yet although we know such skills exist, we would find it difficult to explain exactly what they might comprise!

Today, medical procedural skills are still usually learnt by practicing on patients in an apprenticeship. This also is the case for interventional radiology (IR), where it is just as important to avoid isolating the acquisition of motor skills from key knowledge and rules, in particular those that relate to the clinical condition of the patient. Therefore training is carried out in a broad, carefully structured curriculum, which also develops the behaviour and attitudes that are essential to professionalism.1 Basic procedural skills (cognitive, psychomotor) are the essential building blocks for more complex tasks. These ‘core’ elements include clinical, anatomical, imaging, therapeutic, as well as other cognitive, and the fine motor, skills to use touch and imaging to guide the manipulation of needles, wires and catheters. Hence these fundamental skills must be automated to avoid exceeding the learner's attention capacity when moving to more complex tasks in patients.2 Once acquired, these technical skills are maintained by regular practice, or may need to be refreshed by re-training.

The training and maintenance of core skills underpins IR procedural practice, yet a dearth of invasive diagnostic work in the wake of exponential developments in non-invasive imaging, has reduced the available opportunities for basic training. European, and other, working time regulations,3, 4 together with a schedule for modernizing medical careers, have further condensed the time available to train. In addition, it is a paradox that while we should ‘first do no harm’, an essential component of patient-centred learning is feedback on errors made. At the same time, apprenticeship training prolongs procedures and occupies expert mentors, resulting in more expensive patient management.5, 6 Complex procedures such as carotid stenting bring further problems: how is the trainee to learn without exposing the patient to risk? Further, how is the trainee to be assessed, to provide feedback to motivate learning and evidence for certification? As yet there is little in the way of objective assessment methods in routine use for IR skills. Therefore a pressing need for a reappraisal of our methods of training and assessing the high stakes skills of IR exists, with an emphasis on defining minimum standards for success, and implementing an alternative to patient-centred learning.

Possible alternatives include various simulations such as models, animals and computer based methods. Simple deformable models of anatomy have been used to train and assess in surgery,7 and these also can be punctured by needles under ultrasound guidance, or act as a conduit to train catheter and guide wire skills.8 Such models often are expensive, are ultimately destroyed by multiple needle punctures, and the materials used cannot be easily altered to change anatomy and ‘pathology’. Training also can use animals, which provide real world physiology and ‘feel’, although it is difficult to reproduce human pathology states in animals, and their anatomy is somewhat dissimilar to that of humans.9, 10 Animals also are expensive to maintain, and their use has raised political and ethical issues. Technology based simulation, on the other hand, introduces the potential to attain high levels of fidelity (accuracy) through use of an interface with a computer generated virtual environment.

The question of whether medical simulation facilitates learning was the subject of a systematic review, which showed some evidence of benefit when learners could perform repetitive practice using simulations and when educational feedback was provided.11 There also was a need for tasks to range in difficulty, and to be integrated with the curriculum. When criteria of functionality such as these are met, and in particular if the simulation, its content and fidelity, are appropriate, medical simulators would seem more likely to provide training and assessment of procedural skills, as well as rare complications, contrast reactions, and other medical emergencies. Training could thus become learner-centred, performed at the learner's pace, remotely from patients, with a new opportunity to learn safely from mistakes.12 Hence a pressing need to implement an alternative to patient centred learning, to define minimum standards for success, and to train and acquire new skills, has driven an explosion of interest in medical simulation. As the great promise of ‘virtual reality’ is embraced by specialities and industry, it is important to examine the evidence, to define its validity and its developing role.

In terms of the effectiveness or validity of a simulation to train a procedure, only that part of the simulation being used for the training needs to be validated. Face validity exists where the simulation appears to a trainee to resemble the real world task. Content validity is determined by subject experts who attest as to whether the simulation accurately replicates the procedure or process it claims to model. For assessment purposes, content validity should also confirm whether the metrics used are relevant to correct performance of the target procedure: construct validity determines whether the simulator can actually use these metrics. Concurrent validity compares the assessment with a gold standard, such as with the performance of experts. Concurrent validity reflects performance at the time of testing, whereas predictive validity predicts future competence in patients as confirmed in a subsequent clinical study. Ultimately there should be proof that the skills acquired by simulator training transfer to procedures performed in patients (transfer of training), and are then maintained over time.

An evaluation of experts and novices using endovascular simulation as a training aid for carotid artery stenting reported a significant improvement in performance of both novices and experts after a period of training for 30–60min, the novices rapidly achieving scores close to those of the experts.13 The time to complete the procedure was the only significant discriminator between novices and experts: while a surrogate measure of technical ability, time alone is clearly an imprecise discriminator of quality. A further study of endovascular simulator training demonstrated significant improvement in an observer's subjective scoring of simulated catheter manipulation, as well as in objective metrics of time to completion, fluoroscopy time and contrast dose.14 Observer based scores included correctness of procedure sequencing, though without weighting there is a lack of discrimination between trivial and critical procedural tasks, introducing uncertainty over claims of construct validity.

Operator skills common to both iliac and renal simulations have been identified in the Mentice VIST-VR, with a proposal of the existence of separate core skills for renal cannulation in the simulation.15 As yet, the relevance of such observations in a simulator to real world interventional performance, remains unclear. Studies of transfer of training skills are of fundamental importance in establishing this relevance of simulator training. One such study randomised subjects to either no training or simulator training with mentorship, and demonstrated skills transfer to procedures in patients in the latter group.16 An absence of equivalent mentorship in the non-simulator trained group does not provide a control for the powerful training, particularly for cognitive skills, provided by mentorship, and possibly accounting for at least some of the observed effect.

The CIRSE and SIR simulation task forces have pointed out the limited evidence which supports use of contemporary simulations for the acquisition of fine motor skills, and that experience on a simulator cannot yet be regarded as equivalent to training involving performance of actual endovascular procedures in patients.17 They do however confirm the suitability of using these devices within a defined, mentored curriculum to train relevant cognitive and knowledge elements, and aspects of procedural experience such as learning the correct sequence of procedural steps and selection of appropriate tools. Indeed, many medical errors result from incorrect procedural sequencing and such training seems likely to be beneficial prior to performing procedures on patients.

Hence the successful demonstration of clinical benefit using computer based simulator models to train skills in laparoscopy, colonoscopy and anaesthetics18, 19, 20 has yet to be convincingly reproduced for endovascular simulations, where interactions with tissues are occurring via long and flexible instruments. More faithful reproduction of the subtle, real world visual and tactile, cues and actions of the operator may be required to reliably train lower level motor skills that are important to success and patient safety in IR. Indeed, in a range of applications, workers are exploring the physical properties of tissues in order to better define and quantify instrument-tissue interactions and hence the requirements of the human-computer interface.21, 22, 23, 24, 25, 26, 27, 28, 29 There can be little doubt that in the longer term, computer based simulation has the potential to reproduce relevant aspects of the real world task, to train motor skills that today still require the fidelity of a ‘real patient’ environment. To attain this potential, however, also requires attention to the human factors involved in procedural medicine.

Without a clear understanding of the skills used by an expert cyclist to stay upright it would indeed be difficult to embark on the development of a simulator to train and assess such skills. In order for medical simulators to realize their full potential, their content and metrics need to be drawn from a breakdown of actual procedures performed in the overarching curriculum,30, 31 and simulators need to be able to use the chosen metrics. While there is a developing impetus to use simulator models to train interventional skills,32, 33 documentation of simulator development is often incomplete, lacking an identifiable source of the content and metrics used, and with no reference to specific training curricula. Hence, when considering whether a simulator is suitable for training and assessment in a curriculum it is important to understand how and by whom the test items were developed, and whether the metrics used are appropriate to the desired training objective.

Transparency of simulator development therefore should be an essential requirement, particularly if there are aspirations to use this technology for credentialing. Training organizations should set a minimum level of documentation of a simulation's development process and validation. Specific curricular training objectives that are already met by current simulations should be identified, and subject experts should review curricula to identify those tasks that require development of new simulations. This might include access to fundamental skills training in patients becoming unavailable or limited, or because the training task is high risk, or because training about infrequent or dangerous adverse events is desired.34 These tasks should then be analysed to identify the skills used by experts, as well as relevant metrics and an estimation of the fidelity required.

Extensive collaborative efforts are required to address the fidelity and human factors issues required for training low level motor skills, revisiting simulator specification, and aligning task definitions and metrics to target curricula. This brings a great opportunity for specialities with similar or parallel training objectives to work, together and with industry, to identify congruent areas for simulation within their curricula. Hence the work of simulator manufacturers could become more uniform and relevant across specialty borders. The best use of such simulations will be to provide training and assessment of proficiency within the wider curricula of the certifying organisations, including the core skills, knowledge and attitudes that are essential to realising the ultimate benefit: that of patient safety.

Back to Article Outline

References 

  1. Structured training in clinical radiology. 4th ed.. London: Education Board of the Faculty of Clinical Radiology, the Royal College of Radiologists; September 2004;
  2. Dankeman J, Chmarra MK, Verdaasdonk EGG, Stassen LPS, Grimbergen CA. Fundamental aspects of learning minimally invasive surgical skills – review. Minim Invasive Ther Allied Technol. 2005;14:247–256
  3. IDS working time. European Working Time Directive: the full text of the directive. <http://www.incomesdata.co.uk/information/worktimedirective.htm> [accessed 04.05.06].
  4. Department of Trade and Industry Working Time Regulations. <http://www.dti.gov.uk/employment/employment-legislation/employment-guidance/page14232.html> [accessed 04.05.06].
  5. Bridges M, Diamond DL. The financial impact of training surgical residents in the operating room. Am J Surg. 1999;177:28–32
  6. Crofts TJ, Griffiths JM, Sharma S, Wygrala J, Aitken RJ. Surgical training: an objective assessment of recent changes for a single health board. BMJ. Mar 1997;314:891
  7. Brehmer M, Tolley DA. Validation of a bench model for endoscopic surgery in the upper urinary tract. Eur Urol. 2002;42:175–180
  8. Chong CK, How TV, Black RA, Shortland AP, Harris PL. Development of a simulator for endovascular repair of abdominal aortic aneurysms. Ann Biomed Eng. 1998;26:798–802
  9. Lunderquist A, Ivancev K, Wallace S, Enge I, Laerum F, Kolbenstvedt AN. The acquisition of skills in interventional radiology by supervised training on animal models: a three year multicentre experience. Cardiovasc Intervent Radiol. 1995;18:209–211
  10. Dondelinger RF, Ghysels MP, Brisbois D, Donkers E, Snaps FR, Saunders L, et al. Relevant radiological anatomy of the pig as a training model in interventional radiology. Eur Radiol. 1998;8:1254–1273
  11. Issenberg SB, McGaghie WC, Petrusa ER, Lee Cordon D, Scalese RJ. Features and uses of high fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10–28
  12. Dawson S. Procedural simulation: a primer. J Vasc Interv Radiol. Feb 2006;17(2 Pt 1):205–213
  13. Hsu JH, Younan D, Pandalai S, Gillespie BT, Jain RA, Schippert DW, et al. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. J Vasc Surg. 2004;40:1118–1124
  14. Dayal R, Faries PL, Lin SC, Bernheim J, Hollenbeck S, DeRubertis B, et al. Computer simulation as a component of catheter based training. J Vasc Surg. 2004;40:1112–1117
  15. Neequaye SK, Aggarwal R, Brightwell R, Van Herzeele I, Darzi A, Cheshire NJ. Identification of skills common to renal and iliac endovascular procedures performed on a virtual reality simulator. [Journal Article. Randomized Controlled Trial] Eur J Vasc Endovasc Surg. 2007 May;33(5):525–532
  16. Chaer RA, DeRubertis BG, Lin SC, Bush HL, Karowski JK, Birk D, et al. Simulation improves resident performance in catheter based intervention. Ann Surg. 2006;244(3):343–349
  17. Gould DA, Reekers JA, Kessel DO, Chalmers NC, Sapoval M, Patel AA, et al. Simulation devices in interventional radiology: validation pending. J Vasc Interv Radiol. 2006;17:215–216
  18. Seymour NE, Gallagher AG, Roman SA, O`Brein MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomised, double-blinded study. Yale University & Queen's University, Belfast Ann Surg. 2002;236:458–463[discussion 463–4]
  19. Sedlack R, Kolars J. Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol. 2004;99:33–37
  20. Rowe R, Cohen R. An evaluation of a virtual reality airway simulator. Anesth Analg. 2002;95:62–66
  21. DiMaio SP, Salcudean SE. Interactive simulation of needle insertion models. IEEE Trans Biomed Eng. July 2005;1167–1179
  22. Alterovitz R, Pouliot J, Taschereau R. Simulating needle insertion and radioactive seed implantation for prostate brachytherapy. In:  Westwood JD,  Haluck RS,  Hoffman HM,  Mogel GT,  Phillips R,  Robb RA editor. Medicine meets virtual reality 11. IOS press; 2003;p. 19–25
  23. Kataoka H, Toshikatsu W, Kiyoyuki C. Measurement of the tip and friction force acting on a needle during penetration. In: Medical image computing and computer-assisted intervention MICCAI 2002. Tokyo, Japan; September 2002. pp. 216–223.
  24. O'leary MD, Simone C, Washio T. Robotic needle insertion: effects of friction and needle geometry. In: Proceedings of the 2003 IEEE international conference on robotics and automation. Taipei, Taiwan; 2003. p. 1774–1780.
  25. Chanthasopeephan T, Desai J, Lau ACW. Study of soft tissue cutting forces and cutting speeds. In:  Westwood JD,  Haluck RS,  Hoffman HM,  Mogel GT,  Phillips R,  Robb RA editor. Medicine meets virtual reality 11. IOS press; 2004;p. 56–62
  26. Ottensmeyer MP. TeMPeST 1-D. An instrument for measuring solid organ soft tissue properties. Exp Tech. May/June 2002;26:48–50
  27. Brouwer I, Ustin J, Bentley L, Sherman A, Dhruv N, Tendick F, et al. Measuring in vivo animal soft tissue properties for haptic modelling in surgical simulation. In:  Westwood JD,  Haluck RS,  Hoffman HM,  Mogel GT,  Phillips R,  Robb RA editor. Medicine meets virtual reality 2001. IOS Press; 2001;p. 69–74
  28. Healey AE, Evans JC, Murphy MG. In vivo force during arterial interventional radiology needle puncture procedures. In:  Westwood JD,  Haluck RS,  Hoffman HM,  Mogel GT,  Phillips R,  Robb RA editor. Medicine meets virtual reality 13. IOS Press; 2005;p. 178–184
  29. Chami G, Ward JW, Wills DPM, Phillips R, Sherman KP. Smart tool for force measurements during knee arthroscopy: in vivo human study. In: Proceedings of medicine meets virtual reality 14. Long Beach; January 2006. pp. 85–9.
  30. Gould DA, Healey AE, Johnson SJ, Lewandowski WE, Kessel DO. Metrics for an interventional radiology curriculum: a case for standardisation?. Stud Health Technol Inform. 2006;119:159–164
  31. Johnson SJ, Healey AE, Evans JC, Murphy MG, Crawshaw M, Gould DA. Physical and cognitive task analysis in interventional radiology. Clin Radiol. January 2006;61(1):97–103
  32. Medical Simulation Corporation, SimSuite Centres. <http://www.medsimulation.com/education_system/centers.asp> [accessed 04.05.06].
  33. Virtual Medical Worlds monthly news service for the virtual medical community. <http://www.hoise.com/vmw/02/articles/vmw/LV-VM-04-02-22.html> [accessed 04.05.06].
  34. Gould DA, Becker GJ, Kessel DO, Lewandowski WE, Patel AA. JITF; 15.03.07. Establishing the role of simulation-based training in current interventional curricula. <http://www.cirse.org/files/File/Curricular_insertion_sites_for_simulation.pdf> [accessed 09.06.07].

PII: S1078-5884(08)00049-X

doi:10.1016/j.ejvs.2008.01.008

European Journal of Vascular & Endovascular Surgery
Volume 35, Issue 6 , Pages 633-636, June 2008

Article Tools