An approach to value-based simulator selection: The creation and evaluation of the simulator value index tool

Published:January 19, 2018DOI:



      Currently there is no reliable, standardized mechanism to support health care professionals during the evaluation of and procurement processes for simulators. A tool founded on best practices could facilitate simulator purchase processes.


      In a 3-phase process, we identified top factors considered during the simulator purchase process through expert consensus (n = 127), created the Simulator Value Index (SVI) tool, evaluated targeted validity evidence, and evaluated the practical value of this SVI. A web-based survey was sent to simulation professionals. Participants (n = 79) used the SVI and provided feedback. We evaluated the practical value of 4 tool variations by calculating their sensitivity to predict a preferred simulator.


      Seventeen top factors were identified and ranked. The top 2 were technical stability/reliability of the simulator and customer service, with no practical differences in rank across institution or stakeholder role. Full SVI variations predicted successfully the preferred simulator with good (87%) sensitivity, whereas the sensitivity of variations in cost and customer service and cost and technical stability decreased (≤54%). The majority (73%) of participants agreed that the SVI was helpful at guiding simulator purchase decisions, and 88% agreed the SVI tool would help facilitate discussion with peers and leadership.


      Our findings indicate the SVI supports the process of simulator purchase using a standardized framework. Sensitivity of the tool improved when factors extend beyond traditionally targeted factors. We propose the tool will facilitate discussion amongst simulation professionals dealing with simulation, provide essential information for finance and procurement professionals, and improve the long-term value of simulation solutions. Limitations and application of the tool are discussed.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Britt R.C.
        • Novosel T.J.
        • Britt L.D.
        • Sullivan M.
        The impact of central line simulation before the ICU experience.
        Am J Surg. 2009; 197: 533-536
        • Dayan A.B.
        • Ziv A.
        • Berkenstadt H.
        • Munz Y.
        A simple, low-cost platform for basic laparoscopic skills training.
        Surg Innov. 2008; 15: 136-142
        • Conroy S.
        Competence and retention in performance of the lumbar puncture procedure in a task trainer model.
        Simul Healthc. 2010; 5: 133-138
        • Parent R.J.
        • Plerhoples T.A.
        • Long E.E.
        • et al.
        Early, intermediate, and late effects of a surgical skills “boot camp” on an objective structured assessment of technical skills: a randomized controlled study.
        J Am Coll Surg. 2010; 10: 984-989
        • Miller G.
        The magical number seven, plus or minus two: some limits on our capacity for processing information.
        Psychol Rev. 1956; 63: 81-97
        • Shumaila Y.
        • Foxall G.R.
        • Pallister J.G.
        Technology acceptance: a meta-analysis of the TAM: part 1.
        J Model Manag. 2007; 2: 251-280
        • Shumaila Y.
        • Foxall G.R.
        • Pallister J.G.
        Technology acceptance: a meta-analysis of the TAM: part 2.
        J Model Manag. 2007; 2: 281-304
        • Dalkey N.
        • Helme O.
        An experimental application of the Delphi method to the use of experts.
        Manag Sci. 1963; 9: 458-467
        • Brown B.
        Delphi Process: A Methodology Used for the Elicitation of Opinions of Experts, P-3925.
        Rand Co, Santa Monica (CA)1968
        • Rasch G.
        Probabilistic Models for Some Intelligence and Attainment Tests (Copenhagen, Danish Institute for Educational Research).
        The University of Chicago Press, Chicago (IL)1960/1980
        • Andrich D.
        Rasch Models for Measurement.
        SAGE Publications, Beverly Hills (CA)1988
        • Sweet R.M.
        • Hananel D.
        • Lawrenz F.
        A unified approach to validation, reliability, and education study design for surgical technical skills training.
        Arch Surg. 2010; 145: 197-201
        • Cook D.A.
        • Brydges R.
        • Zendejas B.
        • Hamstra S.J.
        • Hatala R.
        Technology-enhanced simulation to assess health professionals: a systematic review of validity evidence, research methods, and reporting quality.
        Acad Med. 2013; 88: 872-883
        • American College of Surgeons
        Resources for Accredited Educational Institutes (AEIs) [Internet].
        (Chicago: American College of Surgeons)
        Date: 2017
        Date accessed: December 18, 2017