Advertisement

An immersive “simulation week” enhances clinical performance of incoming surgical interns improved performance persists at 6 months follow-up

Published:January 26, 2015DOI:https://doi.org/10.1016/j.surg.2014.09.024

      Background

      The transition from student to intern can be challenging. The “August” or “July effect” describes increased errors and reduced patient safety during this transition. The study objectives were to develop, pilot, and evaluate clinical performance after an immersive simulation course for incoming interns.

      Methods

      Graduating students were recruited for a 1-week immersive simulation course. Controls received no simulation training. Primary outcome (at baseline, and 1 and 6 months) was clinical performance on Objective Structured Clinical Examinations (OSCE) of clinical procedures and surgical technical skills. Secondary outcomes were self-reported confidence and clinical procedure logbook data.

      Results

      Nineteen students were recruited. Sixteen completed the 6-month follow-up, 10 in the intervention group and 6 in the control group. No differences were demonstrated between interventions and controls at baseline (OSCE [median, 66 vs 78; P = .181], technical skills [48 vs 52.5; P = .381], and confidence [101 vs 96; P = .368]). Interventions outperformed controls at 1 month (OSCE [111 vs 82; P = .001], technical skills [78.5 vs 63; P = .030], and confidence [142 vs 119; P < .001]), and 6 months (OSCE [107 vs 93; P = .007], technical skills [92.5 vs 69; P = .044], and confidence [148 vs 129; P = .022]). No differences were observed in numbers of clinical procedures performed at 1 (P = .958), 4 (P = .093), or 6 months (P = .713).

      Conclusion

      The immersive simulation course objectively improved subjects' clinical skills, technical skills, and confidence. Despite similar clinical experience as controls, the intervention group's improved performance persisted at 6 months follow-up. This feasible and effective intervention to ease transition from student to intern could reduce errors and enhance patient safety.
      The transition from medical student to surgical intern can be intimidating with consequences for patient safety as demonstrated by the “July” or “August” effect.
      • Young J.Q.
      • Ranji S.R.
      • Wachter R.M.
      • Lee C.M.
      • Niehaus B.
      • Auerbach A.D.
      “July effect”: impact of the academic year-end changeover on patient outcomes: a systematic review.
      • Jen M.H.
      • Bottle A.
      • Majeed A.
      • Bell D.
      • Aylin P.
      Early In-hospital mortality following trainee doctors' first day at work.
      Anecdotally, trainees have reported feeling ill-prepared and extremely nervous in their first couple of weeks as a surgical intern. Indeed, the prevalence of psychiatric morbidity and burnout has been shown to rise during the intern year.
      • Willcock S.M.
      • Daly M.G.
      • Tennant C.C.
      • Allard B.J.
      Burnout and psychiatric morbidity in new medical graduates.
      Traditionally, “shadowing” the current intern was used to ease the transition. This is a form of work experience, which may be referred to as a subinternship in the United States, where the incoming student follows the incumbent surgical intern to experience their daily duties. Since 2012, this practice has been made compulsory in the UK. The move was in part a response to the so-called August effect, whereby increased patient mortality was found to be associated with the August changeover and arrival of newly qualified doctors.
      • Jen M.H.
      • Bottle A.
      • Majeed A.
      • Bell D.
      • Aylin P.
      Early In-hospital mortality following trainee doctors' first day at work.
      The effect of changeover of junior doctors is referred to as the “July effect” in the United States. A recent review found evidence of increased mortality and reduced efficiency in hospitals owing to the changeover of doctors at the end of the year.
      • Young J.Q.
      • Ranji S.R.
      • Wachter R.M.
      • Lee C.M.
      • Niehaus B.
      • Auerbach A.D.
      “July effect”: impact of the academic year-end changeover on patient outcomes: a systematic review.
      Easing the transition from medical student to intern could potentially provide clinical benefits for patients in addition to making the experience less fear-inducing for the clinician. During shadowing, the exposure to clinical scenarios is largely based on chance occurrences. Moreover, there is a feeling among the profession that 4 days of shadowing is not enough on its own to protect patients.
      • House J.
      NHS shadowing scheme not enough to keep patients safe.
      There have been encouraging reports of intensive simulated skills courses targeting medical students to prepare them for the transition to junior surgeon.
      • Meier A.H.
      • Henry J.
      • Marine R.
      • Murray W.B.
      Implementation of a Web- and simulation-based curriculum to ease the transition from medical school to surgical internship.
      • Esterl Jr., R.M.
      • Henzi D.L.
      • Cohn S.M.
      Senior medical student “Boot Camp”: can result in increased self-confidence before starting surgery internships.
      • Peyre S.E.
      • Peyre C.G.
      • Sullivan M.E.
      • Towfigh S.
      A surgical skills elective can improve student confidence prior to internship.
      • Tocco N.
      • Brunsvold M.
      • Kabbani L.
      • Lin J.
      • Stansfield B.
      • Mueller D.
      • et al.
      Innovation in internship preparation: an operative anatomy course increases senior medical students' knowledge and confidence.
      Studies in the United States have shown that simulated skills courses or boot camp can increase the confidence of junior surgical interns.
      • Esterl Jr., R.M.
      • Henzi D.L.
      • Cohn S.M.
      Senior medical student “Boot Camp”: can result in increased self-confidence before starting surgery internships.
      • Peyre S.E.
      • Peyre C.G.
      • Sullivan M.E.
      • Towfigh S.
      A surgical skills elective can improve student confidence prior to internship.
      • Okusanya O.T.
      • Kornfield Z.N.
      • Reinke C.E.
      • Morris J.B.
      • Sarani B.
      • Williams N.N.
      • et al.
      The effect and durability of a pregraduation boot cAMP on the confidence of senior medical student entering surgical residencies.
      • Todd S.R.
      • Fahy B.N.
      • Paukert J.
      • Johnson M.L.
      • Bass B.L.
      Surgical intern survival skills curriculum as an intern: does it help?.
      They have largely been small-scale feasibility studies with self-assessed confidence as the primary outcome and no clinical metrics. Subjects on these courses have in particular found the “hands-on” training sessions to be of most benefit.
      • Esterl Jr., R.M.
      • Henzi D.L.
      • Cohn S.M.
      Senior medical student “Boot Camp”: can result in increased self-confidence before starting surgery internships.
      There is also some objective evidence of improvement in both knowledge
      • Tocco N.
      • Brunsvold M.
      • Kabbani L.
      • Lin J.
      • Stansfield B.
      • Mueller D.
      • et al.
      Innovation in internship preparation: an operative anatomy course increases senior medical students' knowledge and confidence.
      • Boehler M.L.
      • Rogers D.A.
      • Schwind C.J.
      • Fortune J.
      • Ketchum J.
      • Dunnington G.
      A senior elective designed to prepare medical students for surgical residency.
      and technical skills.
      • Brunt L.M.
      • Halpin V.J.
      • Klingensmith M.E.
      • Tiemann D.
      • Matthews B.D.
      • Spitler J.A.
      • et al.
      Accelerated skills preparation and assessment for senior medical students entering surgical internship.
      Although the majority of studies have focused on generic surgical skills, specialty-specific boot camps have also been shown to improve technical skills in first year residents.
      • Sonnadara R.R.
      • Van Vliet A.
      • Safir O.
      • Alman B.
      • Ferguson P.
      • Kraemer W.
      • et al.
      Orthopedic boot camp: examining the effectiveness of an intensive surgical skills course.
      In 2012, Fernandez et al
      • Fernandez G.L.
      • Page D.W.
      • Coe N.P.
      • Lee P.C.
      • Patterson L.A.
      • Skylizard L.
      • et al.
      Boot cAMP: educational outcomes after 4 successive years of preparatory simulation-based training at onset of internship.
      reported encouraging results from 4 successive years of a simulation-based surgical boot camp. One of the challenges they highlight is the need for protected teaching time and how to balance this against duty-hour regulations. A proposed solution is to implement this type of training before PGY-1. In fact, as early as 2004, the American Surgical Association Blue Ribbon Committee recommended collaboration between the Division of Education of the American College of Surgeons, the Accreditation Council for Graduate Medical Education, Association of American Medical Colleges, surgical boards, and medical schools to develop a surgical preparation curriculum to assist the transition from medical student to surgery resident.
      • Debas H.T.
      • Bass B.L.
      • Brennan M.F.
      • Flynn T.C.
      • Folse J.R.
      • Freischlag J.A.
      • et al.
      American Surgical Association Blue Ribbon Committee Report on Surgical Education: 2004.
      Progress has been made in the United States; however, there remains a lack of supporting evidence that these encouraging results can be applied to the training system in the UK.
      Competence in medical school examinations assesses both cognitive and to some extent practical skills. The focus of these examinations is on testing knowledge and targeted technical or practical skills. However, this does not necessarily provide adequate preparation for the real-world situations that a surgical intern faces. The purpose of this study was to develop, pilot, and evaluate an immersive, intensive, simulated 1-week course that recreates experiences and situations that a surgical doctor is likely to face in their first weeks as a clinician. It aims specifically to prepare final year medical students for the transition into their surgical internship. This project allows the subject to learn and practice in a controlled, simulated environment away from the patient.

      Hypothesis

      Attendance at a 1-week intensive simulation course will better prepare medical students for the transition to the role of surgical intern as measured by clinical performance (primary outcome) and trainee confidence and clinical procedures logbook (secondary outcome).

      Methods

      Subjects

      Final year students who had completed medical school and were due to commence their internship in August 2012 were eligible for the study. Eligible subjects who voluntarily expressed an interest in participation were met by a study investigator (PS). Both written and verbal information were provided and written consent was then obtained for all subjects. Advice regarding ethical approval was sought from the institutional review board and a waiver was recommended.

      Settings and scenarios

      All assessments and interventions took place in a controlled, simulated environment.

      Simulated ward

      A recently validated, immersive, high-fidelity ward simulator in which realistic perioperative patient encounters can be recreated.
      • Pucher P.H.
      • Aggarwal R.
      • Srisatkunam T.
      • Darzi A.
      Validation of the Simulated Ward Environment for Assessment of Ward-Based Surgical Care.

      Simulated operating suite

      An immersive, high-fidelity replica of an operating theatre, complete with an operating table, anesthetic equipment, and surgical scrubbing facilities.
      • Aggarwal R.
      • Undre S.
      • Moorthy K.
      • Vincent C.
      • Darzi A.
      The simulated operating theatre: comprehensive training for surgical teams.

      Clinical skills laboratory

      The clinical skills laboratory is complete, with simulation equipment for basic clinical procedures, such as venipuncture and intravenous cannulation, male and female urinary catheterization, and arterial blood gas sampling.

      Virtual reality skills laboratory

      The virtual reality skills laboratory proves access to the LapMentor virtual reality laparoscopic simulator (Simbionix Corporation, Cleveland, OH).
      The SIMweek intervention consisted of a 1-week intensive simulation boot camp using the environments as described. The interdisciplinary faculty consisted of general surgeons, gynecologists, and nursing staff, with experience ranging from a current surgical intern to attending level. The week comprised of 3 day shifts (from 8 am to 8 pm) and 1 night shift (from 8 pm to 8 am). The format for each day was similar, focusing on practical and technical skills rather than didactic teaching. Further details of the activities during the week of simulation can be found in Fig 1.
      The procedures and tutorials were chosen to represent those most likely to be performed by the subjects in their first weeks as an intern. Therefore, a decision was made not to teach intermediate laparoscopic skills, but rather to focus on basic skills, such as laparoscopic camera navigation. In addition, the core procedures for Foundation Year 1 (intern year) competencies

      The Foundation Programme. Core procedures & assessment forms.

      were used in conjunction with the experiences of the broad ranging faculty to maximize the clinical relevance of the immersive course. In the UK, satisfactory completion of the intern year is mandatory to obtain full registration from the General Medical Council. To enhance the realism, certain tasks were repeated throughout the course and the weighting of how many repetitions were based on the frequency with which these tasks are performed in the clinical setting. In addition, between stations, subjects had breaks and mock bleeps (or pager calls) that occurred sporadically throughout the day (including during breaks). Every subject received ≥2 mock bleeps during the day shifts and ≥3 overnight. The number varied depending on their performance. For example, if performance was felt to be lacking in 1 aspect, such as initial assessment, then a further mock bleep would be administered to provide an extra chance to practice and build on the feedback received from the previous mock bleep. An example is provided in Fig 2.

      Study design

      In the interests of feasibility for recruitment, a nonrandomized, controlled trial design was employed (because the intervention group were required to participate in a week-long course, this was likely to conflict with travel plans which are common at this stage of training). The study design is outlined in Fig 3. All subjects underwent baseline assessment of technical skills and confidence before starting their internship posts. The control group had no further intervention beyond standard medical school training. From 2012, this included 1 week of intern shadowing.
      The intervention group underwent an immersive, intensive, 1-week simulation course, followed by a postintervention assessment of technical skills and confidence. The intervention group continued with the compulsory intern shadowing. Follow-up assessments of technical skills and confidence were performed for both intervention and control groups at 1 and 6 months into their intern year. All subjects were in addition instructed to keep a clinical logbook of procedures performed for the first month and for a 1-week sampling period at 4 and 6 months. A 1-week sample was used because it was felt to be unfeasible to request subjects to reliably record all clinical procedures performed over a 6-month period.

      Outcomes

      Primary outcome: Clinical performance

      We assessed clinical performance using clinical objective structured clinical examinations (OSCEs), surgical technical skills, and laparoscopic skills assessments at baseline, after the SIMweek intervention (intervention group only), 1 month into the internship, and 6 months into the internship.

      Secondary outcomes

      Self-assessed trainee confidence questionnaires

      We asses confidence at baseline, after the SIMweek (intervention group only), and at 1 and 6 months into the internship as for primary outcomes.

      Clinical procedures logbook

      We collected clinical procedures logbooks at weeks 1, 2, 3, and 4, and at months 4 and 6.

      Assessment tools

      OSCEs

      Basic clinical skills were assessed using OSCE stations. The assessments are all procedures taken from the core procedures for Foundation Year 1 (intern year) competencies.

      The Foundation Programme. Core procedures & assessment forms.

      Score sheets were derived from the competency assessments provided by the Foundation Programme

      The Foundation Programme. Core procedures & assessment forms.

      for venipuncture, male urethral catheterization, intravenous cannulation, and arterial blood gas sampling.
      Surgery-specific OSCEs comprised 3 surgical technical skills stations and 1 laparoscopic skills station assessed using previously validated tasks and metrics
      • Mackay S.
      • Datta V.
      • Chang A.
      • Shah J.
      • Kneebone R.
      • Darzi A.
      Multiple Objective Measures of Skill (MOMS): a new approach to the assessment of technical ability in surgical trainees.
      • Aggarwal R.
      • Crochet P.
      • Dias A.
      • Misra A.
      • Ziprin P.
      • Darzi A.
      Development of a virtual reality training curriculum for laparoscopic cholecystectomy.
      • Martin J.A.
      • Regehr G.
      • Reznick R.
      • MacRae H.
      • Murnaghan J.
      • Hutchison C.
      • et al.
      Objective structured assessment of technical skill (OSATS) for surgical residents.
      : Interrupted suture closure of a wound (assessed using the objective structured assessment of technical skills; objective structured assessment of technical skill [OSATS]); mattress suture closure of a wound (OSATS); excision of a skin lesion (OSATS); and laparoscopic camera navigation assessed using time and accuracy. Scoring was performed by surgical faculty trained for assessment of medical school final OSCE examinations. OSCE stations were video recorded (omitting the face) and a 25% sample were assessed by a second blinded reviewer to ensure accuracy of scoring.

      Confidence questionnaire

      Confidence was assessed using a self-assessment questionnaire consisting of 35 questions relating to confidence, each marked on a 5-point Likert scale (the questions are provided in Appendix 1). This was developed based on those used for the previously mentioned studies on surgical skills courses for medical students.
      • Esterl Jr., R.M.
      • Henzi D.L.
      • Cohn S.M.
      Senior medical student “Boot Camp”: can result in increased self-confidence before starting surgery internships.
      • Peyre S.E.
      • Peyre C.G.
      • Sullivan M.E.
      • Towfigh S.
      A surgical skills elective can improve student confidence prior to internship.
      • Okusanya O.T.
      • Kornfield Z.N.
      • Reinke C.E.
      • Morris J.B.
      • Sarani B.
      • Williams N.N.
      • et al.
      The effect and durability of a pregraduation boot cAMP on the confidence of senior medical student entering surgical residencies.
      • Todd S.R.
      • Fahy B.N.
      • Paukert J.
      • Johnson M.L.
      • Bass B.L.
      Surgical intern survival skills curriculum as an intern: does it help?.
      Questions that were not felt to be relevant to UK Foundation Year (intern) doctors were removed.

      Clinical procedures logbook

      We conducted telephone interview of subjects for the number of bedside procedures performed either independently or while supervised.

      Statistical analysis

      Nonparametric statistical methods were employed with the use of Mann–Whitney U test for between-group comparisons and the Wilcoxon signed-rank test to analyze data within groups. Interrater reliability for clinical OSCEs was assessed using Cohen's kappa. Interrater reliability for surgical technical skills was assessed using Cronbach's alpha. All statistical analyses were performed using the Statistical Package for the Social Science version 20 (SPSS, Inc, Chicago, IL).

      Results

      Ten students were recruited to and completed the intervention arm. Nine students were recruited as controls; however, 2 students were unable to start their intern posts and were therefore excluded from all analyses. One control was lost to follow-up after 3 weeks and was therefore also excluded from all analyses, leaving 6 students in the control arm.

      Baseline

      No differences were demonstrated in baseline clinical OSCE, surgical technical skills, confidence, or laparoscopic camera operating scores between intervention and control groups.

      Primary outcome: Clinical performance

      Table I and Fig 4 display the results for the basic clinical OSCEs and surgical technical skills assessments. The intervention group outperformed the control group at both 1 and 6 months in both the clinical OSCEs and the surgical technical skills. Intervention performance significantly improved from baseline to after the course; however, no improvements were made in performance from after the course to 1 or 6 months. Control performance significantly improved from 1 to 6 months. Video-based interrater reliability revealed a good level of agreement for clinical OSCEs with a Cohen's kappa of 0.632 (P < .001) and a good level of agreement for surgical technical skills with a Cronbach's alpha of 0.836.
      Table IPrimary outcomes
      OutcomeBaselinePost SIMweek1 Month6 Month
      InterventionControlInterventionInterventionControlInterventionControl
      Clinical OSCEs
       Median (IQR)66 (23.3)78 (9)112 (5.75)111 (7.25)82 (13.5)107 (7.13)93 (14.6)
      Between-group difference over time
      Baseline1 Month6 Month
      Intervention vs controlP = .181P = .001P = .007
      Within-group difference over time
      Post SIMweek vs baseline1 Month vs post SIMweek1 Month vs baseline6 vs 1 month
      InterventionP = .005P = .646P = .005P = .173
      ControlP = .492P = .043
      Surgical technical skills
      Total OSATS score for 3 tasksBaselinePost SIMweek1 Month6 Month
      InterventionControlInterventionInterventionControlInterventionControl
      Median (IQR)48 (9)52.5 (23.3)78 (13.3)78.5 (10.8)63 (18.5)92.5 (19.8)69 (23.8)
      Between-group difference over time
      Baseline1 Month6 Month
      Intervention vs controlP = .381P = .030P = .044
      Within-group difference over time
      Post SIMweek vs baseline1 Month vs post SIMweek1 Month vs baseline6 vs 1 month
      InterventionP = .005P = .624P = .011P = .092
      ControlP = .078P = .042
      Figure thumbnail gr4
      Fig 4Clinical performance. A, Clinical objective structured clinical examinations. B, Surgical technical skills.
      The only significant differences in laparoscopic skills demonstrated between intervention and control groups were in accuracy using the 0° laparoscopic task at 6 months (90% vs 71.4; P = .030) and time using the 30° laparoscopic task at 6 months (129 vs 177 seconds; P = .006).

      Secondary outcomes

      Confidence

      Table II and Fig 5 display the results for self-reported confidence. Both intervention and control confidence significantly improved from baseline to 1 month and from 1 to 6 months. However, the intervention group was significantly more confident than the control group at both the 1- and 6-month assessments. There was no difference between post-SIMweek and 1 month confidence for the intervention group.
      Table IISecondary outcomes
      Self-reported confidence
      BaselinePost SIMweek1 Month6 Month
      InterventionControlInterventionInterventionControlInterventionControl
      Median (IQR)101 (12.3)95.5 (15)149 (20.3)142 (11)119 (17.5)148 (20.5)129 (25.8)
      Between-group difference over time
      Baseline1 Month6 Month
      Intervention vs controlP = .368P < .001P = .022
      Within-group difference over time
      Post SIMweek vs baseline1 Month vs post SIMweek1 Month vs baseline6 vs 1 month
      InterventionP = .005P = .240P = .005P = .018
      ControlP = .028P = .028
      Clinical procedures logbook
      Total bedside procedures1 Month4 Month
      Last full week of work.
      6 Month
      Last full week of work.
      InterventionControlInterventionControlInterventionControl
      Median (IQR)71 (52.8)73.5 (29.5)28.5 (16.3)17.5 (8.25)19 (14)15.5 (36.3)
      Between group difference over time
      Intervention vs controlP = .958P = .093P = .713
      Last full week of work.

      Clinical procedures logbook

      Table II displays the results for the clinical procedures logbook. No differences were observed between intervention and control total numbers of bedside procedures at any time point, although there was a nonsignificant trend toward a greater number in the intervention group at 4 months.

      Discussion

      A 1-week, immersive simulation course significantly improved graduating interns' objectively measured clinical skills, surgical technical skills, and self-reported confidence. This improvement persisted at the 1- and 6-month follow-up assessments when compared with controls, although there were no demonstrable differences in the volume of clinical experience. This result suggests that improvements in training quality rather than increasing time in training can improve outcomes. The SIMweek course accelerated subjects' acquisition of clinical skills, technical skills, and confidence, so that at the start of their internship these were at a level similar to that seen at 1 month into the internship. The intervention group's clinical performance did not significantly improve from 1 to 6 months; however, their confidence continued to significantly improve and their technical skills demonstrated a trend toward improvement although, this did not attain statistical significance. By contrast, the control group's clinical skills did not display any significant improvement at 1 month compared with baseline; however, their confidence significantly improved and their technical skills demonstrated a nonsignificant trend toward improvement. Control clinical skills, technical skills, and confidence significantly improved from 1 month to the 6-month follow-up period, while remaining lower than that of the intervention group. No differences were observed between intervention and control groups in numbers of clinical procedures performed at 1, 4, or 6 months. The SIMweek course is thus a feasible and effective intervention to ease the transition from medical student to surgical intern.
      An interesting and perhaps surprising observation was the fact that intervention subjects' performance did not improve from after the course to 1 or 6 months, in contrast with the control group, which did. The authors feel that this is likely owing to the high performance of the intervention subjects on the postcourse assessments. Owing to their proximity to the top end of the assessment scale (median 112 out of a possible maximum of 119 for the clinical OSCE, 78 out of a maximum of 105 for the surgical technical skill, and 149 out of a maximum 175 for self-reported confidence), there was in fact little room for improvement without a near-perfect performance. For future studies, perhaps more challenging tasks may allow the intervention subjects to demonstrate further improvement.
      The results of this study are encouraging; however, for an intervention to make a successful transition from the laboratory into clinical implementation, the costs of such a course must be considered. The SIMweek course consumables, including venue and equipment hire but excluding faculty expenses (because this was provided on a voluntary basis), came to <$450 per subject. Because this inaugural course was run as a feasibility study, it was run with intentionally small numbers to provide high faculty to student ratios. The costs per subject would be reduced further if run with larger numbers. The cost may prove a valuable investment when seen in the context of the cost of providing paid shadowing. In addition, it is worth considering the potential effects this could have on the August and July effect and the subsequent cost and patient safety benefits. The significantly better clinical and technical skills of the intervention group persisted over 6 months, although the control group continued to close the gap. This improvement could potentially have been translated into better clinical outcomes for those patients treated by the intervention group. It must be remembered, however, that this study was not designed to evaluate effects on patient outcomes and future data on this aspect of the program would be of interest. These data would also inform on the incremental benefits of further additional skills training, such as in the American College of Surgeons/Association of Program Directors in Surgery/ASE Resident Prep Curriculum

      American College of Surgeons. ACS/APDS/ASE resident prep curriculum. Chicago: American College of Surgeons.

      being piloted in the United States. In theory, this practice could produce greater clinical benefits for patients; however, this addition would need to be carefully balanced against the increased resources required.
      The controlled simulated environment allows subjects to learn from their mistakes without any compromise to patient care. The immersive environment provides a realistic environment and allows for contextualized learning and practice of technical procedures. For example, a mock bleep scenario for poor urine output allows the subject to obtain a focused history and examination before coming to the conclusion that catheterization is required and this can then be performed using a simulator. Although shadowing an intern can also be useful, it relies on chance occurrences. The use of simulation can be 1 method to ensure coverage of relevant experience. Previous studies have shown promising results for the use of simulation to prepare medical students for the transition to surgical intern.
      • Esterl Jr., R.M.
      • Henzi D.L.
      • Cohn S.M.
      Senior medical student “Boot Camp”: can result in increased self-confidence before starting surgery internships.
      • Peyre S.E.
      • Peyre C.G.
      • Sullivan M.E.
      • Towfigh S.
      A surgical skills elective can improve student confidence prior to internship.
      • Okusanya O.T.
      • Kornfield Z.N.
      • Reinke C.E.
      • Morris J.B.
      • Sarani B.
      • Williams N.N.
      • et al.
      The effect and durability of a pregraduation boot cAMP on the confidence of senior medical student entering surgical residencies.
      • Boehler M.L.
      • Rogers D.A.
      • Schwind C.J.
      • Fortune J.
      • Ketchum J.
      • Dunnington G.
      A senior elective designed to prepare medical students for surgical residency.
      • Brunt L.M.
      • Halpin V.J.
      • Klingensmith M.E.
      • Tiemann D.
      • Matthews B.D.
      • Spitler J.A.
      • et al.
      Accelerated skills preparation and assessment for senior medical students entering surgical internship.
      These studies have often used self-reported confidence as their primary outcome.
      • Esterl Jr., R.M.
      • Henzi D.L.
      • Cohn S.M.
      Senior medical student “Boot Camp”: can result in increased self-confidence before starting surgery internships.
      • Peyre S.E.
      • Peyre C.G.
      • Sullivan M.E.
      • Towfigh S.
      A surgical skills elective can improve student confidence prior to internship.
      • Okusanya O.T.
      • Kornfield Z.N.
      • Reinke C.E.
      • Morris J.B.
      • Sarani B.
      • Williams N.N.
      • et al.
      The effect and durability of a pregraduation boot cAMP on the confidence of senior medical student entering surgical residencies.
      Although confidence is likely an important factor in improving the anxieties of starting as surgical intern, this study adds significantly to the body of evidence by using objectively measured clinical performance as its primary outcome and providing data about skills retention 6 months after the intervention. Demonstration of persistently improved clinical performance may have a direct impact on patient care. This aspect is not assessed in this study, because all performance measurements were conducted in the simulated environment owing to the described benefits. The potential effect of SIMweek on patient outcomes is an area for future work, where performance measurements should ideally be performed in the clinical environment.
      The main limitation in the study design is the lack of randomization, a pragmatic decision made because of the inherent difficulties of recruiting to a study that requires commitment to 1 week of a subjects' time during the holiday period between graduation and starting their clinical posts. In addition, owing to the enthusiasm for the SIMweek course among the volunteers, it was anticipated that there would be a dropout bias in the control arm, even if randomized. Previously reported boot camp studies have also employed a nonrandomized design.
      • Okusanya O.T.
      • Kornfield Z.N.
      • Reinke C.E.
      • Morris J.B.
      • Sarani B.
      • Williams N.N.
      • et al.
      The effect and durability of a pregraduation boot cAMP on the confidence of senior medical student entering surgical residencies.
      Where this study improves on this is by ensuring baseline assessments were completed for both control and intervention arms. Despite the nonrandomized design, the baseline scores between intervention and control arms revealed no differences in clinical skills, surgical technical skills, or confidence. Another limitation was the small number of controls. This was in part owing to the need for baseline testing at a time when students are requested to curtail their holidays to start their compulsory shadowing. Because they were in the control arm, there was not a clear benefit to participating, although future simulation courses were offered to mitigate this. Two of the recruited controls had to be excluded because they were unable to commence their posts and 1 control was lost to follow-up after 3 weeks. Despite the small numbers, the observed differences demonstrated significance, thus reducing the risk of a type II error. There is a risk that there was a difference between the 2 groups at baseline that was not significant owing to the small numbers. Nevertheless, the clinical and surgical technical skills performance was higher in the control group than the intervention group. Thus, the improvement with the intervention may have been underestimated, adding more weight to the rejection of the null hypothesis. It must, however, be remembered that the secondary outcome of self-reported confidence was actually greater in the intervention group at baseline. The SIMweek curriculum used in this study was developed from expert consensus. An essential next step to improve the curriculum would be to ensure that it is refined using both evidence-based methodology and, importantly, feedback from the end-users, namely, the surgical interns.
      In conclusion, the immersive simulation course improved subjects' objectively assessed clinical skills, surgical technical skills, and self-reported confidence level. Despite similar clinical experience as controls, the improved performance of the intervention group persisted at both 1 and 6 months. This feasible and effective intervention to ease transition from student to intern could reduce error and enhance patient safety.

      Appendix

      Appendix 1Confidence questionnaire
      Please rate your confidence levels to manage the following conditions or perform the following procedures:
      1. Initial management of patients with perioperative chest pain.
       (No confidence)12345(Confident)
      2. Initial management of a postoperative fever.
       (No confidence)12345(Confident)
      3. Initial management of sepsis.
       (No confidence)12345(Confident)
      4. Initial management of postoperative bleeding.
       (No confidence)12345(Confident)
      5. Acute management of a gastrointestinal bleed.
       (No confidence)12345(Confident)
      6. Initial management of oliguria.
       (No confidence)12345(Confident)
      7. Placing a peripheral IV cannula.
       (No confidence)12345(Confident)
      8. Placing a Foley urinary catheter in a female patient.
       (No confidence)12345(Confident)
      9. Placing a Foley urinary catheter in a male patient.
       (No confidence)12345(Confident)
      10. Managing electrolyte imbalances in perioperative patients.
       (No confidence)12345(Confident)
      11. Prescribing fluids and electrolytes.
       (No confidence)12345(Confident)
      12. Nasogastric tube placement and management.
       (No confidence)12345(Confident)
      13. Basic peri-operative pain management.
       (No confidence)12345(Confident)
      14. Write and understand the essential components of a daily progress note.
       (No confidence)12345(Confident)
      15. Understand the essential components of a surgical note.
       (No confidence)12345(Confident)
      16. Prescribing common medications that surgical FY1s order.
       (No confidence)12345(Confident)
      17. Identify correct central venous catheter placement on a chest radiograph.
       (No confidence)12345(Confident)
      18. Identify correct nasogastric tube placement on a radiograph.
       (No confidence)12345(Confident)
      19. Identify common surgical diagnoses on differing radiographic studies.
       (No confidence)12345(Confident)
      20. How to prep and drape a surgical patient.
       (No confidence)12345(Confident)
      21. Perform an instrument tie.
       (No confidence)12345(Confident)
      22. Perform a simple interrupted closure.
       (No confidence)12345(Confident)
      23. Place a horizontal mattress suture.
       (No confidence)12345(Confident)
      24. Perform a vertical mattress suture.
       (No confidence)12345(Confident)
      25. Make a skin incision.
       (No confidence)12345(Confident)
      26. Excise a 2-cm sebaceous cyst on the back.
       (No confidence)12345(Confident)
      27. Know basic settings on lap camera/insufflator.
       (No confidence)12345(Confident)
      28. Drive a 0° laparoscopic camera.
       (No confidence)12345(Confident)
      29. Drive a 30° laparoscopic camera.
       (No confidence)12345(Confident)
      30. Identifying surgical instruments by name and function.
       (No confidence)12345(Confident)
      31. Perform and interpret arterial blood gas analysis.
       (No confidence)12345(Confident)
      32. Identify anatomical images on computed tomography.
       (No confidence)12345(Confident)
      33. Identify anatomical images on radiographs.
       (No confidence)12345(Confident)
      34. Global confidence.
       (No confidence)12345(Confident)
      35. I am/was well prepared to be a surgical FY1 at the start of my job.
       (No confidence)12345(Confident)

      References

        • Young J.Q.
        • Ranji S.R.
        • Wachter R.M.
        • Lee C.M.
        • Niehaus B.
        • Auerbach A.D.
        “July effect”: impact of the academic year-end changeover on patient outcomes: a systematic review.
        Ann Intern Med. 2011; 155: 309-315
        • Jen M.H.
        • Bottle A.
        • Majeed A.
        • Bell D.
        • Aylin P.
        Early In-hospital mortality following trainee doctors' first day at work.
        Plos One. 2009; 4: e7103
        • Willcock S.M.
        • Daly M.G.
        • Tennant C.C.
        • Allard B.J.
        Burnout and psychiatric morbidity in new medical graduates.
        Med J Aust. 2004; 181: 357-360
        • House J.
        NHS shadowing scheme not enough to keep patients safe.
        Lancet. 2012; 380: 459
        • Meier A.H.
        • Henry J.
        • Marine R.
        • Murray W.B.
        Implementation of a Web- and simulation-based curriculum to ease the transition from medical school to surgical internship.
        Am J Surg. 2005; 190: 137-140
        • Esterl Jr., R.M.
        • Henzi D.L.
        • Cohn S.M.
        Senior medical student “Boot Camp”: can result in increased self-confidence before starting surgery internships.
        Curr Surg. 2006; 63: 264-268
        • Peyre S.E.
        • Peyre C.G.
        • Sullivan M.E.
        • Towfigh S.
        A surgical skills elective can improve student confidence prior to internship.
        J Surg Res. 2006; 133: 11-15
        • Tocco N.
        • Brunsvold M.
        • Kabbani L.
        • Lin J.
        • Stansfield B.
        • Mueller D.
        • et al.
        Innovation in internship preparation: an operative anatomy course increases senior medical students' knowledge and confidence.
        Am J Surg. 2013; 206: 269-279
        • Okusanya O.T.
        • Kornfield Z.N.
        • Reinke C.E.
        • Morris J.B.
        • Sarani B.
        • Williams N.N.
        • et al.
        The effect and durability of a pregraduation boot cAMP on the confidence of senior medical student entering surgical residencies.
        J Surg Educ. 2012; 69: 536-543
        • Todd S.R.
        • Fahy B.N.
        • Paukert J.
        • Johnson M.L.
        • Bass B.L.
        Surgical intern survival skills curriculum as an intern: does it help?.
        Am J Surg. 2011; 202: 713-718
        • Boehler M.L.
        • Rogers D.A.
        • Schwind C.J.
        • Fortune J.
        • Ketchum J.
        • Dunnington G.
        A senior elective designed to prepare medical students for surgical residency.
        Am J Surg. 2004; 187: 695-697
        • Brunt L.M.
        • Halpin V.J.
        • Klingensmith M.E.
        • Tiemann D.
        • Matthews B.D.
        • Spitler J.A.
        • et al.
        Accelerated skills preparation and assessment for senior medical students entering surgical internship.
        J Am Coll Surg. 2008; 206: 897-904
        • Sonnadara R.R.
        • Van Vliet A.
        • Safir O.
        • Alman B.
        • Ferguson P.
        • Kraemer W.
        • et al.
        Orthopedic boot camp: examining the effectiveness of an intensive surgical skills course.
        Surgery. 2011; 149: 745-749
        • Fernandez G.L.
        • Page D.W.
        • Coe N.P.
        • Lee P.C.
        • Patterson L.A.
        • Skylizard L.
        • et al.
        Boot cAMP: educational outcomes after 4 successive years of preparatory simulation-based training at onset of internship.
        J Surg Educ. 2012; 69: 242-248
        • Debas H.T.
        • Bass B.L.
        • Brennan M.F.
        • Flynn T.C.
        • Folse J.R.
        • Freischlag J.A.
        • et al.
        American Surgical Association Blue Ribbon Committee Report on Surgical Education: 2004.
        Ann Surg. 2005; 241: 1-8
        • Pucher P.H.
        • Aggarwal R.
        • Srisatkunam T.
        • Darzi A.
        Validation of the Simulated Ward Environment for Assessment of Ward-Based Surgical Care.
        Ann Surg. 2013; 259: 215-221
        • Aggarwal R.
        • Undre S.
        • Moorthy K.
        • Vincent C.
        • Darzi A.
        The simulated operating theatre: comprehensive training for surgical teams.
        Qual Safe Health Care. 2004; 13: i27-i32
      1. The Foundation Programme. Core procedures & assessment forms.

        • Mackay S.
        • Datta V.
        • Chang A.
        • Shah J.
        • Kneebone R.
        • Darzi A.
        Multiple Objective Measures of Skill (MOMS): a new approach to the assessment of technical ability in surgical trainees.
        Ann Surg. 2003; 238: 291-300
        • Aggarwal R.
        • Crochet P.
        • Dias A.
        • Misra A.
        • Ziprin P.
        • Darzi A.
        Development of a virtual reality training curriculum for laparoscopic cholecystectomy.
        Br J Surg. 2009; 96: 1086-1093
        • Martin J.A.
        • Regehr G.
        • Reznick R.
        • MacRae H.
        • Murnaghan J.
        • Hutchison C.
        • et al.
        Objective structured assessment of technical skill (OSATS) for surgical residents.
        Br J Surg. 1997; 84: 273-278
      2. American College of Surgeons. ACS/APDS/ASE resident prep curriculum. Chicago: American College of Surgeons.