Advertisement

Relationship of procedural numbers with meaningful procedural autonomy in general surgery residents

Published:December 22, 2017DOI:https://doi.org/10.1016/j.surg.2017.10.011

      Abstract

      Background

      Concerns exist regarding the competency of general surgery graduates with performing core general surgery procedures. Current competence assessment incorporates minimal procedural numbers requirements.

      Methods

      Based on the Zwisch scale we evaluated the level of autonomy achieved by categorical PGY1-5 general surgery residents at 14 U.S. general surgery resident training programs between September 1, 2015 and December 31, 2016. With 5 of the most commonly performed core general surgery procedures, we correlated the level of autonomy achieved by each resident with the number of procedures they had performed before the evaluation period, with the intent of identifying specific target numbers that would correlate with the achievement of meaningful autonomy for each procedure with most residents.

      Results

      Whereas a definitive target number was identified for laparoscopic appendectomy (i.e. 25), for the other 4 procedures studied (i.e. laparoscopic cholecystectomy, 52; open inguinal hernia repair, 42; ventral hernia repair, 35; and partial colectomy, 60), target numbers identified were less definitive and/or were higher than many residents will experience during their surgical residency training.

      Conclusions

      We conclude that procedural target numbers are generally not effective in predicting procedural competence and should not be used as the basis for determining residents' readiness for independent practice.

      Introduction

      Concerns have been raised about the effectiveness of current general surgery resident training in adequately preparing residents for independent practice,
      • Bell R.H.
      • Biester T.W.
      • Tabuenca A.
      • et al.
      Operative experience of residents in US general surgery programs: a gap between expectation and experience.
      • Sachdeva A.K.
      • Flynn T.C.
      • Brigham T.P.
      • et al.
      Interventions to address challenges associated with the transition from residency training to independent surgical practice.
      • Kairys J.C.
      • McGuire K.
      • Crawford A.G.
      • Yeo C.J.
      Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees?.
      • Damadi A.
      • Davis A.T.
      • Saxe A.
      • Apelgren K.
      ACGME duty-hour restrictions decrease resident operative volume: a 5-year comparison at an ACGME-accredited university general surgery residency.
      • Fronza J.S.
      • Prystowsky J.P.
      • DaRosa D.
      • Fryer J.P.
      Surgical residents' perception of competence and relevance of the clinical curriculum to future practice.
      • Bucholz E.M.
      • Sue G.R.
      • Yeo H.
      • Roman S.A.
      • Bell Jr, R.H.
      • Sosa J.A.
      Our trainees' confidence: results from a national survey of 4136 US general surgery residents.
      • Mattar S.G.
      • Alseidi A.A.
      • Jones D.B.
      • et al.
      General surgery residency inadequately prepares trainees for fellowship: results of a survey of fellowship program directors.
      including residents' ability to perform core general surgery procedures independently. Currently the final summative evaluation of each resident's operative competence, which is required in order to take the American Board of Surgery (ABS) qualifying exam, is based primarily on 2 things. The first is the cumulative end-of-rotation assessments, which combine judgment of operative performance with that of other performance parameters. This assessment process is not standardized, and there is significant variability between programs, including the assessment methods used for operative performance. Within programs, operative performance assessments are typically not completed consistently or in a timely manner by faculty.
      • Williams R.G.
      • Chen X.P.
      • Sanfey H.
      • Markwell S.J.
      • Mellinger J.D.
      • Dunnington G.L.
      The measured effect of delay in completing operative performance ratings on clarity and detail of ratings assigned.
      The second requirement is achievement of minimal procedural numbers as defined by the Accreditation Council for Graduate Medical Education (ACGME)/ABS. These number requirements are not based on individual procedures, but rather on defined categories of related procedures. No data currently exist to demonstrate that there is correlation between these defined category numbers requirements and individual resident competence with specific procedures within the same defined category.
      • Nygaard R.M.
      • Daly S.R.
      • Van Camp J.M.
      General surgery resident case logs: do they accurately reflect resident experience?.
      Furthermore, there is limited evidence supporting reliable correlation between the numbers of a specific procedure that trainees are exposed to and the achievement of procedural competence for that specific procedure.
      • Abdelrahman T.
      • Long J.
      • Egan R.
      • Lewis W.G.
      Operative experience vs. competence: a curriculum concordance and learning curve analysis.
      • De Siqueira J.R.
      • Gough M.J.
      Correlation between experience targets and competence for general surgery certification.
      Competency-based training requires that trainees achieve an acceptable level of competence with specific entrustable professional activities (EPAs)
      • Ten Cate O.
      Trust, competence, and the supervisor's role in postgraduate training.
      • Hauer K.E.
      • Ten Cate O.
      • Boscardin C.
      • Irby D.M.
      • Iobst W.
      • O'Sullivan P.S.
      Understanding trust as an essential element of trainee supervision and learning in the workplace.
      before they can be entrusted to perform this activity independently in clinical practice. This determination is therefore not necessarily dependent on the number of procedures they have performed. Performance of core operative procedures is perhaps the most essential EPA for surgical training. Resident autonomy in performing core operative procedures is a critical indicator of competency because supervising surgeons are unlikely to grant significant autonomy to a resident unless they believe he or she is competent. In this study we will attempt to determine if there is correlation between the numbers of core procedures performed by surgery residents and the level of competency they achieve with that procedure.

      Methods

      Resident operative performance assessments were collected between September 1, 2015, and December 31, 2017, at 14 general surgery residency training programs belonging to the Procedural Learning and Safety Collaborative (PLSC). Data were collected using SIMPL, a smart phone app-based system that was developed to facilitate timely assessment of residents' operative autonomy and performance with every procedure they participate in.
      • DaRosa D.A.
      • Zwischenberger J.B.
      • Meyerson S.L.
      • et al.
      A theory-based model for teaching and assessing residents in the operating room.
      • George B.C.
      • Teitelbaum E.N.
      • Meyerson S.L.
      • et al.
      Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance.
      • Bohnen J.D.
      • George B.C.
      • Williams R.G.
      • et al.
      The feasibility of real-time intraoperative performance assessment with SIMPL (System for Improving and Measuring Procedural Learning): early experience from a multi-institutional trial.
      • George B.C.
      • Bohnen J.D.
      • Williams R.G.
      • et al.
      Readiness of US general surgery residents for independent practice.
      SIMPL assessments include 3 scales used to determine 1) the autonomy level achieved by the resident during the procedure using the Zwisch scale (Fig 1); 2) the performance level of the resident during the operative procedure
      • George B.C.
      • Teitelbaum E.N.
      • Meyerson S.L.
      • et al.
      Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance.
      • Bohnen J.D.
      • George B.C.
      • Williams R.G.
      • et al.
      The feasibility of real-time intraoperative performance assessment with SIMPL (System for Improving and Measuring Procedural Learning): early experience from a multi-institutional trial.
      • George B.C.
      • Bohnen J.D.
      • Williams R.G.
      • et al.
      Readiness of US general surgery residents for independent practice.
      ; and 3) the patient-related relative complexity of the case compared with other identical/similar procedures.
      • George B.C.
      • Teitelbaum E.N.
      • Meyerson S.L.
      • et al.
      Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance.
      • Bohnen J.D.
      • George B.C.
      • Williams R.G.
      • et al.
      The feasibility of real-time intraoperative performance assessment with SIMPL (System for Improving and Measuring Procedural Learning): early experience from a multi-institutional trial.
      • George B.C.
      • Bohnen J.D.
      • Williams R.G.
      • et al.
      Readiness of US general surgery residents for independent practice.
      After the completion of a surgical procedure the SIMPL assessment process is initiated by the resident (or the supervising surgical faculty member) who first identifies the faculty surgeon (or resident) that they just operated with from a drop-down list in the app. Next they identify the procedure that was performed from a procedure list derived from the ACGME case log taxonomy. To ensure that assessments are performed in a timely manner, those that are not completed within 72 hours of initiation cannot be entered into the SIMPL database.
      Fig. 1
      Fig. 1The Zwisch scale for progressive procedural autonomy with defining descriptions for each of the 4 levels. 1) SHOW AND TELL STAGE: Resident essentially observes and assists, but may participate in some parts of the procedure such as opening and/or closing the incision. Attending “shows” resident how procedure is done and “tells” resident what needs to be known. 2) ACTIVE HELP STAGE: Resident begins to assume the surgeon role in some parts of the procedure with the attending providing active guidance (i.e. verbal and/or physical), including periodically swapping “surgeon” and “assistant” roles with the resident. When resident assumes the surgeon role, the attending actively assists, essentially guiding the resident through the procedure. 3) PASSIVE HELP STAGE: The resident is capable of safely doing significant parts of the procedure without active guidance (i.e. verbal and/or physical) from the attending, who passively assists. 4) SUPERVISION ONLY STAGE: The resident can safely and effectively perform the procedure using operating room staff or a junior resident to assist. The attending provides no significant verbal or physical guidance/intervention to the resident, but provides supervision and consultation if needed.
      To facilitate further development of the SIMPL assessment process and collaborative research efforts using the data collected by the app, a consortium of general surgery residency programs known as the PLSC was formed, which, at the time of this study, consisted of 14 U.S. general surgery residency training programs. Before being granted access to SIMPL, all participating surgical faculty and residents at each of these programs were required to participate in a 1-hour frame of reference training session to confirm common understanding of how to use the SIMPL app,
      • George B.C.
      • Teitelbaum E.N.
      • Darosa D.A.
      • et al.
      Duration of faculty training needed to ensure reliable or performance ratings.
      including the 3 assessment scales discussed earlier. Furthermore, individual programs were required to confirm that ≥70% of both residents and faculty had completed this training before general programmatic access to SIMPL being granted to trained users.
      Although all procedures performed by a categorical general surgery resident with a supervising surgical faculty attending were included in SIMPL, for this study, our analyses focused on the 5 procedures that were most frequently assessed during the study period (i.e. laparoscopic cholecystectomy, laparoscopic appendectomy, open inguinal hernia repair, ventral hernia repair, and partial colectomy) and that were considered “core” general surgery procedures by national surgical leadership.
      • Resident Education SC
      SCORE: Curriculum Outline for General Surgery 2016–2017.
      This study included data collected from all participating categorical general surgery residents (PGY1-5). For each individual resident participating in the study, a mean Zwisch score was determined based on the autonomy level they achieved during their total experience with each of these 5 procedures they performed during the entire study period. Only residents who had performed the procedure being evaluated at least 3 times were included in the analyses for that specific procedure. Additionally, the number of each of these procedures each resident had performed from the start of their residency until the start of the study period was collected from their ACGME case logs. For each of the 5 core procedures evaluated, individual residents' mean Zwisch scores achieved during the study period were plotted against the number of procedures they had performed before the study period. We determined the best fitted linear line to summarize the relationship between prior procedural experience and autonomy level achieved. Meaningful autonomy was defined as having achieved a mean Zwisch score of ≥3 (based on numerical conversion of the Zwisch scale: i.e. Show and Tell = 1, Active Help = 2, Passive Help = 3, Supervision Only = 4) for the procedure in question during the study period. The point where the best-fit line intersected with this meaningful autonomy level was used to determine the number of procedures at which the majority of the resident population should achieve meaningful procedural autonomy. We refer to this as the “transition point” (Table and Figs 2A–2E). All analyses were performed using R: a language and environment for statistical computing, version 3.2.2.
      TableRelationship of Procedural Numbers with Meaningful Autonomy in Surgical Residents
      ProcedureResidents
      Includes all participating residents who had performed the procedure ≥3 times.
      #s
      Inception point
      Inception point (IP): The number of procedures at which the resident population first begins to achieve meaningful autonomy, exclusive of performance outliers (i.e. residents who achieved meaningful autonomy after performing ≥5 fewer procedures than any other resident required to achieve meaningful autonomy).
      procedure #s
      Transition point
      Transition point (TP): Based on the line of “best fit” assigned to the plotted data points, the number of procedures at which the majority of the resident population transitions to meaningful autonomy.
      procedure #s
      % Meaningful autonomy
      Based on the Zwisch scale of progressive autonomy, the 2 highest autonomy levels (i.e. Passive Help and Supervision Only).
      IP—TP (n)
      % Meaningful autonomy
      Based on the Zwisch scale of progressive autonomy, the 2 highest autonomy levels (i.e. Passive Help and Supervision Only).
      beyond IP (n)
      % Meaningful autonomy
      Based on the Zwisch scale of progressive autonomy, the 2 highest autonomy levels (i.e. Passive Help and Supervision Only).
      beyond TP (n)
      Laparoscopic Appendectomy55132560% (9)83.8% (31)100% (22)
      Laparoscopic Cholecystectomy89195256.4% (22)62.9% (39)73.9% (17)
      Open Inguinal Hernia Repair5494258.1% (18)52.6% (20)28.6% (2)
      Ventral Hernia Repair40203563.2% (12)60.9% (14)50% (2)
      Partial Colectomy39236040% (8)45.5% (10)100% (2)
      * Includes all participating residents who had performed the procedure ≥3 times.
      Inception point (IP): The number of procedures at which the resident population first begins to achieve meaningful autonomy, exclusive of performance outliers (i.e. residents who achieved meaningful autonomy after performing ≥5 fewer procedures than any other resident required to achieve meaningful autonomy).
      Transition point (TP): Based on the line of “best fit” assigned to the plotted data points, the number of procedures at which the majority of the resident population transitions to meaningful autonomy.
      § Based on the Zwisch scale of progressive autonomy, the 2 highest autonomy levels (i.e. Passive Help and Supervision Only).
      Fig. 2
      Fig. 2Dot plot graphs correlating mean Zwisch scores achieved by individual general surgery residents with each of the 5 most commonly assessed core general surgery procedures (2A–2E) during the study period (September 1, 2015–December 31, 2016), with the number of times they participated in the performance of each of these procedures before the study period. “Best-fit” lines () with 95% confidence bands were applied to the graphs to summarize the correlation for each procedure evaluated. The dotted horizontal line () represents the threshold for meaningful procedural autonomy (i.e. mean Zwisch score ≥3) with all procedures. The dotted vertical lines () represent the procedural performance numbers that should correlate with the achievement of meaningful autonomy in the majority of the resident population as determined by the point of intersection of the best-fit line with the threshold line for meaningful autonomy (i.e. the “transition point”). The continuous vertical line () reflects the procedural performance number associated with the earliest achievement of meaningful autonomy in the resident population (i.e. the “inception point”). 2A: Laparoscopic Appendectomy. 2B: Laparoscopic Cholecystectomy. 2C: Open Inguinal Hernia Repair. 2D: Ventral Hernia Repair. 2E: Partial Colectomy.
      We also sought to establish at what number of procedures the resident population first begins to achieve meaningful autonomy for each procedure. We refer to this as the “inception point” (Table and Figs 2A–2E) and define it as the number of procedures at which meaningful autonomy was first achieved by a trainee. To avoid outlier performances in defining the inception point, we excluded any resident performances where meaningful autonomy was achieved in isolation (i.e. no other trainees achieved meaningful autonomy despite participating in at least 5 additional procedures).

      Results

      During the study period we collected SIMPL assessments for 10,130 total surgical performances (332 different types of procedures) by 536 surgical residents who were assessed by 444 supervising surgical faculty. Of the SIMPL assessments performed, 73.4% (7,437) involved procedures categorized as “core” for general surgery.
      • Resident Education SC
      SCORE: Curriculum Outline for General Surgery 2016–2017.
      For laparoscopic appendectomy, 55 residents performed ≥3 procedures during the study period. Of these, 31 (56.4%) achieved a mean autonomy level of ≥3. Based on the best-fit line of the intersection point with the meaningful autonomy line, the “transition point” was determined to be at 25 procedures, and all 22 (100%) of the residents who performed more than 25 laparoscopic appendectomies achieved meaningful autonomy (Fig 2A). With laparoscopic cholecystectomy, 89 residents performed ≥3 procedures during the study period. Of these, 41 (46.1%) achieved a mean autonomy level of ≥3. The transition point was determined to be at 52 procedures, and 17 (73.9%) of the 23 residents who had performed this many procedures achieved meaningful autonomy (Fig 2B). With open inguinal hernia repair, 54 residents performed ≥3 procedures during the study period. Of these, 21 (38.9%) achieved a mean autonomy level ≥3. The transition point was determined to be at 42 procedures. However, only 2 (28.6%) of the 7 residents who performed this many procedures achieved meaningful autonomy (Fig 2C). For ventral hernia repair, 40 residents performed ≥3 procedures during the study period. Of these, 14 (35%) achieved a mean autonomy level of ≥3. The transition was determined to be at 35, although only 2 (50%) of the 4 residents who performed this many procedures achieved meaningful autonomy (Fig 2D). For partial colectomy, 39 residents performed ≥3 procedures during the study period. Of these, 11 (28.2%) achieved a mean autonomy level of ≥3. The transition point was determined to be at 60, but only 2 residents performed this many procedures, although both (100%) achieved meaningful autonomy (Fig 2E). These results are summarized in Table.
      The procedural performance numbers that reflect when the residents first started to achieve meaningful autonomy (i.e. “the inception point”) are also summarized in Table. For laparoscopic appendectomy, the inception point occurred at 13 procedures, as none (0%) of the residents who had performed <13 procedures achieved meaningful autonomy, whereas 36 (87.8%) of those who performed ≥13 procedures achieved meaningful autonomy. For laparoscopic cholecystectomy, the inception point was 19 procedures, as none (0%) of the residents who had performed <19 procedures achieved meaningful autonomy, whereas 39 (62.9%) of those who performed ≥19 procedures achieved meaningful autonomy. With open inguinal hernia repair, the inception point was 9, as only 1 (6.25%) of the residents who performed <9 procedures achieved meaningful autonomy, whereas 20 (52.6%) of those who performed ≥9 procedures achieved meaningful autonomy. With ventral hernia repair, the inception point was 20 procedures, as none (0%) of the residents who performed <20 procedures achieved meaningful autonomy, whereas 14 (60.8%) of those who performed ≥20 procedures achieved meaningful autonomy. Finally, with partial colectomy, the inception point was 23 procedures, as only 1 (5.6%) of the residents who performed <23 procedures achieved meaningful autonomy, whereas 10 (47.6%) of those who performed ≥23 procedures achieved meaningful autonomy.

      Discussion

      The findings in this study suggest that specific procedural target numbers that correlate with the achievement of procedural competency in surgical residency training may be identifiable for some core surgical procedures, but will likely be difficult to define for most of the 132 core procedures. We identified what appear to be reliable target numbers for laparoscopic appendectomy and laparoscopic cholecystectomy, the two most commonly performed procedures. With laparoscopic appendectomy, the target number was most reliable, as all (100%) residents who performed at least 25 procedures achieved meaningful autonomy. With laparoscopic cholecystectomy, a significant majority (73.9%) of residents who performed at least 52 procedures achieved meaningful autonomy. Although it is beyond the scope of this study, it will be essential to attempt to establish why the other 26.1% of residents who performed ≥52 procedures did not achieve meaningful autonomy with laparoscopic cholecystectomy. Because this is the most frequently performed procedure by general surgery residents, procedural numbers may be potentially less of an impediment in attaining procedural autonomy than with other procedures, and other factors may have impeded resident progression to meaningful autonomy. Although individual resident ability may be an important factor, the level of autonomy an individual resident achieves may also be influenced by faculty rater stringency, faculty willingness to grant autonomy, programmatic culture, programmatic patient populations, and other factors.
      With open inguinal hernia, ventral hernia, and partial colectomy, the target numbers identified using the “best-fit” process need to be further refined once additional observations are available. With open inguinal hernia and ventral hernia, the target numbers did not clearly identify a transition point beyond which the majority of residents achieved meaningful autonomy. For partial colectomy, technically this transition point was identified by the target number, although the majority of residents who achieved meaningful autonomy beyond the target number consisted of only 2 residents. These analyses may be further enhanced by increased data collection.
      Another concern is that the procedural target numbers identified using this approach may be higher than the numbers that most residents may experience during their entire residency. We have previously shown that in the last 6 months of their training PGY5 residents achieve meaningful autonomy, with only 69% of the core general surgery procedures they perform,
      • George B.C.
      • Bohnen J.D.
      • Williams R.G.
      • et al.
      Readiness of US general surgery residents for independent practice.
      although higher percentages (≥85%) are achieved with the 4 most frequently performed procedures. For less frequently performed core procedures, it may be unrealistic to expect all surgical trainees to achieve these defined targets during their training.
      • Abdelrahman T.
      • Long J.
      • Egan R.
      • Lewis W.G.
      Operative experience vs. competence: a curriculum concordance and learning curve analysis.
      • De Siqueira J.R.
      • Gough M.J.
      Correlation between experience targets and competence for general surgery certification.
      One limitation of this analysis is the lack of a widely accepted objective standard for defining procedural competence. The criteria we have chosen for this study are based on the mean level of autonomy achieved by the resident performing the procedure in question during the defined study period. Because the autonomy level achieved by the resident is entirely dependent on the autonomy level granted by the supervising faculty, higher levels of autonomy are typically granted only when the resident has gained the trust of the supervising faculty based on their demonstration of competence. One might argue that there is no stronger endorsement of a resident's competence by a supervising surgeon than to let the resident do the procedure with a high level of autonomy. Still, other factors, including procedural complexity, programmatic culture, and individual faculty demeanor, may also influence the amount of autonomy granted, and this may confound our results.
      • George B.C.
      • Bohnen J.D.
      • Williams R.G.
      • et al.
      Readiness of US general surgery residents for independent practice.
      There continues to be significant controversy regarding what autonomy level is acceptable for residents in training.
      • Ohye R.G.
      • Jaggers J.J.
      • Sade R.M.
      Most surgeons in training programs allow residents to operate on their patents to satisfy board requirements?.
      • Saliba A.N.
      • Taher A.T.
      • Tamim H.
      • et al.
      Impact of resident involvement in surgery (IRIS-NSQIP): looking at the bigger picture based on the American College of Surgeons-NSQIP database.
      • Meyerson S.L.
      • Teitelbaum E.N.
      • George B.C.
      • Schuller M.C.
      • DaRosa D.A.
      • Fryer J.P.
      Defining the autonomy gap: when expectations do not meet reality in the operating room.
      • Safavi A.
      • Lai S.
      • Butterworth S.
      • Hameed M.
      • Schiller D.
      • Skarsgard E.
      Does operative experience during residency correlate with reported competency of recent general surgery graduates?.
      • Fryer J.P.
      • Teitelbaum E.N.
      • George B.C.
      • et al.
      Effect of ongoing assessment of resident operative autonomy on the operating room environment.
      Some express significant concern about granting residents significant autonomy during their training for patient safety reasons, because they are less experienced and therefore less competent than their supervising faculty surgeon. Others believe that granting supervised autonomy during training to ensure residents are ready for independent practice should be considered more acceptable than sending them out into independent practice unprepared and that increasing the focus on progressive autonomy during training may lead to more residents achieving meaningful autonomy during their training.
      We have used a mean Zwisch level ≥3 to define meaningful autonomy, which essentially means that the resident has achieved either Passive Help (PH) or Supervision Only (SO) for the majority of times they have performed that procedure during the study period. Although SO may be a more ideal goal for trainees because it most closely mimic independent practice, PH also reflects meaningful resident autonomy and may be a more comfortable scenario for supervising faculty. Furthermore SO requires that someone other than the supervising faculty assists the resident with the procedure, which is not always logistically possible. Furthermore, if SO was considered the required criteria for meaningful autonomy, the procedural number requirements would be even higher and the number of residents who achieve those levels would be lower. For these reasons we have included both PH and SO in the definition of meaningful autonomy.
      Regardless, the criteria we have used in this study to define meaningful autonomy may reflect a higher degree of autonomy than is currently being granted to most surgical residents. It is possible that the criteria we have used to define meaningful autonomy for this study is too stringent and may exclude many residents who are actually competent. Perhaps instead of basing the threshold for meaningful autonomy on achieving PH or SO for the majority of procedures performed during a specific period, it may be reasonable to consider basing it on achieving PH or SO for a designated minimal number of performance assessments (i.e. 3), as is the established standard in some other countries.
      • Abdelrahman T.
      • Long J.
      • Egan R.
      • Lewis W.G.
      Operative experience vs. competence: a curriculum concordance and learning curve analysis.
      • De Siqueira J.R.
      • Gough M.J.
      Correlation between experience targets and competence for general surgery certification.
      Of course these or other less stringent criteria would need to be validated in a separate study.
      Although we have focused on how a trainee's competence with a specific operative procedure may be influenced by their previous experience with that procedure, we have not looked at the impact of experience with other procedures. This impact of transfer of surgical skills
      • Varas J.
      • Mejia R.
      • Riquelme A.
      • et al.
      Significant transfer of surgical skills obtained with an advanced laparoscopic training program to a laparoscopic jejunojejunostomy in a live porcine model: feasibility of learning advanced laparoscopy in a general surgery residency.
      from other surgical procedures has never been adequately studied in operative procedures performed during residency training, but is the basis for the ACGME-defined category requirements. These requirements stipulate that surgical residents perform a minimum number of operative procedures from a defined group of procedures before they are allowed to take their ABS qualifying exams. Specifically, transfer of surgical skills may be a factor for residents who obtain the majority of their procedural experience for a specific procedure during the PGY4 or PGY5 year. Due to a greater overall procedural experience preceding these final years of surgical residency training, these residents may be able to more rapidly achieve meaningful autonomy with a specific procedure than residents who are less experienced.
      Ultimately the purpose of establishing target numbers for procedures needs to be carefully re-evaluated. If the intent is to distinguish those in training who have achieved an acceptable level of autonomy from those who have not for the purposes of determining who is eligible for certification, our data suggest that this may work for only a very limited number of individual core procedures. With most procedures, however, high levels of autonomy are not currently achieved by most residents, and those who do achieve them often have done so after performing procedural numbers that are higher than many residents may have access to.
      However, if target numbers based on individual procedures or defined categories are used as training milestones to trigger more in-depth review of individual residents' operative competence, this may be an effective way to guide subsequent training for residents by directing those who have achieved competence in a specific procedure to shift their focus to other procedures and allowing those who have not yet achieved competence to obtain extra experience with the procedure in question. Identification of procedural numbers that are associated with the earliest and most significant transitions to meaningful procedural autonomy in the resident population, that is, both the inception and transition points, as we have done here (Figs 2A–2E and Table) may be helpful in guiding this process.
      Our data therefore argue against general use of a fixed procedural number as a surrogate for declaring procedural competence for specific procedures. Although procedural numbers are easy to monitor and evaluate, if these numbers are not used in conjunction with a system that accurately monitors progressive competence, they can be misleading in determining who is ready for independent practice. There appears to be some correlation between procedural numbers and progressive autonomy, but there is such a high degree of variability between individual residents that it is likely not possible to select a single procedural target number that would reliably be used to predict procedural competence for all trainees. Therefore serious consideration needs to be given to converting to a system that is based primarily on assessment of competence in the clinical workplace rather than based solely on a procedural target number.
      Finally, with ongoing monitoring of residents' progressive autonomy, problems can be identified at the level of individual residents, at the level of programs, and at a national level. To address problems that are identified at any of these levels, interventions can be deployed and their impact monitored. Faculty can also be monitored to identify problems in granting autonomy and initiate appropriate faculty development initiatives to resolve these problems.

      References

        • Bell R.H.
        • Biester T.W.
        • Tabuenca A.
        • et al.
        Operative experience of residents in US general surgery programs: a gap between expectation and experience.
        Ann Surg. 2009; 249: 719-724
        • Sachdeva A.K.
        • Flynn T.C.
        • Brigham T.P.
        • et al.
        Interventions to address challenges associated with the transition from residency training to independent surgical practice.
        Surgery. 2014; 155: 867-882
        • Kairys J.C.
        • McGuire K.
        • Crawford A.G.
        • Yeo C.J.
        Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees?.
        J Am Coll Surg. 2008; 206: 804
        • Damadi A.
        • Davis A.T.
        • Saxe A.
        • Apelgren K.
        ACGME duty-hour restrictions decrease resident operative volume: a 5-year comparison at an ACGME-accredited university general surgery residency.
        J Surg Educ. 2007; 64: 256
        • Fronza J.S.
        • Prystowsky J.P.
        • DaRosa D.
        • Fryer J.P.
        Surgical residents' perception of competence and relevance of the clinical curriculum to future practice.
        J Surg Educ. 2012; 69: 792-797
        • Bucholz E.M.
        • Sue G.R.
        • Yeo H.
        • Roman S.A.
        • Bell Jr, R.H.
        • Sosa J.A.
        Our trainees' confidence: results from a national survey of 4136 US general surgery residents.
        Arch Surg. 2011; 146: 907-914
        • Mattar S.G.
        • Alseidi A.A.
        • Jones D.B.
        • et al.
        General surgery residency inadequately prepares trainees for fellowship: results of a survey of fellowship program directors.
        Ann Surg. 2013; 258: 440-449
        • Williams R.G.
        • Chen X.P.
        • Sanfey H.
        • Markwell S.J.
        • Mellinger J.D.
        • Dunnington G.L.
        The measured effect of delay in completing operative performance ratings on clarity and detail of ratings assigned.
        J Surg Educ. 2014; 71: 132-138
        • Nygaard R.M.
        • Daly S.R.
        • Van Camp J.M.
        General surgery resident case logs: do they accurately reflect resident experience?.
        J Surg Educ. 2015; 72: e178-e183
        • Abdelrahman T.
        • Long J.
        • Egan R.
        • Lewis W.G.
        Operative experience vs. competence: a curriculum concordance and learning curve analysis.
        J Surg Educ. 2016; 73: 694-698
        • De Siqueira J.R.
        • Gough M.J.
        Correlation between experience targets and competence for general surgery certification.
        Br J Surg. 2016; 103: 921-927
        • Ten Cate O.
        Trust, competence, and the supervisor's role in postgraduate training.
        BMJ. 2006; 333: 748-751
        • Hauer K.E.
        • Ten Cate O.
        • Boscardin C.
        • Irby D.M.
        • Iobst W.
        • O'Sullivan P.S.
        Understanding trust as an essential element of trainee supervision and learning in the workplace.
        Adv Health Sci Educ Theory Pract. 2014; 19: 435-456
        • DaRosa D.A.
        • Zwischenberger J.B.
        • Meyerson S.L.
        • et al.
        A theory-based model for teaching and assessing residents in the operating room.
        J Surg Educ. 2013; 70: 24-30
        • George B.C.
        • Teitelbaum E.N.
        • Meyerson S.L.
        • et al.
        Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance.
        J Surg Educ. 2014; 71: e90-e96
        • Bohnen J.D.
        • George B.C.
        • Williams R.G.
        • et al.
        The feasibility of real-time intraoperative performance assessment with SIMPL (System for Improving and Measuring Procedural Learning): early experience from a multi-institutional trial.
        J Surg Educ. 2016; 73: e118-e130
        • George B.C.
        • Bohnen J.D.
        • Williams R.G.
        • et al.
        Readiness of US general surgery residents for independent practice.
        Ann Surg. 2017; 266: 582-594
        • George B.C.
        • Teitelbaum E.N.
        • Darosa D.A.
        • et al.
        Duration of faculty training needed to ensure reliable or performance ratings.
        J Surg Educ. 2013; 70: 703-708
        • Resident Education SC
        SCORE: Curriculum Outline for General Surgery 2016–2017.
        (Copyright/Published 2016)
        • Ohye R.G.
        • Jaggers J.J.
        • Sade R.M.
        Most surgeons in training programs allow residents to operate on their patents to satisfy board requirements?.
        Ann Thorac Surg. 2016; 101: 18-23
        • Saliba A.N.
        • Taher A.T.
        • Tamim H.
        • et al.
        Impact of resident involvement in surgery (IRIS-NSQIP): looking at the bigger picture based on the American College of Surgeons-NSQIP database.
        J Am Coll Surg. 2016; 222: 30-40
        • Meyerson S.L.
        • Teitelbaum E.N.
        • George B.C.
        • Schuller M.C.
        • DaRosa D.A.
        • Fryer J.P.
        Defining the autonomy gap: when expectations do not meet reality in the operating room.
        J Surg Educ. 2014; 71: e64-e72
        • Safavi A.
        • Lai S.
        • Butterworth S.
        • Hameed M.
        • Schiller D.
        • Skarsgard E.
        Does operative experience during residency correlate with reported competency of recent general surgery graduates?.
        Can J Surg. 2012; 55: S171-S177
        • Fryer J.P.
        • Teitelbaum E.N.
        • George B.C.
        • et al.
        Effect of ongoing assessment of resident operative autonomy on the operating room environment.
        J Surg Educ. 2017; (pii: S1931-7204(16)30329-4)https://doi.org/10.1016/j.jsurg.2016.11.018
        • Varas J.
        • Mejia R.
        • Riquelme A.
        • et al.
        Significant transfer of surgical skills obtained with an advanced laparoscopic training program to a laparoscopic jejunojejunostomy in a live porcine model: feasibility of learning advanced laparoscopy in a general surgery residency.
        Surg Endosc. 2012; 26: 3486-3494