Advertisement

What factors influence attending surgeon decisions about resident autonomy in the operating room?

Published:September 23, 2017DOI:https://doi.org/10.1016/j.surg.2017.07.028

      Background

      Educating residents in the operating room requires balancing patient safety, operating room efficiency demands, and resident learning needs. This study explores 4 factors that influence the amount of autonomy supervising surgeons afford to residents.

      Methods

      We evaluated 7,297 operations performed by 487 general surgery residents and evaluated by 424 supervising surgeons from 14 training programs. The primary outcome measure was supervising surgeon autonomy granted to the resident during the operative procedure. Predictor variables included resident performance on that case, supervising surgeon history with granting autonomy, resident training level, and case difficulty.

      Results

      Resident performance was the strongest predictor of autonomy granted. Typical autonomy by supervising surgeon was the second most important predictor. Each additional factor led to a smaller but still significant improvement in ability to predict the supervising surgeon's autonomy decision. The 4 factors together accounted for 54% of decision variance (r = 0.74).

      Conclusion

      Residents' operative performance in each case was the strongest predictor of how much autonomy was allowed in that case. Typical autonomy granted by the supervising surgeon, the second most important predictor, is unrelated to resident proficiency and warrants efforts to ensure that residents perform each procedure with many different supervisors.
      Supervising surgeons must balance patient safety, operating room efficiency, and resident learning needs when deciding how much guidance to provide residents during operative procedures. Progressively less guidance or, inversely, progressively more autonomy is required if residents are to learn to perform procedures independently by the completion of training. The precise means by which supervising surgeons make these decisions remains a mystery, yet an improved understanding of these decision processes could help identify ways to enhance the quality of surgical training. With this goal in mind, several prior research efforts have investigated factors that influence surgical supervisors' guidance of residents in the operating room. Chen et al
      • Chen X.P.
      • Williams R.G.
      • Sanfey H.A.
      • Dunnington G.L.
      How do supervising surgeons evaluate guidance provided in the operating room?.
      reported that residents with more years of surgical training were afforded more autonomy by the supervisor and that the quality of the trainees' performance was a determinant of the amount of autonomy afforded to them. These investigators also observed that the amount of autonomy individual attending surgeons provided to residents in the operating room depended to a great extent on the attending surgeon. However, the study was based on a single residency training program and, as a result, the number of trainees and attending surgeons was small. In addition, the study was limited to trainee performance of 5 common operative procedures. Most importantly, the study investigated the effects of each variable separately. Because these variables are correlated with each other (eg, the quality of the performance is correlated with years of training), the unique contribution of each variable needs to be determined controlling for the impact of other variables studied.
      In a similar study, George et al
      • George B.C.
      • Teitelbaum E.N.
      • Meyerson S.L.
      • Schuller M.C.
      • DaRosa D.A.
      • Petrusa E.R.
      • et al.
      Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance.
      investigated the guidance behavior of 27 supervising surgeons in a single institution who rated 31 resident performances of 1,490 operations covering 127 different procedures. They found that the amount of autonomy afforded residents increased with number of years of training, except that the average amount of autonomy afforded to fourth- and fifth-year trainees did not differ. The difficulty of the case was inversely associated with the level of autonomy. Supervising surgeons afforded more guidance/less autonomy to trainees for cases viewed as more difficult. As part of their study, George et al investigated the relationship between rating of operative performance quality (using 2 different operative performance instruments) and the amount of autonomy afforded to the trainee. They found that better operative performance led the supervising surgeon to afford the resident more autonomy. In this study, surgeons who rated the operative performance were blinded to the autonomy rating assigned by the supervising surgeon.
      Torbeck et al
      • Torbeck L.
      • Williams R.G.
      • Choi J.
      • Schmitz C.C.
      • Chipman J.G.
      • Dunnington G.L.
      How much guidance is given in the operating room? Factors influencing faculty self-reports, resident perceptions, and faculty/resident agreement.
      approached the study of guidance from a different perspective. In a 2-program study, these investigators explored the opinions of supervising surgeons and trainees about supervising surgeon operating room guidance practices. Respondents reported that the level of resident training affected the amount of guidance provided. Faculty supervisors and residents agreed on this finding. However, the residents thought that guidance practices changed less with increased training than did the faculty supervisors.
      This study is designed to add to the understanding of what motivates attending surgeons to allow more operating room autonomy by investigating 4 factors likely to influence the level of autonomy granted. The study expands the understanding of factors established in earlier research by 1) increasing the number of training programs studied to 14 and 2) increasing the number of attending surgeons, residents, and procedures studied. It adds investigation of the typical guidance practices of each supervising surgeon as a potential determinant of the autonomy afforded for individual resident operative performances, and most importantly it investigates the effect of each variable while controlling for the effects of the other variables studied. The primary outcome measure was the attending surgeon's decision regarding the level of autonomy afforded during the case as reported by the attending surgeon using a previously described smartphone-based operative performance assessment system known as SIMPL.
      • Bohnen J.D.
      • George B.C.
      • Williams R.G.
      • Schuller M.C.
      • DaRosa D.A.
      • Torbeck L.
      • et al.
      The feasibility of real-time intraoperative performance assessment with SIMPL (system for improving and measuring procedural learning): early experience from a multi-institutional trial.
      The 4 influencing factors investigated included the 1) resident's operative performance during the case as reported by the attending surgeon, 2) attending surgeon's typical operating room guidance practices, 3) difficulty of the case, and 4) resident experience level (postgraduate year of surgical training). A better understanding of these 4 factors will help attending surgeons and program administrators balance multiple important goals while ensuring that surgical residents are afforded opportunities for progressively independent practice.

      Methods

       Data

      The results of this study are based on supervising surgeon ratings of categorical general surgery residents for operations performed between September 2015 and June 30, 2016 in 14 residency training programs. The attending surgeons were aware of each resident's level of training (postgraduate training year). These operations included the range of procedures performed by residents in these programs and rated by attending surgeons. For each operative performance, the attending surgeon reported the level of autonomy afforded to the resident using the 4-point Zwisch scale: 1) Show and Tell (the resident observed and learned), 2) Active Help (the attending surgeon alternated between showing and assisting and provided significant guidance throughout the case), 3) Passive Help (the attending surgeon primarily functioned as an assistant to the resident and provided guidance only when needed), and 4) Supervision Only (the attending surgeon supervised and provided guidance only on request from the resident or for patient safety).
      The inverse of guidance by the surgeon is autonomy granted to the resident. As surgeon guidance increases, resident autonomy decreases. There will be occasions in this article where it makes more sense to talk about resident autonomy and other occasions where it makes more sense to talk about guidance by the supervising surgeon. Because we are defining terms, we also make a functional distinction between supervision and guidance. The attending surgeon always provides supervision but chooses to increase or decrease specific guidance on a case-by-case or case-element basis to aid the resident in improving current and subsequent performance of the procedure and to safeguard the patient. The attending surgeon also judged and rated the residents' operative performances using the 5-point SIMPL performance scale: 1) critical deficiencies/unprepared, 2) unfamiliar with the procedure, 3) intermediate level performance, 4) practice ready performance, and 5) exceptional performance.
      The relative difficulty of the case was rated by the supervising surgeon using a 3-point scale: 1) one of the easiest third of cases experienced for this specific procedure, 2) middle third of cases experienced for this specific procedure, and 3) one of the most difficult third of cases experienced for this specific procedure. This is a measure of the difficulty of the case relative to previous experiences of that attending surgeon performing that procedure, not a measure of the complexity of the procedure. For example, Whipple procedures are more complex than laparoscopic appendectomies. Difficulty as measured with this instrument would indicate how difficult this laparoscopic appendectomy was relative to other laparoscopic appendectomies. More will be said about this distinction in the discussion. The scales for all 3 questions in SIMPL are explained in more depth in Bohnen et al.
      • Bohnen J.D.
      • George B.C.
      • Williams R.G.
      • Schuller M.C.
      • DaRosa D.A.
      • Torbeck L.
      • et al.
      The feasibility of real-time intraoperative performance assessment with SIMPL (system for improving and measuring procedural learning): early experience from a multi-institutional trial.

       Primary outcome measure

      Supervising surgeon ratings regarding the amount of autonomy/independence granted to the resident during performance of an operative procedure.

       Predicting (influencing) variables

      Quality of the resident's performance, individual faculty member history with granting autonomy, resident experience measured by number of years in postgraduate surgical training, and difficulty of the case relative to other examples of the same procedure. The typical guidance practice for each attending surgeon was determined by computing the average guidance rating assigned by the attending surgeon to fourth-year residents during the 9-month study period. This approach controlled for confounding of the typical guidance behavior measure that would have occurred using guidance ratings for multiple years because some supervising surgeons typically supervise residents at selected training levels (eg, second-year residents), and this fact would influence the typical guidance behavior measure for those supervising surgeons. Supervising surgeons who rated <3 fourth-year trainees were eliminated from the analysis.

       Data analysis

      Pearson product moment correlations were computed to determine the bivariate correlations among each pair of variables. Partial correlations were performed to determine the unique contribution of each predictor (variable) while statistically controlling for the contribution of other predictors (variables). Multiple linear regression analysis was performed to determine the joint ability of the 4 predictors to predict the autonomy decision (primary outcome measure).

       Ethics approval

      The Human Subjects Review Board at each of the participating institutions approved use of data for this study.

      Results

      In the study, 7,297 operative performance ratings were collected from 424 faculty raters for 487 general surgery categorical residents in 14 residency-training programs. The Figure provides the bivariate correlations for each predictor with the autonomy decision made by the supervising surgeon as well as correlations among the 4 predictor variables. Results demonstrated that the rated overall quality of the operative performance by the resident was the strongest predictor of the supervisor's autonomy decision, followed by typical rater guidance behavior. Trainee experience, measured as postgraduate training year level, was the third most important predictor of the reported amount of guidance. As expected, residents with more experience received higher autonomy ratings. Relative case difficulty played the least important role in the supervisor's autonomy decision. More guidance, and thus less autonomy, was afforded to residents for cases deemed relatively more difficult than other, similar cases.
      Figure thumbnail gr1
      FigIntercorrelations among operative performance quality ratings for the target case, typical guidance habits of supervising surgeon (mean reported guidance level for fourth-year resident performances), trainee experience (postgraduate year of training), case difficulty, and supervising surgeons guidance rating for the target case.
      The intercorrelations among the predictor variables indicate the degree of overlap among each pair of predictor variables. This overlap complicates the interpretation of simple correlations between individual predictors and the attending surgeon's guidance behavior. The Table provides partial correlations that indicate the unique contribution of each of the 4 factors, statistically controlling for the effect of other predictor variables used in this study. These results confirm that the operative performance of the trainee on the observed case is the strongest predictor of the amount of autonomy granted by the supervising surgeon (partial correlation = 0.57, guidance decision variance accounted for by this factor = 47.7%). Again, typical guidance behavior by this attending surgeon (ie, typical Zwisch rating) is the second most important determinant of the level of autonomy granted to the resident by the attending surgeon (partial r = 0.30, variance accounted for 9%). Relative case difficulty accounted for 4.4% of decision variance, and trainee years of experience accounted for 2% of decision variance.
      TableUnique effect of each factor on autonomy decisions statistically controlling for the effect of all other factors used in this study
      Factors (variables) used to predict autonomy decision in this studyCorrelation of variable with autonomy decisions controlling for effect of all other variables (partial correlation)Percent of autonomy decision variance uniquely accounted for by named variable
      Quality of resident's operative performance on this case0.5747.7
      Typical rater autonomy decisions0.309.0
      Case difficulty−0.214.4
      Trainee experience0.142.0
      Multiple linear regression analysis was used to determine the combined ability of all 4 factors to predict the attending surgeon's guidance/autonomy decision. The 4 factors together accounted for 54% of the expert surgeon supervisor's autonomy decision variance (r = 0.74).
      Our results indicate that, of all the factors evaluated in this study, the trainee's performance on the case being observed is the single factor that most correlates with supervising surgeons' decisions regarding the level of autonomy granted to the trainee. Trainee performance during the case is clearly an appropriate factor to consider in making the autonomy decision and probably should be the predominant one.
      The supervising surgeon's typical guidance habits affect autonomy decisions for individual resident performances and are partially due to factors other than resident proficiency in performing the procedure. They are determined by personality characteristics, habits, beliefs and experience of the supervising surgeon. As can be seen in the Table, this is the second most important predictor of the autonomy decision, accounting for 9% of the variance in the autonomy decision-making process. This is a large enough factor to warrant efforts to ensure that residents not only have the opportunity to perform a wide range of different procedures, but that they have the opportunity to perform each procedure under the supervision of a wide range of different supervisors. We think that monitoring and planning the mix of supervising surgeons will be an easier and more effective way of achieving this goal than will efforts to change the guidance behavior of supervising surgeons through faculty development. However, faculty development efforts should be made to remind faculty that the provision of safe, graduated autonomy to residents is critical to assuring resident ability to perform procedures independently by the completion of training. Attending surgeons who frequently achieve this goal should be acknowledged, while attending surgeons who frequently do not provide residents with appropriate autonomy may benefit from faculty development efforts.
      Difficulty of the case also is an appropriate factor to consider when making a decision about the level of autonomy afforded to trainees. Relative case difficulty did influence autonomy decisions in this study, although to a much lesser degree than trainee performance on the case. We were only able to determine the effect of overall case difficulty on the supervisor's autonomy decision due to the nature of the question we used. However, the study by Torbeck et al
      • Torbeck L.
      • Williams R.G.
      • Choi J.
      • Schmitz C.C.
      • Chipman J.G.
      • Dunnington G.L.
      How much guidance is given in the operating room? Factors influencing faculty self-reports, resident perceptions, and faculty/resident agreement.
      provides additional evidence that surgical supervisors vary the amount of autonomy afforded to trainees at various points in the performance depending on the criticality/risk of the procedural step being performed.
      It is less clear whether trainee experience, defined in this study as years of surgical training, should influence the supervisor's autonomy decision when the quality of actual trainee case performance is being observed. The fact that years of training influence the guidance provided under these circumstances probably reflects an increased degree of supervisor caution with residents who have less experience. A logical extension of this research would be to investigate other factors that lead to increased attending surgeon caution in allowing autonomy to the resident. Factors to consider would include time of the academic year when the case was done and prior experience working with the resident.
      Hauer et al
      • Hauer K.E.
      • Ten Cate O.
      • Boscardin C.
      • Irby D.M.
      • Iobst W.
      • O'Sullivan P.S.
      Understanding trust as an essential element of trainee supervision and learning in the workplace.
      provided a literature-based case arguing that trust is a critical determinant of a supervisor's decision regarding how much autonomy to provide to medical trainees. They maintained that 5 factors influence trust of residents: characteristics of the supervisor, the trainee, the relationship between the 2, the nature of the task, and the context in which the task is performed. They elaborated on specific determinants in each of these categories. Most of the work they cited is in fields such as internal medicine, where the supervisor does not observe all aspects of the performance. This influences their description of entrustment decision determinants. Sandhu et al
      • Sandhu G.
      • Magas C.P.
      • Robinson A.B.
      • Scally C.P.
      • Minter R.M.
      Progressive entrustment to achieve resident autonomy in the operating room: a national qualitative study with general surgery faculty and residents.
      conducted a similar study interviewing surgeons regarding how they increasingly step back and safely delegate autonomy to trainees. To the best of our knowledge ours is the first study of factors that influence entrustment decisions and behaviors in US operating rooms based directly on attending surgeon reports of their guidance behavior for individual cases performed by residents and of selected factors (quality of resident performance, relative case difficulty) surrounding each case performance. As such, this study builds upon and broadens the work of Hauer et al,
      • Hauer K.E.
      • Ten Cate O.
      • Boscardin C.
      • Irby D.M.
      • Iobst W.
      • O'Sullivan P.S.
      Understanding trust as an essential element of trainee supervision and learning in the workplace.
      Sandhu et al,
      • Sandhu G.
      • Magas C.P.
      • Robinson A.B.
      • Scally C.P.
      • Minter R.M.
      Progressive entrustment to achieve resident autonomy in the operating room: a national qualitative study with general surgery faculty and residents.
      and Torbeck et al.
      • Torbeck L.
      • Williams R.G.
      • Choi J.
      • Schmitz C.C.
      • Chipman J.G.
      • Dunnington G.L.
      How much guidance is given in the operating room? Factors influencing faculty self-reports, resident perceptions, and faculty/resident agreement.
      The General Surgery Milestones designed for use in the new generation of Accreditation Council of Graduate Medical Education competency based training programs (Cogbill et al
      • Cogbill T.H.
      • Malangoni M.A.
      • Potts J.R.
      • Valentine R.J.
      The general surgery milestones project.
      ) provide a framework for the assessment of general surgery resident development that is essentially a set of descriptors and targets for general surgery resident performance as the resident moves from entry into residency through stages to the completion of residency training. The general surgery milestones on performance of operations and procedures (Patient Care 3) contain 4 performance levels. The Zwisch Autonomy measure used in SIMPL plays a key role in measuring general surgery resident transition from Patient Care 3 Level 2 “moves through portions of common operations without coaching” through Level 3 “moves through the steps of most operations without much coaching” to Level 4 “guides the conduct of most operations.” This Zwisch Autonomy measure assumes that the level of autonomy granted is determined solely by the resident's ability to perform this procedure. The present study cautions that other factors (most importantly typical autonomy granted by the supervising surgeon as measured by individual faculty member history with granting autonomy) influence the level of autonomy granted and that some of those factors are unrelated to resident performance ability.
      Additionally, this study quantified the magnitude of the effect of those factors. The results make it clear that residency program personnel must take systematic steps to assure that many different surgeons supervise each resident in the operating room to assure that such autonomy measures provide a valid estimate of resident operative performance ability.
      Although our study investigated 4 factors that influence each other and the autonomy/guidance decision, there are other factors to be investigated. The 4 factors evaluated in this study accounted for only 54% of the supervising surgeons' autonomy decision variance. Key factors that we did not investigate include
      • 1)
        The supervisor's past experience operating with the resident performing this procedure (familiarity with the performance capabilities of the resident);
      • 2)
        The extent to which supervisors afford less autonomy to residents when performing more complex or less commonly performed procedures (eg, esophagogastrectomy, hepatectomy, retroperitoneal sarcoma, Whipple procedure) as opposed to more straightforward procedures such as cholecystectomy, hernia repair, and colectomy;
      • 3)
        The current SIMPL system allows a single post hoc rating of guidance for the performance as a whole. In actuality, guidance decisions and behaviors change dynamically from one step in the procedure to the next based on complexity and inherent risk of the step;
      • 4)
        The attending surgeon's experience as a surgeon, which is likely to influence comfort in delegating responsibility to the resident; and
      • 5)
        The perioperative work demands, on the attending surgeon, the resident and the operating room.
      Each of these factors has the potential to influence attending surgeon guidance behavior and influence the resident's opportunity to practice performing the procedure independently.

      References

        • Chen X.P.
        • Williams R.G.
        • Sanfey H.A.
        • Dunnington G.L.
        How do supervising surgeons evaluate guidance provided in the operating room?.
        Am J Surg. 2012; 203: 44-48
        • George B.C.
        • Teitelbaum E.N.
        • Meyerson S.L.
        • Schuller M.C.
        • DaRosa D.A.
        • Petrusa E.R.
        • et al.
        Reliability, validity, and feasibility of the Zwisch scale for the assessment of intraoperative performance.
        J Surg Educ. 2014; 71: e90-e96
        • Torbeck L.
        • Williams R.G.
        • Choi J.
        • Schmitz C.C.
        • Chipman J.G.
        • Dunnington G.L.
        How much guidance is given in the operating room? Factors influencing faculty self-reports, resident perceptions, and faculty/resident agreement.
        Surgery. 2014; 156: 797-803
        • Bohnen J.D.
        • George B.C.
        • Williams R.G.
        • Schuller M.C.
        • DaRosa D.A.
        • Torbeck L.
        • et al.
        The feasibility of real-time intraoperative performance assessment with SIMPL (system for improving and measuring procedural learning): early experience from a multi-institutional trial.
        J Surg Educ. 2016; 73: e118-e130
        • Hauer K.E.
        • Ten Cate O.
        • Boscardin C.
        • Irby D.M.
        • Iobst W.
        • O'Sullivan P.S.
        Understanding trust as an essential element of trainee supervision and learning in the workplace.
        Adv Health Sci Educ Theory Pract. 2014; 19: 435-456
        • Sandhu G.
        • Magas C.P.
        • Robinson A.B.
        • Scally C.P.
        • Minter R.M.
        Progressive entrustment to achieve resident autonomy in the operating room: a national qualitative study with general surgery faculty and residents.
        Ann Surg. 2017; 265: 1134-1140
        • Cogbill T.H.
        • Malangoni M.A.
        • Potts J.R.
        • Valentine R.J.
        The general surgery milestones project.
        J Am Coll Surg. 2014; 218: 1056-1062