Advertisement

Early impact of the 2011 ACGME duty hour regulations on surgical outcomes

      Background

      In 2011, the Accreditation Council for Graduate Medical Education (ACGME) implemented additional restrictions on resident work hours. Although the impact of these restrictions on the education of surgical trainees has been examined, the effect on patient safety remains poorly understood.

      Methods

      We used national Medicare Claims data for patients undergoing general (n = 1,223,815) and vascular (n = 475,262) surgery procedures in the 3 years preceding the duty hour changes (January, 2009–June, 2011) and the 18 months thereafter (July, 2011–December, 2012). Hospitals were stratified into quintiles by teaching intensity using a resident to bed ratio. We utilized a difference-in-differences analytic technique, using nonteaching hospitals as a control group, to compare risk-adjusted 30-day mortality, serious morbidity, readmission, and failure to rescue (FTR) rates before and after the duty hour changes.

      Results

      After duty hour reform, no changes were seen in the measured outcomes when comparing teaching with nonteaching hospitals. Even when stratifying by teaching intensity, there were no differences. For example, at the highest intensity teaching hospitals (resident/bed ratio of ≥0.6), mortality rates before and after the duty hour changes were 4.2% and 4.0%, compared with 4.7% and 4.4% for nonteaching hospitals (relative risk [RR], 0.98; 95% CI, 0.89–1.07). Similarly, serious complication (RR, 1.02; 95% CI, 0.98–1.06), FTR (RR, 0.95; 95% CI, 0.87–1.04), and readmission (odds ratio, 1.00; 95% CI, 0.96–1.03) rates were unchanged.

      Conclusion

      In Medicare beneficiaries undergoing surgery at teaching hospitals, outcomes have not improved since the 2011 ACGME duty hour regulations.
      Concerns for patient safety have led to several landmark changes in the structure of resident training over the past decade. In July 2003, the Accreditation Council for Graduate Medical Education (ACGME) enacted the first restrictions to resident duty hours.
      • Philibert I.
      • Friedmann P.
      • Williams W.T.
      ACGME Work Group on Resident Duty Hours. Accreditation Council for Graduate Medical Education
      New requirements for resident duty hours.
      These restrictions limited residents to an average of ≤80 hours per week in the hospital, limited the frequency of on-call shifts to no more than 1 in 3 nights, and mandated ≥1 day off in 7. In 2008, the Institute of Medicine provided further recommendations for resident training, stressing the impact of prolonged and variable shift lengths on trainees' fatigue, as well as the potential link between provider fatigue and patient safety.
      • Ulmer C.
      • Miller Wolman D.
      • Johns M.M.E.
      Resident duty hours: enhancing sleep, supervision, and safety. Institute of Medicine report brief.
      • Iglehart J.K.
      Revisiting duty-hour limits–IOM recommendations for patient safety and resident education.
      In light of this, additional work hour restrictions were enacted in July, 2011.
      • Nasca T.J.
      • Day S.H.
      • Amis Jr., E.S.
      ACGME Duty Hour Task Force
      The new recommendations on duty hours from the ACGME Task Force.
      These restrictions limit postgraduate year 1 (PGY-1) trainees to a 16-hour maximum shift, require a minimum rest period between shifts of 8 hours, and increase the specificity of supervisory requirements.
      Whether these most recent changes have had the intended effect on patient outcomes remains unclear. There is evidence to suggest that these changes may have benefits, largely by reducing trainee fatigue. A number of studies have shown increased medical error rates by fatigued trainees,
      • Barger L.K.
      • Ayas N.T.
      • Cade B.E.
      • Cronin J.W.
      • Rosner B.
      • Speizer F.E.
      • et al.
      Impact of extended-duration shifts on medical errors, adverse events, and attentional failures.
      • Philibert I.
      Sleep loss and performance in residents and nonphysicians: a meta-analytic examination.
      and have demonstrated that targeted interventions to reduce fatigue can decrease error rates.
      • Landrigan C.P.
      • Rothschild J.M.
      • Cronin J.W.
      • Kaushal R.
      • Burdick E.
      • Katz J.T.
      • et al.
      Effect of reducing interns' work hours on serious medical errors in intensive care units.
      However, with the 2011 regulations, there is concern that the shortened shifts have necessitated an increase in transitions of care and patient handoffs as hospitals move to a shift-based care system.
      • Van Eaton E.G.
      • Tarpley J.L.
      • Solorzano C.C.
      • Cho C.S.
      • Weber S.M.
      • Termuhlen P.M.
      Resident education in 2011: three key challenges on the road ahead.
      The increase in handoffs and potential for communication errors may offset any potential gains from decreased provider fatigue. Although a recent study analyzed the impact of these changes on patient outcomes, this study was limited in that it focused on only a small proportion of surgical residencies, and focused only on residency programs participating in the National Surgical Quality Improvement Program (NSQIP).
      • Rajaram R.
      • Chung J.W.
      • Jones A.T.
      • Cohen M.E.
      • Dahlke A.R.
      • Ko C.Y.
      • et al.
      Association of the 2011 ACGME resident duty hour reform with general surgery patient outcomes and with resident examination performance.
      Hospitals participating in NSQIP are already actively engaged in quality improvement initiatives and have a strong culture of safety
      • Cohen M.E.
      • Liu Y.
      • Ko C.Y.
      • Hall B.L.
      Improved surgical outcomes for ACS NSQIP hospitals over time: evaluation of hospital cohorts with up to 8 years of participation.
      ; as such, these hospitals may have robust mechanisms in place that limit their sensitivity to small variations in resident care as a result of the duty hour regulations.
      In this context, we designed a study to evaluate patient outcomes in the period immediately after the most recent duty hour reforms in a national surgery population. To do this, we used Medicare claims data and incorporated a broad range of surgical conditions. This analysis includes essentially every hospital participating in general surgery training. We used a control group of nonteaching hospitals and compared outcomes among hospitals of varying teaching intensity to this nonteaching control.

      Methods

      Data source and study population

      We used complete Medicare claims data for patients undergoing 11 different common general and vascular surgery procedures (cholecystectomy, colectomy, incisional hernia repair, appendectomy, gastrectomy, pancreatic resection, liver resection, esophagectomy, abdominal aortic aneurysm repair, lower extremity bypass, carotid endarterectomy) during 2 time periods: January 2009–June 2011 and July 2011–December 2012. Patients who were <65 years of age or >99 years of age were excluded from the study, as were patients not enrolled in Medicare parts A and B, and patients enrolled in Medicare Advantage. This selection method for Medicare patients has been described previously by our group.
      • Dimick J.B.
      • Nicholas L.H.
      • Ryan A.M.
      • Thumma J.R.
      • Birkmeyer J.D.
      Bariatric surgery complications before vs after implementation of a national policy restricting coverage to centers of excellence.
      We excluded patients whose hospitalization spanned from June 30, 2011, to July 1, 2011, because they would have been hospitalized both before and after the institution of the new regulations. We also included only those patients whose index hospitalization occurred during the study period; that is, we did not include a patient in the post-reform cohort if they had previously been admitted for the same condition before the reform, because this would represent a crossover between the 2 study periods. Patients were thus assigned to the pre-reform or post-reform cohorts based on their index hospitalization. For the small proportion of patients admitted in June of 2011, any outcome measure occurring after July 1 was attributed to the pre-reform period. Further, at the hospital level we excluded hospitals that either opened or closed during the study period, and very small hospitals with <350 admissions per year. The rationale for excluding these small hospitals is that they may not be true acute care facilities, and their outcomes would be less reliable owing to their small size.
      • Volpp K.G.
      • Rosen A.K.
      • Rosenbaum P.R.
      • Romano P.S.
      • Even-Shoshan O.
      • Wang Y.
      • et al.
      Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform.

      Outcome measures

      Patient outcomes were 30-day mortality, serious morbidity, failure to rescue (FTR), and readmission in each time period. To identify morbidity rates, we utilized the Complication Screening Project, a validated method for identifying postoperative complications, which identifies 8 different types of complications from administrative data.
      • Iezzoni L.I.
      • Daley J.
      • Heeren T.
      • Foley S.M.
      • Fisher E.S.
      • Duncan C.
      • et al.
      Identifying complications of care using administrative data.
      The categories of these complications are pulmonary failure, pneumonia, myocardial infarction, deep venous thrombosis or embolism, acute renal failure, postoperative hemorrhage, surgical site infection, and gastrointestinal bleeding. We focused on serious complications, which we defined as the presence of a postoperative complication combined with an extended length of stay (>75th percentile). This method has been used previously, and attempts to increase the specificity of complication measurement by ignoring coded complications that do not have meaningful clinical impact. FTR was measured using methods previously described,
      • Ghaferi A.A.
      • Birkmeyer J.D.
      • Dimick J.B.
      Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients.
      defining FTR as mortality within 30 days after a recognized complication.
      Teaching intensity was measured at each hospital using a ratio of residents to patient beds; this ratio is a commonly used measure of teaching intensity and is linked to Medicare's indirect Medical Education payment adjustments.
      • Nguyen N.X.
      • Sheingold S.H.
      Indirect medical education and disproportionate share adjustments to Medicare inpatient payment rates.
      The number of residents at each hospital is obtained from the Center for Medicare and Medicaid Services' Medicare Cost Reports; the accuracy of this number is considered very high owing to its linkage to graduate medical education funding. We then stratified hospitals into quintiles of teaching intensity using this ratio, in a fashion that has been previously used in studies evaluating teaching hospitals.
      • Volpp K.G.
      • Rosen A.K.
      • Rosenbaum P.R.
      • Romano P.S.
      • Even-Shoshan O.
      • Wang Y.
      • et al.
      Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform.
      • Allison J.J.
      • Kiefe C.I.
      • Weissman N.W.
      • Person S.D.
      • Rousculp M.
      • Canto J.G.
      • et al.
      Relationship of hospital teaching status with quality of care and mortality for Medicare patients with acute MI.
      • Taylor Jr., D.H.
      • Whellan D.J.
      • Sloan F.A.
      Effects of admission to a teaching hospital on the cost and quality of care for Medicare beneficiaries.
      • Volpp K.G.
      • Rosen A.K.
      • Rosenbaum P.R.
      • Romano P.S.
      • Even-Shoshan O.
      • Canamucio A.
      • et al.
      Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform.
      The bottom quintile was all nonteaching hospitals (resident to bed ratio = 0); the remaining quintiles were stratified from “very minor” to “very major” degrees of teaching (very minor teaching > 0–0.05; minor > 0.05–0.25; major > 0.25–0.60; very major > 0.60).

      Statistical analysis

      Our study aimed to evaluate changes in patient outcomes after the institution of the new duty hour regulations in July 2011. We used an econometric technique for evaluating the impact of public policy, the difference-in-differences approach, for our analysis. This method attempts to isolate the effects associated with a policy change and exclude any other changes over the same time period. The difference-in-differences technique compares the hospitals with themselves in the periods before and after duty hour reforms, adjusting for any time invariant differences between the groups of hospitals between the 2 time periods. It also compares the hospitals against a control group, in this case nonteaching hospitals, to prevent any temporal changes that affected outcomes in all hospitals over the same time period from being attributed to the change in resident duty hours.
      Before performing the difference-in-differences analysis, we first assessed the trends in outcomes before the duty hour regulation changes. If there were differing trends in our measured outcomes between the hospital groups in the period leading up to the duty hour change and these differences persisted after the change, these differences could be mistakenly attributed to the duty hour regulations themselves. Over the 2 years before the 2011 regulations, the trends in performance were parallel between the teaching hospitals and nonteaching control group for the majority of our measured outcomes. However, for mortality, for the major and very major teaching intensity quintiles, mortality rates were decreasing at a faster rate than at less teaching intense hospitals and nonteaching hospitals. Therefore, for mortality, we compared the post-reform mortality rate to only the most recent year pre-reform rather than the entire pre-reform period to mitigate this differing trend. It is notable that in the difference-in-differences model this would only exaggerate any significant decrease in mortality among major teaching hospitals, if present.
      To perform the difference-in-differences analysis, we used logistic regression models to evaluate the relative risk (RR) of each of our outcome measures (mortality, serious morbidity, readmission, FTR) before and after the implementation of the 2011 duty hour changes, in each quintile of teaching intensity compared with the nonteaching control group. The RR of each outcome was measured in each hospital quintile comparing the rates before and after the reform and against the nonteaching hospital control. In our models, each outcome measure was analyzed separately and served as the dependent variable. A dummy variable (year of surgery) was used to indicate whether the patient had surgery in the pre- or post-duty hour reform time period. A continuous time variable was included, to account for secular trends over time. Our exposure variable was the teaching status of the hospitals, using teaching status as a dummy variable. The coefficient of this interaction term between the exposure (teaching status) and the pre–post duty hour reform variable indicates the independent effect of the duty hours regulation changes on each outcome measure (the difference-in-differences estimate).
      Further, our regression models were risk adjusted using the Elixhauser comorbidity index.
      • Elixhauser A.
      • Steiner C.
      • Harris D.R.
      • Coffey R.M.
      Comorbidity measures for use with administrative data.
      This commonly used method of risk adjustment has been validated externally; it uses 29 different comorbidities abstracted from administrative data to account for potential differences in the patient populations of our different hospital cohorts. The models were also adjusted for hospital characteristics, including annual volume and case mix. We used RRs rather than odds ratios as our main analysis owing to the limitations in accuracy of odds ratios when examining a common outcome variable.
      All statistical analyses were conducted using STATA 13.0 (College Station, TX). All P values are 2-sided. This study was considered exempt from review by the University of Michigan Institutional Review Board.

      Results

      Patient characteristics before and after the duty hour changes are shown in Table I. We included 1,223,815 general surgery and 475,262 vascular surgery patients in the study. There were small differences in patient demographics across the 2 periods that reached statistical significance (Table I). The post-reform cohort had a slightly higher incidence of comorbid diseases, with higher rates of obesity, diabetes, hypertension, peripheral vascular disease, and congestive heart failure. Table I also demonstrates the characteristics of the hospitals included in the analysis, with mean bed numbers and the number of teaching versus nonteaching hospitals.
      Table IPatient and hospital characteristics before and after institution of 2011 ACGME duty hour changes
      CharacteristicBefore reform (1/2009–6/2011)After reform (7/2011–12/2012)
      Owing to large sample size, P < .05 for all except rates of gastrectomy and esophagectomy.
      Patient characteristics
      n1,109,356589,721
       Median age75.075.0
       Gender (% male)49.849.3
       Black race (%)7.37.4
       Presence of comorbidities (%)
      09.46.9
      1–224.717.9
      >266.075.2
       Most common comorbidities (%)
      Congestive heart failure7.69.3
      Diabetes20.625.3
      Hypertension61.573.6
      Peripheral vascular disease12.215.2
      Obesity6.711.9
      Hospital characteristics
      n3,0902,956
       No. of beds (mean)401.4409.9
       Resident-to-bed ratio (mean).108.109
       Nonteaching (%)45.844.8
       Annual admissions (mean)8,5818,435
      Operative characteristics (%)
       General procedure71.672.9
      Cholecystectomy24.124.3
      Colon resection24.124.5
      Incisional hernia repair12.513.0
      Appendectomy6.46.6
      Gastrectomy1.91.9
      Pancreatic resection1.01.1
      Liver resection0.90.9
      Esophagectomy0.70.7
       Vascular procedure28.427.1
      Abdominal aortic aneurysm repair6.46.2
      Lower extremity bypass6.76.3
      Carotid endarterectomy15.314.5
      Owing to large sample size, P < .05 for all except rates of gastrectomy and esophagectomy.
      The Figure shows the change over time in all outcome variables, before and after the reform, for each quintile of teaching intensity. The rates of mortality, serious morbidity, readmission, and FTR were relatively stable over time for each group of hospital, although the baseline rates of adverse events vary with teaching intensity. Table II shows the rates of mortality, serious morbidity, readmission, and FTR in both the pre-reform and post-reform periods, across each quintile of teaching intensity. The RRs of a simple before-and-after comparison for each outcome are also shown in Table II.
      Figure thumbnail gr1
      FigureTrends in adverse outcomes over time, before and after the institution of the 2011 ACGME duty hour reforms, stratified by level of hospital teaching intensity. A, Mortality. B, Serious complications. C, Readmissions. D, Failure to rescue.
      Table IIAdverse outcomes before and after implementation of duty hours reform in July 2011
      OutcomePatients with adverse outcome before and after duty hour reform (%)Relative risk (95% CI) of adverse outcome
      Before reform (January 2009-June 2011)After reform (July 2011-December 2012)Simple before and after comparison
      Adjusted for patient factors, hospital volume, procedure type.
      Independent effect of duty hours reform
      The independent effect is derived from the Difference-in-differences model and represents changes in outcome before and after the duty hour reform in the quantiles of teaching intensity, compared with nonteaching hospitals (the control group).
      Mortality
       Nonteaching4.74.40.96 (0.91–1.01)1 (Reference)
       Very minor teaching4.74.40.91 (0.82–0.99)1.00 (0.95–1.05)
       Minor teaching4.64.40.99 (0.92–1.06)1.01 (0.96–1.06)
       Major teaching4.74.61.15 (1.06–1.23)1.03 (0.97–1.08)
       Very major teaching4.24.00.95 (0.79–1.11)0.99 (0.90–1.08)
      Serious complications
       Nonteaching11.310.90.95 (0.92–0.98)1 (Reference)
       Very minor teaching12.312.10.93 (0.88–0.98)1.01 (0.98–1.04)
       Minor teaching12.912.40.95 (0.91–0.99)0.99 (0.96–1.01)
       Major teaching14.614.90.98 (0.93–1.03)1.02 (0.99–1.05)
       Very major teaching14.714.60.94 (0.88–1.01)1.02 (0.98–1.06)
      Readmissions
       Nonteaching12.612.21.02 (0.99–1.04)1 (Reference)
       Very minor teaching12.912.41.03 (0.98–1.07)0.99 (0.96–1.01)
       Minor teaching12.912.50.98 (0.95–1.02)0.99 (0.97–1.02)
       Major teaching15.715.00.96 (0.91–1.02)0.98 (0.95–1.01)
       Very major teaching16.616.01.02 (0.95–1.09)1.00 (0.96–1.03)
      Failure to rescue
       Nonteaching17.015.70.97 (0.92–1.02)1 (Reference)
       Very minor teaching16.515.40.95 (0.87–1.04)1.00 (0.95–1.05)
       Minor teaching16.115.51.01 (0.94–1.07)1.03 (0.99–1.07)
       Major teaching15.715.01.12 (1.03–1.20)1.01 (0.95–1.07)
       Very major teaching14.313.30.94 (0.78–1.10)0.95 (0.87–1.04)
      Adjusted for patient factors, hospital volume, procedure type.
      The independent effect is derived from the Difference-in-differences model and represents changes in outcome before and after the duty hour reform in the quantiles of teaching intensity, compared with nonteaching hospitals (the control group).
      The difference-in-differences analysis that compares teaching hospitals versus nonteaching shows no significant changes in the RR of any of our outcome measures after the duty hour changes (Table II). For example, when comparing very major teaching hospitals with nonteaching hospitals, the RR for mortality was 0.99 (95% CI, 0.90–1.08). Similarly, no change was seen in the RR of serious complication (RR, 1.02; 95% CI, 0.98–1.06), readmission (RR, 1.00; 95% CI, 0.96–1.03), or FTR (RR, 0.95; 95% CI, 0.87–1.04) after the 2011 duty hour reforms.
      We also evaluated the change in outcomes within each strata of teaching intensity relative to nonteaching hospitals and found similar results (Table II). We then evaluated outcomes for general and vascular procedures separately, with similar findings. For example, in the very major hospital quintile, the RR of mortality when compared with the nonteaching controls was 0.99 (95% CI, 0.90–1.09) for general surgery procedures and 0.93 (95% CI, 0.72–1.14) for vascular procedures.

      Discussion

      After 20 years without significant change in the structure of resident training programs, concerns for patient safety led the ACGME to increase their oversight and limitations on resident duty hours beginning with the 2003 regulations. In this study, we conducted a large-scale analysis of the most recent 2011 ACGME duty hour restriction on outcomes in a surgery population. We found that a broad range of outcome measures were unchanged after enactment of these new regulations. In this analysis, we compared teaching with nonteaching hospitals; even when evaluating these changes across the spectrum of teaching intensity, we found no effects on patient outcomes.
      Most prior research evaluating the ACGME's duty hour policies has focused on the initial changes in 2003. Several early studies, largely from single institutions, attempted to identify changes in outcomes in the surgery population and showed conflicting results. A recent systematic review of duty hour changes identified 35 such studies examining patient safety that were deemed sufficiently high quality; 48% of these demonstrated no change in patient outcomes and 19% identified any improvement.
      • Ahmed N.
      • Devitt K.S.
      • Keshet I.
      • Spicer J.
      • Imrie K.
      • Feldman L.
      • et al.
      A systematic review of the effects of resident duty hour restrictions in surgery: impact on resident wellness, training, and patient outcomes.
      This review demonstrates the ongoing uncertainty regarding the impact of the duty hour reforms on patient outcomes. Additionally, several studies have attempted to evaluate the impact of these regulations on residents themselves. Overall, a number of studies have demonstrated significant improvement in quality of life,
      • Hutter M.M.
      • Kellogg K.C.
      • Ferguson C.M.
      • Abbott W.M.
      • Warshaw A.L.
      The impact of the 80-hour resident workweek on surgical residents and attending surgeons.
      • Fletcher K.E.
      • Reed D.A.
      • Arora V.M.
      Patient safety, resident education and resident well-being following implementation of the 2003 ACGME duty hour rules.
      and negligible effects on postgraduate resident education.
      • de Virgilio C.
      • Yaghoubian A.
      • Lewis R.J.
      • Stabile B.E.
      • Putnam B.A.
      The 80-hour resident workweek does not adversely affect patient outcomes or resident education.
      • Moonesinghe S.R.
      • Lowery J.
      • Shahi N.
      • Millen A.
      • Beard J.D.
      Impact of reduction in working hours for doctors in training on postgraduate medical education and patients' outcomes: systematic review.
      There is much less literature evaluating the recent 2011 duty hour changes. Two recent studies evaluated the impact of these changes on patient outcomes. A study published by the FIRST trial group containing NSQIP participating hospitals found no impact of the duty hour regulations.
      • Rajaram R.
      • Chung J.W.
      • Jones A.T.
      • Cohen M.E.
      • Dahlke A.R.
      • Ko C.Y.
      • et al.
      Association of the 2011 ACGME resident duty hour reform with general surgery patient outcomes and with resident examination performance.
      However, their analysis only included a subset of teaching hospitals representing a minority of the surgery residency programs in the United States. Participation in the NSQIP may have acted as a confounder, given that these hospitals are engaged actively in high-level quality improvement initiatives that may mask changes resulting from resident quality of care. A similarly structured analysis was published in a Medicare population; however, this study included only mortality and readmissions data as outcome measures, included a mixed medical and surgical population, and had limited postintervention follow-up.
      • Patel M.S.
      • Volpp K.G.
      • Small D.S.
      • Hill A.S.
      • Even-Shoshan O.
      • Rosenbaum L.
      • et al.
      Association of the 2011 ACGME resident duty hour reforms with mortality and readmissions among hospitalized Medicare patients.
      Our study builds on these existing studies by including a broader set of patient outcome measures in a national patient population encompassing essentially all surgery residency programs. In addition to studying mortality rates and a composite measure of complication rates, we also assessed readmissions, because hospital discharges require careful communication and coordination of care. We also included FTR as an outcome measure, because rescue itself requires timely diagnosis and management, typically performed by residents at the bedside. In our analysis, we felt that FTR would be a particularly sensitive metric to identify any changes as a result of the duty hour regulations.
      There have been additional studies focusing on intermediate outcomes, such as handoffs or medical errors. A recent study demonstrated that trainees receive inadequate training on giving and receiving effective patient sign-out, and that a number of practical barriers exist to effective handoffs in surgical residencies. In 1 preliminary study of night-float–based shift systems in a medical residency program, the night-float model was actually abandoned after nursing staff reported increased medical errors and communication lapses.
      • Desai S.V.
      • Feldman L.
      • Brown L.
      • Dezube R.
      • Yeh H.C.
      • Punjabi N.
      • et al.
      Effect of the 2011 vs 2003 duty hour regulation-compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff: a randomized trial.
      Moreover, there have been several studies raising concern for the educational impact on surgery trainees.
      • Lee D.Y.
      • Myers E.A.
      • Rehmani S.S.
      • Wexelman B.A.
      • Ross R.E.
      • Belsley S.S.
      • et al.
      Surgical residents' perception of the 16-hour work day restriction: concern for negative impact on resident education and patient care.
      • Schwartz S.I.
      • Galante J.
      • Kaji A.
      • Dolich M.
      • Easter D.
      • Melcher M.L.
      • et al.
      Effect of the 16-hour work limit on general surgery intern operative case volume: a multi-institutional study.
      • Sen S.
      • Kranzler H.R.
      • Didwania A.K.
      • Schwartz A.C.
      • Amarnath S.
      • Kolars J.C.
      • et al.
      Effects of the 2011 duty hour reforms on interns and their patients: a prospective longitudinal cohort study.
      A recent multicenter study demonstrated that the 16-hour restriction may impact negatively an intern's operative volume,
      • Schwartz S.I.
      • Galante J.
      • Kaji A.
      • Dolich M.
      • Easter D.
      • Melcher M.L.
      • et al.
      Effect of the 16-hour work limit on general surgery intern operative case volume: a multi-institutional study.
      and residents themselves have expressed that the new regulations have affected their quality of life, levels of fatigue, and education negatively.
      • Lee D.Y.
      • Myers E.A.
      • Rehmani S.S.
      • Wexelman B.A.
      • Ross R.E.
      • Belsley S.S.
      • et al.
      Surgical residents' perception of the 16-hour work day restriction: concern for negative impact on resident education and patient care.
      • Drolet B.C.
      • Sangisetty S.
      • Tracy T.F.
      • Cioffi W.G.
      Surgical residents' perceptions of 2011 Accreditation Council for Graduate Medical Education duty hour regulations.
      Given these potential unintended negative consequences, it is important to understand whether these changes are having their intended impact on patient outcomes. Our study adds to the understanding of this policy change by demonstrating in a national surgical population the impact on patient outcomes.
      Our study has several important limitations. First, this was a retrospective study of Medicare data. The use of administrative data has several limitations pertinent to our findings. However, given our goals of assessing the national impact of the duty hour regulations, this is clearly the most appropriate dataset and the advantages afforded by Medicare data outweigh these limitations. In addition to providing a large national sample, the Medicare population is an inherently high-risk one, and may have greater sensitivity to small variations in the quality of patient care. There are well-known limitations in coding of complications within administrative datasets. Therefore, we limited our assessment to a recognized and validated subset of complications using a composite measure.
      • Iezzoni L.I.
      • Daley J.
      • Heeren T.
      • Foley S.M.
      • Fisher E.S.
      • Duncan C.
      • et al.
      Identifying complications of care using administrative data.
      Further, to increase the specificity of the complication codes, we combined them with an extended length of stay criterion, which we labeled serious complications.
      • Dimick J.B.
      • Nicholas L.H.
      • Ryan A.M.
      • Thumma J.R.
      • Birkmeyer J.D.
      Bariatric surgery complications before vs after implementation of a national policy restricting coverage to centers of excellence.
      Administrative data also have limitations in the accuracy of coded comorbidities. In our dataset the post-reform cohort seemed to have higher rates of obesity and obesity-related comorbidities, including diabetes and hypertension, likely reflecting national trends over time. To minimize the impact of this change in comorbidities over time, we used the best available measures of risk adjustment,
      • Iezzoni L.I.
      • Daley J.
      • Heeren T.
      • Foley S.M.
      • Fisher E.S.
      • Duncan C.
      • et al.
      Identifying complications of care using administrative data.
      • Elixhauser A.
      • Steiner C.
      • Harris D.R.
      • Coffey R.M.
      Comorbidity measures for use with administrative data.
      and included a time variable in our regression model. The use of the difference-in-differences analytic method and the longitudinal nature of the study mitigate bias owing to unmeasured comorbidities as well. Another limitation is the limited postintervention follow-up period. We have used the most recently available Medicare data to maximize the duration of the study period. In addition, the 2011 duty hours most significantly impact PGY-1 trainees, suggesting that any impact on outcomes should be seen in the short term rather than the long term. Our study used a common measure to assess teaching intensity; however, the accuracy of this for surgery training programs may be limited. Within the Medicare data, we were not able to specifically assess the quantity of surgical trainees at a given hospital, only the overall number of residents in all disciplines. However, general surgery programs are among the most common residencies at teaching hospitals. Further, at the highest quintile of teaching intensity, these hospitals are typically large, academic medical centers that would universally have surgery training programs. The resident to bed ratio in Medicare data may not reflect fully the resident complement of a hospital, because a number of large hospitals have additional unfunded trainees over their Medicare cap. However, this would bias our measure of teaching intensity toward underestimating intensity, thereby strengthening the comparison with the nonteaching control group. The use of only a surgery patient population may represent a limitation as well, given that the duty hour regulation changes have impacted all disciplines of resident trainees. However, postoperative patients are particularly high risk for an in-hospital error and require intensive floor care. The timely diagnosis and rescue from surgical complications is a surveillance-dependent process in which residents play a key role.
      • Ghaferi A.A.
      • Birkmeyer J.D.
      • Dimick J.B.
      Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients.
      These findings have important implications for resident work hour regulations set by the ACGME. The intention of the 2003 regulations was primarily to improve patient outcomes through a strategy of fatigue mitigation by duty hour regulation. Importantly, these same regulations were also meant to improve trainees' work–life balance and quality of training. Despite much initial controversy, the 2003 duty hour regulations have now come to be largely regarded as salutary for trainees' quality of life and safe for patient care. The impact of the 2011 regulations remains much less clear. There has been concern that these policy changes have not had their intended impact on trainees—that they have not succeeded in reducing fatigue and may have had unintended negative consequences by increasing work compression and increasing patient handoffs. The regulation of resident work hours has affected hospitals significantly as well, forcing changes in staffing patterns, supervision, and increased hiring of mid-level practitioners to offset the reduced hours of trainees.
      • Doherty G.M.
      Surgery resident education 1986-2008: effort, respect, and advocacy.
      Although our data demonstrate no impact on patient outcomes, this does not reflect the degree of effort undertaken by hospitals to maintain the same level of care. Further, the implementation of the 16-hour shift limit and the required 8-hour break between shifts have limited individual residency programs' flexibility to the extent that they may not be able to meet the educational needs of their trainees. Residents themselves echo a number of these concerns, particularly regarding the impact on their quality of life outside the hospital. In a recent national survey, Drolet et al
      • Drolet B.C.
      • Sangisetty S.
      • Tracy T.F.
      • Cioffi W.G.
      Surgical residents' perceptions of 2011 Accreditation Council for Graduate Medical Education duty hour regulations.
      found that the majority of surgery residents believed the 2011 regulations had worsened both quality of life for senior trainees and preparation for their role as senior residents and attendings.
      In acknowledgement of this growing level of concern, the ACGME (working with the American Board of Surgery and the American College of Surgeons) has authorized the first prospective, randomized trial comparing different duty hour paradigms, the Flexibility in Surgery Resident Duty Hours Trial (FIRST Trial).

      Flexibility In Duty Hour Requirements for Surgical Trainees Trial (FIRST). 2014. http://www.thefirsttrial.org. Accessed August 1, 2014.

      This trial has randomized hospitals/residency programs to 2 study arms: a usual care, or control arm continuing the current 2011 duty hours, and an intervention arm that liberalized programs to a more flexible set of requirements closely resembling the 2003 regulations. Our study is especially relevant in light of this landmark trial, because our early findings demonstrate no improvement in patient outcomes after the onset of this policy, thereby supporting equipoise between the flexible and usual care arms of the FIRST trial. With regard to duty hour reform, there is significant concern that the scales have been tipped too far in favor of restrictions without clear benefit to patients and with significant drawbacks for trainees. If this is indeed true, policy changes to pursue less restrictive and more flexible regulations may help to strike the ideal balance between optimal patient outcomes and optimal education in surgery.

      References

        • Philibert I.
        • Friedmann P.
        • Williams W.T.
        • ACGME Work Group on Resident Duty Hours. Accreditation Council for Graduate Medical Education
        New requirements for resident duty hours.
        JAMA. 2002; 288: 1112-1114
        • Ulmer C.
        • Miller Wolman D.
        • Johns M.M.E.
        Resident duty hours: enhancing sleep, supervision, and safety. Institute of Medicine report brief.
        The National Academies Press, Washington (DC)2009
        • Iglehart J.K.
        Revisiting duty-hour limits–IOM recommendations for patient safety and resident education.
        N Engl J Med. 2008; 359: 2633-2635
        • Nasca T.J.
        • Day S.H.
        • Amis Jr., E.S.
        • ACGME Duty Hour Task Force
        The new recommendations on duty hours from the ACGME Task Force.
        N Engl J Med. 2010; 363: e3
      1. Resident duty hours in the learning and working environment: comparison of 2003 and 2011 standards. Accreditation Council for Graduate Medical Education, Chicago2011
        • Barger L.K.
        • Ayas N.T.
        • Cade B.E.
        • Cronin J.W.
        • Rosner B.
        • Speizer F.E.
        • et al.
        Impact of extended-duration shifts on medical errors, adverse events, and attentional failures.
        PLoS Med. 2006; 3: e487
        • Philibert I.
        Sleep loss and performance in residents and nonphysicians: a meta-analytic examination.
        Sleep. 2005; 28: 1392-1402
        • Landrigan C.P.
        • Rothschild J.M.
        • Cronin J.W.
        • Kaushal R.
        • Burdick E.
        • Katz J.T.
        • et al.
        Effect of reducing interns' work hours on serious medical errors in intensive care units.
        N Engl J Med. 2004; 351: 1838-1848
        • Van Eaton E.G.
        • Tarpley J.L.
        • Solorzano C.C.
        • Cho C.S.
        • Weber S.M.
        • Termuhlen P.M.
        Resident education in 2011: three key challenges on the road ahead.
        Surgery. 2011; 149: 465-473
        • Rajaram R.
        • Chung J.W.
        • Jones A.T.
        • Cohen M.E.
        • Dahlke A.R.
        • Ko C.Y.
        • et al.
        Association of the 2011 ACGME resident duty hour reform with general surgery patient outcomes and with resident examination performance.
        JAMA. 2014; 312: 2374-2384
        • Cohen M.E.
        • Liu Y.
        • Ko C.Y.
        • Hall B.L.
        Improved surgical outcomes for ACS NSQIP hospitals over time: evaluation of hospital cohorts with up to 8 years of participation.
        Ann Surg. 2015; (Feb 26 [Epub ahead of print])
        • Dimick J.B.
        • Nicholas L.H.
        • Ryan A.M.
        • Thumma J.R.
        • Birkmeyer J.D.
        Bariatric surgery complications before vs after implementation of a national policy restricting coverage to centers of excellence.
        JAMA. 2013; 309: 792-799
        • Volpp K.G.
        • Rosen A.K.
        • Rosenbaum P.R.
        • Romano P.S.
        • Even-Shoshan O.
        • Wang Y.
        • et al.
        Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform.
        JAMA. 2007; 298: 975-983
        • Iezzoni L.I.
        • Daley J.
        • Heeren T.
        • Foley S.M.
        • Fisher E.S.
        • Duncan C.
        • et al.
        Identifying complications of care using administrative data.
        Med Care. 1994; 32: 700-715
        • Ghaferi A.A.
        • Birkmeyer J.D.
        • Dimick J.B.
        Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients.
        Ann Surg. 2009; 250: 1029-1034
        • Nguyen N.X.
        • Sheingold S.H.
        Indirect medical education and disproportionate share adjustments to Medicare inpatient payment rates.
        Medicare Medicaid Res Rev. 2011; 1: 4
        • Allison J.J.
        • Kiefe C.I.
        • Weissman N.W.
        • Person S.D.
        • Rousculp M.
        • Canto J.G.
        • et al.
        Relationship of hospital teaching status with quality of care and mortality for Medicare patients with acute MI.
        JAMA. 2000; 284: 1256-1262
        • Taylor Jr., D.H.
        • Whellan D.J.
        • Sloan F.A.
        Effects of admission to a teaching hospital on the cost and quality of care for Medicare beneficiaries.
        N Engl J Med. 1999; 340: 293-299
        • Volpp K.G.
        • Rosen A.K.
        • Rosenbaum P.R.
        • Romano P.S.
        • Even-Shoshan O.
        • Canamucio A.
        • et al.
        Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform.
        JAMA. 2007; 298: 984-992
        • Elixhauser A.
        • Steiner C.
        • Harris D.R.
        • Coffey R.M.
        Comorbidity measures for use with administrative data.
        Med Care. 1998; 36: 8-27
        • Ahmed N.
        • Devitt K.S.
        • Keshet I.
        • Spicer J.
        • Imrie K.
        • Feldman L.
        • et al.
        A systematic review of the effects of resident duty hour restrictions in surgery: impact on resident wellness, training, and patient outcomes.
        Ann Surg. 2014; 259: 1041-1053
        • Hutter M.M.
        • Kellogg K.C.
        • Ferguson C.M.
        • Abbott W.M.
        • Warshaw A.L.
        The impact of the 80-hour resident workweek on surgical residents and attending surgeons.
        Ann Surg. 2006; 243 (discussion 71-5): 864-871
        • Fletcher K.E.
        • Reed D.A.
        • Arora V.M.
        Patient safety, resident education and resident well-being following implementation of the 2003 ACGME duty hour rules.
        J Gen Intern Med. 2011; 26: 907-919
        • de Virgilio C.
        • Yaghoubian A.
        • Lewis R.J.
        • Stabile B.E.
        • Putnam B.A.
        The 80-hour resident workweek does not adversely affect patient outcomes or resident education.
        Curr Surg. 2006; 63 (discussion 40): 435-439
        • Moonesinghe S.R.
        • Lowery J.
        • Shahi N.
        • Millen A.
        • Beard J.D.
        Impact of reduction in working hours for doctors in training on postgraduate medical education and patients' outcomes: systematic review.
        BMJ. 2011; 342: d1580
        • Patel M.S.
        • Volpp K.G.
        • Small D.S.
        • Hill A.S.
        • Even-Shoshan O.
        • Rosenbaum L.
        • et al.
        Association of the 2011 ACGME resident duty hour reforms with mortality and readmissions among hospitalized Medicare patients.
        JAMA. 2014; 312: 2364-2373
        • Desai S.V.
        • Feldman L.
        • Brown L.
        • Dezube R.
        • Yeh H.C.
        • Punjabi N.
        • et al.
        Effect of the 2011 vs 2003 duty hour regulation-compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff: a randomized trial.
        JAMA Intern Med. 2013; 173: 649-655
        • Lee D.Y.
        • Myers E.A.
        • Rehmani S.S.
        • Wexelman B.A.
        • Ross R.E.
        • Belsley S.S.
        • et al.
        Surgical residents' perception of the 16-hour work day restriction: concern for negative impact on resident education and patient care.
        J Am Coll Surg. 2012; 215: 868-877
        • Schwartz S.I.
        • Galante J.
        • Kaji A.
        • Dolich M.
        • Easter D.
        • Melcher M.L.
        • et al.
        Effect of the 16-hour work limit on general surgery intern operative case volume: a multi-institutional study.
        JAMA Surg. 2013; 148: 829-833
        • Sen S.
        • Kranzler H.R.
        • Didwania A.K.
        • Schwartz A.C.
        • Amarnath S.
        • Kolars J.C.
        • et al.
        Effects of the 2011 duty hour reforms on interns and their patients: a prospective longitudinal cohort study.
        JAMA Intern Med. 2013; 173: 657-662
        • Drolet B.C.
        • Sangisetty S.
        • Tracy T.F.
        • Cioffi W.G.
        Surgical residents' perceptions of 2011 Accreditation Council for Graduate Medical Education duty hour regulations.
        JAMA Surg. 2013; 148: 427-433
        • Doherty G.M.
        Surgery resident education 1986-2008: effort, respect, and advocacy.
        World J Surg. 2009; 33: 378-385
      2. Flexibility In Duty Hour Requirements for Surgical Trainees Trial (FIRST). 2014. http://www.thefirsttrial.org. Accessed August 1, 2014.