Innovation| Volume 172, ISSUE 2, P655-662, August 2022

Who doesn’t fit? A multi-institutional study using machine learning to uncover the limits of opioid prescribing guidelines



      Many U.S. institutions have adopted postsurgical opioid-prescribing guidelines to standardize prescribing practices, and yet there is inherent variability in patients’ opioid consumption after surgery. The utility of these guidelines is limited by the fact that some patients’ needs will inevitably exceed them, and yet there are no evidence-based tools to help providers identify these patients. In this study we aimed to maximize the value of these guidelines by training machine learning models to predict patients whose needs will be met by these smaller recommended prescriptions, and patients who may require an additional degree of personalization. The aim of the present study was to develop predictive models for determining whether a surgical patient's postdischarge opioid requirement will fall above or below common opioid prescribing guidelines.


      We conducted a retrospective cohort study of surgical patients at one institution from 2017 to 2018. Patients were called after discharge to collect opioid consumption data. Machine learning models were used to identify outlier opioid consumers (ie, exceeding our institutional prescribing guidelines) using diagnosis codes, medical history, in-hospital opioid use, and perioperative factors as predictors. External validation was performed on opioid consumption data collected at a second institution from 2020 to 2021, and sensitivity analysis was performed using a third institution’s prescribing guidelines.


      The development and external validation cohorts included 1,867 and 498 patients, respectively. Age, body mass index, tobacco use, preoperative opioid exposure, and in-hospital opioid consumption were the strongest predictors of postdischarge consumption. A lasso regression model exhibited an area under the receiver operating characteristic curve of 0.74 (95% confidence interval 0.67–0.81) in predicting postdischarge opioid consumption. External validation of a limited lasso model yielded an area under the receiver operating characteristic curve of 0.67 (0.60–0.74). Performance was preserved when evaluated on another institution’s guidelines (area under the receiver operating characteristic curve 0.76 [0.72–0.80]).


      Patient characteristics reliably predict postdischarge opioid consumption in relation to prescribing guidelines for both opioid-naive and exposed populations. This model may be used to help providers confidently follow prescribing guidelines for patients with typical opioid responsiveness and correctly pursue more personalized prescribing for others.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Friedman J.
        • Akre S.
        COVID-19 and the drug overdose crisis: uncovering the deadliest months in the United States, January‒July 2020.
        Am J Public Health. 2021; 111: 1284-1291
        • Centers for Disease Control and Prevention
        Vital statistics rapid release: provisional drug overdose data.
        • Sinatra R.
        Causes and consequences of inadequate management of acute pain.
        Pain Med. 2010; 11: 1859-1871
        • Kaafarani H.M.A.
        • Eid A.I.
        • Antonelli D.M.
        • et al.
        Description and impact of a comprehensive multispecialty multidisciplinary intervention to decrease opioid prescribing in surgery.
        Ann Surg. 2019; 270: 452-462
        • Howard R.
        • Waljee J.
        • Brummett C.
        • Englesbe M.
        • Lee J.
        Reduction in opioid prescribing through evidence-based prescribing guidelines.
        JAMA Surg. 2018; 153: 285-287
        • Hill M.V.
        • Stucke R.S.
        • Billmeier S.E.
        • Kelly J.L.
        • Barth R.J.
        Guideline for discharge opioid prescriptions after inpatient general surgical procedures.
        J Am Coll Surg. 2018; 226
        • Porter E.D.
        • Bessen S.Y.
        • Molloy I.B.
        • et al.
        Guidelines for patient-centeredopioid prescribing and optimal FDA-compliant disposal of excess pills after inpatient operation: prospective clinical trial.
        J Am Coll Surg. 2021; : 232
        • Overton H.N.
        • Hanna M.N.
        • Bruhn W.E.
        • et al.
        Opioid-prescribing guidelines for common surgical procedures: an expert panel consensus.
        J Am Coll Surg. 2018; 227: 411-418
        • Sekhri S.
        • Arora N.S.
        • Cottrell H.
        • et al.
        Probability of opioid prescription refilling after surgery: does initial prescription dose matter?.
        Ann Surg. 2018; 268: 271-276
        • Scully R.E.
        • Schoenfeld A.J.
        • Jiang W.
        • et al.
        Defining optimal length of opioid pain medication prescription after common surgical procedures.
        JAMA Surg. 2018; 153: 37-43
        • Robinson K.A.
        • Thiels C.A.
        • Stokes S.
        • et al.
        Comparing clinician consensus recommendations to patient-reported opioid use across multiple hospital systems.
        Ann Surg. 2020; 275: e361-e365
        • Wunsch H.
        • Wijeysundera D.N.
        • Passarella M.A.
        • Neuman M.D.
        Opioids prescribed after low-risk surgical procedures in the United States, 2004–2012.
        JAMA. 2016; 315: 1654-1657
        • Ladha K.S.
        • Neuman M.D.
        • Broms G.
        • et al.
        Opioid prescribing after surgery in the United States, Canada, and Sweden.
        JAMA Netw Open. 2019; 2e1910734
        • Cron D.C.
        • Englesbe M.J.
        • Bolton C.J.
        • et al.
        Preoperative opioid use is independently associated with increased costs and worse outcomes after major abdominal surgery.
        Ann Surg. 2017; 265: 695-701
        • Perlmutter B.
        • Wynia E.
        • McMichael J.
        • et al.
        Effect of pre-operative opioid exposure on surgical outcomes in elective laparoscopic cholecystectomy.
        Am J Surg. 2022; 223 (Published online June 24): 764-769
        • Opioid Prescribing Engagement Network
        Opioid prescribing recommendations.
      1. Clinical Query 2.
        Date accessed: September 6, 2021
        • Centers for Disease Control and Prevention
        Data resources.
        • Robinson K.A.
        • Duncan S.
        • Austrie J.
        • et al.
        Opioid consumption after gender-affirming mastectomy and two other breast surgeries.
        J Surg Res. 2020; : 251
      2. Kennedy CJ, Marwaha JS, Scalise PN, et al. Nonresponse adjustment using clinical and perioperative patient characteristics is critical for understanding post-discharge opioid consumption. bioRxiv. Published online July 7, 2021. Accessed April 28, 2022.

        • Stensland K.D.
        • Chang P.
        • Jiang D.
        • et al.
        Reducing postoperative opioid pill prescribing via a quality improvement approach.
        Int J Qual Health Care. 2021; 33
        • Centers for Disease Control and Prevention
        Defining adult overweight & obesity.
        Date accessed: September 6, 2021
        • Altman D.G.
        • Bland J.M.
        How to obtain the P value from a confidence interval.
        BMJ. 2011; 343: d2304
      3. scikit-learn.
        Date accessed: September 6, 2021
      4. Chen T, Guestrin C. XGBoost. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2016.

        • Deyo R.A.
        • Hallvik S.E.
        • Hildebran C.
        • et al.
        Association between initial opioid prescribing patterns and subsequent long-term use among opioid-naïve patients: a statewide retrospective cohort study.
        J Gen Intern Med. 2017; 32: 21-27
        • Brat G.A.
        • Agniel D.
        • Beam A.
        • et al.
        Postsurgical prescriptions for opioid naive patients and association with overdose and misuse: retrospective cohort study.
        BMJ. 2018; 360: j5790
        • Substance Abuse and Mental Health Services Administration
        Key substance use and mental health indicators in the United States.
        • Steyerberg E.W.
        Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating.
        Springer International Publishing, 2019
        • Collins G.S.
        • Reitsma J.B.
        • Altman D.G.
        • Moons K.G.M.
        Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement.
        Ann Intern Med. 2015; 162: 55-63
        • Hur J.
        • Tang S.
        • Gunaseelan V.
        • et al.
        Predicting postoperative opioid use with machine learning and insurance claims in opioid-naïve patients.
        Am J Surg. 2021; 222: 659-665
        • Lee B.
        • Zhao W.
        • Yang K.C.
        • Ahn Y.Y.
        • Perry B.L.
        Systematic evaluation of state policy interventions targeting the US opioid epidemic, 2007–2018.
        JAMA Netw Open. 2021; 4e2036687
        • Agarwal S.
        • Bryan J.D.
        • Hu H.M.
        • et al.
        Association of state opioid duration limits with postoperative opioid prescribing.
        JAMA Netw Open. 2019; 2e1918361
        • Bleicher J.
        • Stokes S.M.
        • Brooke B.S.
        • Glasgow R.E.
        • Huang L.C.
        Patient-centered opioid prescribing: breaking away from one-size-fits-all prescribing guidelines.
        J Surg Res. 2021; 264: 1-7
        • Howard R.
        • Fry B.
        • Gunaseelan V.
        • et al.
        Association of opioid prescribing with opioid consumption after surgery in Michigan.
        JAMA Surg. 2019; 154e184234
        • Thiels C.A.
        • Ubl D.S.
        • Yost K.J.
        • et al.
        Results of a prospective, multicenter initiative aimed at developing opioid-prescribing guidelines after surgery.
        Ann Surg. 2018; 268: 457-468

      Supplementary References

        • Steyerberg E.W.
        • Vickers A.J.
        • Cook N.R.
        • et al.
        Assessing the performance of prediction models: a framework for traditional and novel measures.
        Epidemiology. 2010; 21: 128-138
        • Sperrin M.
        • Martin G.P.
        • Sisk R.
        • Peek N.
        Missing data should be handled differently for prediction than for description or causal explanation.
        J Clin Epidemiol. 2020; 125: 183-187