Advertisement

Development and validation of a model for surveillance of postoperative bleeding complications using structured electronic health records data

  • Adam R. Dyas
    Correspondence
    Reprint requests: Adam R. Dyas, MD, Resident Physician, Department of Surgery, University of Colorado Denver, Anschutz Medical Campus, 12631 E. 17th Avenue, C-310, Aurora, CO 80045.
    Affiliations
    Department of Surgery, University of Colorado School of Medicine, Aurora, CO

    Surgical Outcomes and Applied Research Program, University of Colorado School of Medicine, Aurora, CO
    Search for articles by this author
  • Yaxu Zhuang
    Affiliations
    Department of Surgery, University of Colorado School of Medicine, Aurora, CO

    Surgical Outcomes and Applied Research Program, University of Colorado School of Medicine, Aurora, CO

    Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO
    Search for articles by this author
  • Robert A. Meguid
    Affiliations
    Department of Surgery, University of Colorado School of Medicine, Aurora, CO

    Surgical Outcomes and Applied Research Program, University of Colorado School of Medicine, Aurora, CO

    Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, CO
    Search for articles by this author
  • William G. Henderson
    Affiliations
    Surgical Outcomes and Applied Research Program, University of Colorado School of Medicine, Aurora, CO

    Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, CO

    Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO
    Search for articles by this author
  • Helen J. Madsen
    Affiliations
    Department of Surgery, University of Colorado School of Medicine, Aurora, CO

    Surgical Outcomes and Applied Research Program, University of Colorado School of Medicine, Aurora, CO
    Search for articles by this author
  • Michael R. Bronsert
    Affiliations
    Surgical Outcomes and Applied Research Program, University of Colorado School of Medicine, Aurora, CO

    Adult and Child Center for Health Outcomes Research and Delivery Science, University of Colorado School of Medicine, Aurora, CO
    Search for articles by this author
  • Kathryn L. Colborn
    Affiliations
    Department of Surgery, University of Colorado School of Medicine, Aurora, CO

    Surgical Outcomes and Applied Research Program, University of Colorado School of Medicine, Aurora, CO

    Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO
    Search for articles by this author
Published:September 20, 2022DOI:https://doi.org/10.1016/j.surg.2022.08.021

      Abstract

      Background

      Postoperative bleeding complications surveillance is done primarily through manual chart review. The purpose of this study was to develop and validate a detection model for postoperative bleeding complications using structured electronic health records data.

      Methods

      Patients who underwent operations at 1 of 5 hospitals within our local health system between 2013 and 2019 and whose complications were reported by the American College of Surgeons National Surgical Quality Improvement Program were included. Electronic health records data were linked to American College of Surgeons National Surgical Quality Improvement Program data using personal health identifiers. Electronic health records predictors included diagnosis codes mapped to PheCodes, procedure names, and medications within 30 days after surgery. We defined bleeding events as the transfusion of red blood cell components within 30 days after surgery. The knockoff filter and the lasso were used to develop a model in a training set of operations from January 2013 to March 2017. Performance of each model was tested in a held-out data set of patients who underwent operations from March 2017 to October 2019.

      Results

      A total of 30,639 patients were included; 1,112 patients (3.6%) had a bleeding event. Eight predictor variables were selected by the knockoff filter. When applied to the test set, specificity was 94%, sensitivity was 94%, area under the curve was 0.97, and accuracy was 93%. Calibration was consistent in lower predicted risk patients, whereas the model slightly overpredicted risk in high-risk patients.

      Conclusion

      We created a parsimonious, accurate model for identifying patients with bleeding complications. This model can be used to augment manual chart review for surveillance and reporting of perioperative bleeding complications, enabling inclusion of all surgeries in quality improvement efforts.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Meguid R.A.
        • Bronsert M.R.
        • Juarez-Colunga E.
        • Hammermeister K.E.
        • Henderson W.G.
        Surgical Risk Preoperative Assessment System (SURPAS): I. Parsimonious, clinically meaningful groups of postoperative complications by factor analysis.
        Ann Surg. 2016; 263: 1042-1048
        • Wu W.C.
        • Smith T.S.
        • Henderson W.G.
        • et al.
        Operative blood loss, blood transfusion, and 30-day mortality in older patients after major noncardiac surgery.
        Ann Surg. 2010; 252: 11-17
        • Smilowitz N.R.
        • Oberweis B.S.
        • Nukala S.
        • et al.
        Association between anemia, bleeding, and transfusion with long-term mortality following noncardiac surgery.
        Am J Med. 2016; 129 (e312): 315-323
        • Rahbari N.N.
        • Garden O.J.
        • Padbury R.
        • et al.
        Post-hepatectomy haemorrhage: a definition and grading by the International Study Group of Liver Surgery (ISGLS).
        HPB (Oxford). 2011; 13: 528-535
        • Ali J.M.
        • Wallwork K.
        • Moorjani N.
        Do patients who require re-exploration for bleeding have inferior outcomes following cardiac surgery?.
        Interact Cardiovasc Thorac Surg. 2019; 28: 613-618
        • Moulton M.J.
        • Creswell L.L.
        • Mackey M.E.
        • Cox J.L.
        • Rosenbloom M.
        Reexploration for bleeding is a risk factor for adverse outcomes after cardiac operations.
        J Thorac Cardiovasc Surg. 1996; 111: 1037-1046
        • Schieman C.
        • Wigle D.A.
        • Deschamps C.
        • et al.
        Patterns of operative mortality following esophagectomy.
        Dis Esophagus. 2012; 25: 645-651
        • Song W.
        • Yuan Y.
        • Peng J.
        • et al.
        The delayed massive hemorrhage after gastrectomy in patients with gastric cancer: characteristics, management opinions and risk factors.
        Eur J Surg Oncol. 2014; 40: 1299-1306
        • Lu J.W.
        • Ding H.F.
        • Wu X.N.
        • et al.
        Intra-abdominal hemorrhage following 739 consecutive pancreaticoduodenectomy: risk factors and treatments.
        J Gastroenterol Hepatol. 2019; 34: 1100-1107
        • Ngo A.
        • Masel D.
        • Cahill C.
        • Blumberg N.
        • Refaai M.A.
        Blood banking and transfusion medicine challenges during the COVID-19 pandemic.
        Clin Lab Med. 2020; 40: 587-601
        • Stokes M.E.
        • Ye X.
        • Shah M.
        • et al.
        Impact of bleeding-related complications and/or blood product transfusions on hospital costs in inpatient surgical patients.
        BMC Health Serv Res. 2011; 11: 135
        • Zbrozek A.
        • Magee G.
        Cost of bleeding in trauma and complex cardiac surgery.
        Clin Ther. 2015; 37: 1966-1974
        • Newcomb A.E.
        • Dignan R.
        • McElduff P.
        • Pearse E.J.
        • Bannon P.
        Bleeding After cardiac surgery is associated with an increase in the total cost of the hospital stay.
        Ann Thorac Surg. 2020; 109: 1069-1078
        • Bilimoria K.Y.
        • Liu Y.
        • Paruch J.L.
        • et al.
        Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons.
        J Am Coll Surg. 2013; 217 (e831–e833): 833-842
        • Massarweh N.N.
        • Kaji A.H.
        • Itani K.M.F.
        Practical guide to surgical data sets: Veterans Affairs Surgical Quality Improvement Program (VASQIP).
        JAMA Surg. 2018; 153: 768-769
        • Meguid R.A.
        • Bronsert M.R.
        • Juarez-Colunga E.
        • Hammermeister K.E.
        • Henderson W.G.
        Surgical Risk Preoperative Assessment System (SURPAS): II. Parsimonious risk models for postoperative adverse outcomes addressing need for laboratory variables and surgeon specialty-specific models.
        Ann Surg. 2016; 264: 10-22
        • Meguid R.A.
        • Bronsert M.R.
        • Juarez-Colunga E.
        • Hammermeister K.E.
        • Henderson W.G.
        Surgical Risk Preoperative Assessment System (SURPAS): III. Accurate preoperative prediction of 8 adverse outcomes using 8 predictor variables.
        Ann Surg. 2016; 264: 23-31
        • Colborn K.L.
        • Bronsert M.
        • Amioka E.
        • Hammermeister K.
        • Henderson W.G.
        • Meguid R.
        Identification of surgical site infections using electronic health record data.
        Am J Infect Control. 2018; 46: 1230-1235
        • Goto M.
        • Ohl M.E.
        • Schweizer M.L.
        • Perencevich E.N.
        Accuracy of administrative code data for the surveillance of healthcare-associated infections: a systematic review and meta-analysis.
        Clin Infect Dis. 2014; 58: 688-696
        • Hu Z.
        • Simon G.J.
        • Arsoniadis E.G.
        • Wang Y.
        • Kwaan M.R.
        • Melton G.B.
        Automated detection of postoperative surgical site infections using supervised methods with electronic health record data.
        Stud Health Technol Inform. 2015; 216: 706-710
        • Ju M.H.
        • Ko C.Y.
        • Hall B.L.
        • Bosk C.L.
        • Bilimoria K.Y.
        • Wick E.C.
        A comparison of 2 surgical site infection monitoring systems.
        JAMA Surg. 2015; 150: 51-57
        • Branch-Elliman W.
        • Strymish J.
        • Kudesia V.
        • Rosen A.K.
        • Gupta K.
        Natural language processing for real-time catheter-associated urinary tract infection surveillance: results of a pilot implementation trial.
        Infect Control Hosp Epidemiol. 2015; 36: 1004-1010
        • Choudhuri J.A.
        • Pergamit R.F.
        • Chan J.D.
        • et al.
        An electronic catheter-associated urinary tract infection surveillance tool.
        Infect Control Hosp Epidemiol. 2011; 32: 757-762
        • Colborn K.L.
        • Bronsert M.
        • Hammermeister K.
        • Henderson W.G.
        • Singh A.B.
        • Meguid R.A.
        Identification of urinary tract infections using electronic health record data.
        Am J Infect Control. 2019; 47: 371-375
        • Selby L.V.
        • Narain W.R.
        • Russo A.
        • Strong V.E.
        • Stetson P.
        Autonomous detection, grading, and reporting of postoperative complications using natural language processing.
        Surgery. 2018; 164: 1300-1305
        • Bronsert M.
        • Singh A.B.
        • Henderson W.G.
        • Hammermeister K.
        • Meguid R.A.
        • Colborn K.L.
        Identification of postoperative complications using electronic health record data and machine learning.
        Am J Surg. 2020; 220: 114-119
        • Branch-Elliman W.
        • Strymish J.
        • Itani K.M.
        • Gupta K.
        Using clinical variables to guide surgical site infection detection: a novel surveillance strategy.
        Am J Infect Control. 2014; 42: 1291-1295
        • Gundlapalli A.V.
        • Divita G.
        • Redd A.
        • et al.
        Detecting the presence of an indwelling urinary catheter and urinary symptoms in hospitalized patients using natural language processing.
        J Biomed Inform. 2017; 71S: S39-S45
        • Hsu H.E.
        • Shenoy E.S.
        • Kelbaugh D.
        • et al.
        An electronic surveillance tool for catheter-associated urinary tract infection in intensive care units.
        Am J Infect Control. 2015; 43: 592-599
        • Meyer A.
        • Zverinski D.
        • Pfahringer B.
        • et al.
        Machine learning for real-time prediction of complications in critical care: a retrospective study.
        Lancet Respir Med. 2018; 6: 905-914
        • Wei W.Q.
        • Bastarache L.A.
        • Carroll R.J.
        • et al.
        Evaluating phecodes, clinical classification software, and ICD-9-CM codes for phenome-wide association studies in the electronic health record.
        PLoS One. 2017; 12e0175508
        • Wu P.
        • Gifford A.
        • Meng X.
        • et al.
        Mapping ICD-10 and ICD-10-CM codes to phecodes: workflow development and initial evaluation.
        JMIR Med Inform. 2019; 7e14325
        • Barber R.a.C.E.
        Controlling the false discovery rate via knockoffs.
        Ann Stat. 2015; 43: 2055-2085
        • Candes E.
        • Fan Y.
        • Janson L.
        • et al.
        Panning for gold: “model-X” knockoffs for high dimensional controlled variable selection.
        J R Stat Soc Series B Stat Methodol. 2018; 80: 551-577
      1. R Program for Statistical Computing. knockoff: The Knockoff Filter for Controlled Variable Selection [computer program] [cited 2021 Jan 12]. Available from https://cran.r-project.org/web/packages/knockoff/index.html

        • Friedman J.
        • Hastie T.
        • Tibshirani R.
        Regularization paths for generalized linear models via coordinate descent.
        J Stat Softw. 2010; 33: 1-22
        • Robin X.
        • Turck N.
        • Hainard A.
        • et al.
        pROC: an open-source package for R and S+ to analyze and compare ROC curves.
        BMC Bioinformatics. 2011; 12: 77