Advertisement

The win ratio: A novel approach to define and analyze postoperative composite outcomes to reflect patient and clinician priorities

  • J. Madison Hyer
    Affiliations
    Department of Surgery, The Ohio State University Wexner Medical Center and James Comprehensive Cancer Center, Columbus, OH
    Search for articles by this author
  • Adrian Diaz
    Affiliations
    Department of Surgery, The Ohio State University Wexner Medical Center and James Comprehensive Cancer Center, Columbus, OH
    Search for articles by this author
  • Timothy M. Pawlik
    Correspondence
    Reprint requests: Timothy M. Pawlik, MD, MPH, MTS, PhD, FACS, FRACS (Hon), Department of Surgery, The Ohio State University Wexner Medical Center, Suite 670, 395 West 12th Avenue, Columbus, OH 43210.
    Affiliations
    Department of Surgery, The Ohio State University Wexner Medical Center and James Comprehensive Cancer Center, Columbus, OH
    Search for articles by this author
Published:August 26, 2022DOI:https://doi.org/10.1016/j.surg.2022.07.024

      Abstract

      Background

      The “win ratio” (WR) is a novel statistical technique that hierarchically weighs various postoperative outcomes (eg, mortality weighted more than complications) into a composite metric to define an overall benefit or “win.” We sought to use the WR to assess the impact of social vulnerability on the likelihood of achieving a “win” after hepatopancreatic surgery.

      Methods

      Individuals who underwent an elective hepatopancreatic procedure between 2013 and 2017 were identified using the Medicare database, which was merged with the Center for Disease Control and Prevention's Social Vulnerability Index. The win ratio was defined based on a hierarchy of postoperative outcomes: 90-day mortality, perioperative complications, 90-day readmissions, and length of stay. Patients matched based on procedure type, race, sex, age, and Charlson Comorbidity Index score were compared and assessed relative to win ratio.

      Results

      Among 32,557 Medicare beneficiaries who underwent hepatectomy (n = 11,621, 35.7%) or pancreatectomy (n = 20,936, 64.3%), 16,846 (51.7%) patients were male with median age of 72 years (interquartile range 68–77) and median Charlson Comorbidity Index of 3 (interquartile range 2–8), and a small subset of patients were a racial/ethnic minority (n = 3,759, 11.6%). Adverse events associated with lack of a postoperative optimal outcome included 90-day mortality (n = 2,222, 6.8%), postoperative complication (n = 8,029, 24.7%), readmission (n = 6,349, 19.5%), and length of stay (median: 7 days, interquartile range 5–11). Overall, the patients from low Social Vulnerability Index areas were more likely to “win” with a textbook outcome (win ratio 1.07, 95% confidence interval 1.01–1.12) compared with patients from high social vulnerability counties; in contrast, there was no difference in the win ratio among patients living in average versus high Social Vulnerability Index (win ratio 1.04, 95% confidence interval 0.98–1.10). In assessing surgeon volume, patients who had a liver or pancreas procedure performed by a high-volume surgeon had a higher win ratio versus patients who were treated by a low-volume surgeon (win ratio 1.21, 95% confidence interval 1.16–1.25). In contrast, there was no difference in the win ratio (win ratio 1.01, 95% confidence interval 0.97–1.06) among patients relative to teaching hospital status.

      Conclusion

      Using a novel statistical approach, the win ratio ranked outcomes to create a composite measure to assess a postoperative “win.” The WR demonstrated that social vulnerability was an important driver in explaining disparate postoperative outcomes.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Institute of Medicine
        Performance Measurement: Accelerating Improvement.
        The National Academies Press, Washington (DC)2005
        • Parina R.P.
        • Chang D.C.
        • Rose J.A.
        • Talamini M.A.
        Is a low readmission rate indicative of a good hospital?.
        J Am Coll Surg. 2015; 220: 169-176
        • Ghaferi A.A.
        • Birkmeyer J.D.
        • Dimick J.B.
        Complications, failure to rescue, and mortality with major inpatient surgery in medicare patients.
        Ann Surg. 2009; 250: 1029-1034
        • Birkmeyer J.D.
        • Siewers A.E.
        • Finlayson E.V.A.
        • et al.
        Hospital volume and surgical mortality in the United States.
        N Engl J Med. 2002; 346: 1128-1137
        • Dimick J.B.
        • Ghaferi A.A.
        • Osborne N.H.
        • Ko C.Y.
        • Hall B.L.
        Reliability adjustment for reporting hospital outcomes with surgery.
        Ann Surg. 2012; 255: 703-707
        • Dimick J.B.
        • Welch H.G.
        The zero mortality paradox in surgery.
        J Am Coll Surg. 2008; 206: 13-16
        • Dimick J.B.
        • Staiger D.O.
        • Birkmeyer J.D.
        Ranking hospitals on surgical mortality: the importance of reliability adjustment.
        Health Serv Res. 2010; 45: 1614-1629
        • Dimick J.B.
        • Welch H.G.
        • Birkmeyer J.D.
        Surgical mortality as an indicator of hospital quality. The problem with small sample size.
        JAMA. 2004; 292: 847-851
        • Merkow R.P.
        • Hall B.L.
        • Cohen M.E.
        • et al.
        Validity and feasibility of the American College of Surgeons colectomy composite outcome quality measure.
        Ann Surg. 2013; 257: 483-489
        • Dimick J.B.
        • Staiger D.O.
        • Baser O.
        • Birkmeyer J.D.
        Composite measures for predicting surgical mortality in the hospital.
        Health Affairs. 2009; 28: 1189-1198
        • Dimick J.B.
        • Staiger D.O.
        • Osborne N.H.
        • Nicholas L.H.
        • Birkmeyer J.D.
        Composite measures for rating hospital quality with major surgery.
        Health Serv Res. 2012; 47: 1861-1879
        • Dimick J.B.
        • Birkmeyer N.J.
        • Finks J.F.
        • et al.
        Composite measures for profiling hospitals on bariatric surgery performance.
        JAMA Surg. 2014; 149: 10-16
        • Merath K.
        • Chen Q.
        • Bagante F.
        • et al.
        Textbook outcomes among medicare patients undergoing hepatopancreatic surgery.
        Ann Surg. 2020; 271: 1116-1123
        • Bilimoria K.Y.
        • Barnard C.
        An evolving hospital quality star rating system from CMS: aligning the stars.
        JAMA. 2021; 325: 2151-2152
        • US Centers for Medicare & Medicaid Services
        Overall hospital quality star rating | Provider Data Catalog.
        • Bilimoria K.Y.
        • Barnard C.
        The new CMS hospital quality star ratings: the stars are not aligned.
        JAMA. 2016; 316: 1761-1762
        • Redfors B.
        • Gregson J.
        • Crowley A.
        • et al.
        The win ratio approach for composite endpoints: practical guidance based on previous experience.
        Eur Heart J. 2020; 41: 4391-4399
        • Spitzer E.
        • Van Mieghem N.M.
        • Pibarot P.
        • et al.
        Rationale and design of the Transcatheter Aortic Valve Replacement to UNload the Left ventricle in patients with ADvanced heart failure (TAVR UNLOAD) trial.
        Am Heart J. 2016; 182: 80-88
        • Bartunek J.
        • Terzic A.
        • Davison B.A.
        • et al.
        Cardiopoietic cell therapy for advanced ischaemic heart failure: results at 39 weeks of the prospective, randomized, double blind, sham-controlled CHART-1 clinical trial.
        Eur Heart J. 2017; 38: 648-660
        • Maurer M.S.
        • Schwartz J.H.
        • Gundapaneni B.
        • et al.
        Tafamidis treatment for patients with transthyretin amyloid cardiomyopathy.
        N Engl J Med. 2018; 379: 1007-1016
        • Pocock S.J.
        • Ariti C.A.
        • Collier T.J.
        • Wang D.
        The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities.
        Eur Heart J. 2012; 33: 176-182
        • Abdalla S.
        • Montez-Rath M.E.
        • Parfrey P.S.
        • Chertow G.M.
        The win ratio approach to analyzing composite outcomes: an application to the EVOLVE trial.
        Contemp Clin Trials. 2016; 48: 119-124
        • Milojevic M.
        • Head S.J.
        • Andrinopoulou E.R.
        • et al.
        Hierarchical testing of composite endpoints: applying the win ratio to percutaneous coronary intervention versus coronary artery bypass grafting in the SYNTAX trial.
        EuroIntervention. 2017; 13: 106-114
        • Luo X.
        • Tian H.
        • Mohanty S.
        • Tsai W.Y.
        An alternative approach to confidence interval estimation for the win ratio statistic.
        Biometrics. 2015; 71: 139-145
        • Bakal J.A.
        • Roe M.T.
        • Ohman E.M.
        • et al.
        Applying novel methods to assess clinical outcomes: insights from the TRILOGY ACS trial.
        Eur Heart J. 2015; 36: 385-392a
        • Agency for Toxic Substances and Disease Registry
        CDC’s Social Vulnerability Index.
        https://svi.cdc.gov/
        Date: 2018
        Date accessed: March 14, 2020
        • The Leapfrong Group
        Surgeon volume and surgical appropriateness.
        http://www.leapfroggroup.org/surgeon-volume
        Date: 2017
        Date accessed: March 10, 2019
        • Lopes R.D.
        • de Barros e Silva P.G.M.
        • Furtado R.H.M.
        • et al.
        Therapeutic versus prophylactic anticoagulation for patients admitted to hospital with COVID-19 and elevated D-dimer concentration (ACTION): an open-label, multicentre, randomised, controlled trial.
        Lancet. 2021; 397: 2253-2263
        • Merath K.
        • Chen Q.
        • Bagante F.
        • et al.
        A multi-institutional international analysis of textbook outcomes among patients undergoing curative-intent resection of intrahepatic cholangiocarcinoma.
        JAMA Surg. 2019; 154e190571
        • Azap R.A.
        • Paredes A.Z.
        • Diaz A.
        • Hyer J.M.
        • Pawlik T.M.
        The association of neighborhood social vulnerability with surgical textbook outcomes among patients undergoing hepatopancreatic surgery.
        Surgery. 2020; 168: 868-875
        • Hyer J.M.
        • Tsilimigras D.I.
        • Diaz A.
        • et al.
        High social vulnerability and “textbook outcomes” after cancer operation.
        J Am Coll Surg. 2021; 232: 351-359
        • Diaz A.
        • Hyer J.M.
        • Barmash E.
        • Azap R.
        • Paredes A.Z.
        • Pawlik T.M.
        County-level social vulnerability is associated with worse surgical outcomes especially among minority patients.
        Ann Surg. 2021; 274: 881-891
        • Carmichael H.
        • Moore A.
        • Steward L.
        • Velopulos C.G.
        Using the Social Vulnerability Index to examine local disparities in emergent and elective cholecystectomy.
        J Surg Res. 2019; 243: 160-164
        • Birkmeyer J.D.
        • Stukel T.A.
        • Siewers A.E.
        • Goodney P.P.
        • Wennberg D.E.
        • Lucas F.L.
        Surgeon volume and operative mortality in the United States.
        N Engl J Med. 2003; 349: 2117-2127
        • Sheetz K.H.
        • Chhabra K.R.
        • Smith M.E.
        • Dimick J.B.
        • Nathan H.
        Association of discretionary hospital volume standards for high-risk cancer surgery with patient outcomes and access, 2005–2016.
        JAMA Surg. 2019; 154: 1005-1012
        • Amini N.
        • Spolverato G.
        • Kim Y.
        • Pawlik T.M.
        Trends in hospital volume and failure to rescue for pancreatic surgery.
        J Gastrointest Surg. 2015; 19: 1581-1592
        • Burke L.G.
        • Frakt A.B.
        • Khullar D.
        • Orav E.J.
        • Jha A.K.
        Association between teaching status and mortality in US hospitals.
        JAMA. 2017; 317: 2105-2113
        • Dimick J.B.
        • Cowan J.A.
        • Colletti L.M.
        • Upchurch G.R.
        Hospital teaching status and outcomes of complex surgical procedures in the United States.
        Arch Surg. 2004; 139: 137-141
        • Mehta R.
        • Paredes A.Z.
        • Tsilimigras D.I.
        • et al.
        Influence of hospital teaching status on the chance to achieve a textbook outcome after hepatopancreatic surgery for cancer among Medicare beneficiaries.
        Surgery. 2020; 168: 92-100
        • Iezzoni L.I.
        • Daley J.
        • Heeren T.
        • et al.
        Identifying complications of care using administrative data.
        Med Care. 1994; 32: 700-715
        • Iezzoni L.I.
        Assessing quality using administrative data.
        Ann Intern Med. 1997; 127: 666-674
        • Weingart S.N.
        • Iezzoni L.I.
        • Davis R.B.
        • et al.
        Use of administrative data to find substandard care: validation of the complications screening program.
        Med Care. 2000; 38: 796-806
        • Lawthers A.G.
        • McCarthy E.P.
        • Davis R.B.
        • Peterson L.E.
        • Palmer R.H.
        • Iezzoni L.I.
        Identification of in-hospital complications from claims data. Is it valid?.
        Med Care. 2000; 38: 785-795