Advertisement

Databases for surgical health services research: American College of Surgeons National Surgical Quality Improvement Program

Published:February 02, 2018DOI:https://doi.org/10.1016/j.surg.2017.12.021
      The impetus for the creation of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) can be traced back to the creation of the Veterans Affairs NSQIP (VA NSQIP) in the 1990s. Anecdotal evidence at the time seemed to indicate that the operative mortality within the VA system was greater than the national average, and subsequent legislation mandated that VA hospitals compare risk-adjusted surgical outcomes to the national average.
      • Khuri S.F.
      • Daley J.
      • Henderson W.
      • et al.
      The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program.
      Three nonfederal hospitals joined the VA NSQIP program in 1999, demonstrating that rigorous data collection and risk-adjustment methods have broader applicability in surgical quality research. The ACS subsequently conducted the Patient Safety in Surgery Study, which reduced morbidity and mortality in participating hospitals by utilizing VA NSQIP methodology to identify process shortcomings outside the VA setting.
      • Khuri S.F.
      • Henderson W.G.
      • Daley J.
      • et al.
      The patient safety in surgery study: background, study design, and patient populations.
      ACS NSQIP was created in 2004 and began enrolling new private sector hospitals. There has been significant growth during the intervening years, and the database includes >5.5 million cases with >990,000 new cases in 2016 (Fig 1).
      Fig. 1
      Fig. 1Annual and cumulative PUF case volume.

      Participation in ACS NSQIP and Hospital-Specific Feedback

      Participating hospitals are required to have at least two dedicated staff members: a Surgeon Champion and a Surgical Clinical Reviewer. The Surgeon Champion is a surgeon at the hospital who works to raise awareness about the importance of data collection and analysis, answer clinical questions, and support quality improvement efforts. The Surgeon Champion and a Surgical Clinical Reviewer collects all clinical data variables through meticulous chart review for procedures in both the inpatient and outpatient settings. This includes preoperative risk factors, intraoperative variables, complication rates, and 30-day postoperative morbidity and mortality outcomes.
      Feedback occurs primarily in the form of reports that are actionable at the hospital level. A semiannual report is generated twice a year along with individual site summaries. These data allow hospitals to compare their risk-adjusted surgical outcomes to other participating sites. Risk-adjusted outcomes are reported as odds ratios using hierarchical modeling, allowing for comparison between the specific site and the “average” ACS NSQIP hospital.
      • Cohen M.E.
      • Ko C.Y.
      • Bilimoria K.Y.
      • et al.
      Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.
      Performance significantly above expectations for a given parameter is noted to be “exemplary,” while those that are significantly below expectations may be marked as “needs improvement.”

      Utility of Local ACS NSQIP Feedback in Quality Improvement Initiatives and Research

      Hospital-level data is both a driver of surgical care quality improvement initiatives and the basis for powerful local quality research. Data have been leveraged for quality improvement research of varying size and scope, from individual surgical departments to large regional hospital research collaboratives.
      • Guillamondegui O.D.
      • Gunter O.L.
      • Hines L.
      • et al.
      Using the National Surgical Quality Improvement Program and the Tennessee Surgical Quality Collaborative to improve surgical outcomes.
      Improvement has been demonstrated in a variety of domains, including pneumonia, venous thromboembolism prophylaxis, and surgical site infection.
      • Maggard-Gibbons M.
      The use of report cards and outcome measurements to improve the safety of surgical care: the American College of Surgeons National Surgical Quality Improvement Program.
      Programs focusing on surgical quality improvement using ACS NSQIP data also have been used in cost effectiveness research, demonstrating that reduction in postoperative complication rates often more than compensates for the initial cost of NSQIP participation and the subsequent quality improvement programs.
      • Hollenbeak C.S.
      • Boltz M.M.
      • Wang L.
      • et al.
      Cost-effectiveness of the National Surgical Quality Improvement Program.

      Utilizing National Data for Research and Large-Scale Quality Improvement

      While the local feedback associated with ACS NSQIP participation is extremely powerful as a quality improvement and research tool, the registry also is a well-known platform for retrospective surgical outcomes and quality research. Early concerns about data validity faded after a series of audits demonstrating high reliability, and ACS NSQIP data quality compares favorably to other administrative databases.
      • Lawson E.H.
      • Hall B.L.
      • Louie R.
      • et al.
      Association between occurrence of a postoperative complication and readmission: implications for quality improvement and cost savings.
      • Huffman K.M.
      • Cohen M.E.
      • Ko C.Y.
      • Hall B.L.
      A comprehensive evaluation of statistical reliability in ACS NSQIP profiling models.
      ACS NSQIP has since become one of the preeminent surgical research datasets, having been cited in >1,100 peer reviewed publications as of 2015 (Fig 2).
      Utilization of the national database for research primarily occurs through the participant use data file (PUF). This document is distributed annually with a delay of ≈9 months (e.g., 2016 data were released in the fall of 2017). The PUF can be accessed through the ACS website by anyone who has signed a data use agreement and has internal approval from an ACS NSQIP participating institution. There are multiple varieties of PUFs, including the standard PUF (containing all cases) as well as procedure-targeted PUFs for colectomy, gynecology, hepatectomy, hysterectomy, pancreatectomy, and vascular cases. The standard PUF contains >300 variables, including all demographic and relevant preoperative variables, as well as comprehensive 30-day outcome and complication measures.
      User Guide for the 2016 ACS NSQIP participant use data file (PUF).
      Procedure-targeted PUFs include more detailed variables about specific operations and can be merged easily with the standard PUF, but these targeted data are collected only at a subset of participating hospitals. The PUF contains no hospital identifiers and no personal health information. As such, retrospective research with the ACS NSQIP PUF is compliant with the Health Insurance Portability and Accountability Act and routinely exempt from Institutional Review Board approval. It also is important to note that, because of the data abstraction methods used in ACS NSQIP, data contained in PUF documents do not encompass all cases performed at ACS NSQIP hospitals. Rather, it is a random and representative sample of cases performed.

      Future Directions of ACS NSQIP

      ACS NSQIP continues to be the largest surgical quality improvement platform in the world. Considering the significant improvements experienced by participating hospitals, as well as the increasing national focus on quality improvement, we expect that ACS NSQIP will continue to grow and incorporate more domestic and international institutions in the future. As the largest surgical data registry, ACS NSQIP will continue to be at the cutting edge of data abstraction, quality control, and risk-adjustment.

      Summary

      ACS NSQIP is the culmination of a decades-long national focus on surgical quality and patient safety. Participation in ACS NSQIP provides powerful data feedback for at the hospital level, affording the opportunity for robust local and reginal quality improvement efforts and research. Beyond the actionable quality improvement data provided, ACS NSQIP also is well known as being the basis of one of the most powerful clinical research datasets in the world. These two opportunities to leverage ACS NSQIP for research purposes are distinct, with both mechanisms contributing significantly to improvements in the quality of surgical care during the past decade.

      References

        • Khuri S.F.
        • Daley J.
        • Henderson W.
        • et al.
        The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program.
        Ann Surg. 1998; 228: 491-507
        • Khuri S.F.
        • Henderson W.G.
        • Daley J.
        • et al.
        The patient safety in surgery study: background, study design, and patient populations.
        J Am Coll Surg. 2007; 204: 1089-1102
        • Cohen M.E.
        • Ko C.Y.
        • Bilimoria K.Y.
        • et al.
        Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus.
        J Am Coll Surg. 2013; 217 (e1): 336-346
        • Guillamondegui O.D.
        • Gunter O.L.
        • Hines L.
        • et al.
        Using the National Surgical Quality Improvement Program and the Tennessee Surgical Quality Collaborative to improve surgical outcomes.
        J Am Coll Surg. 2012; 214 (discussion 714-6): 709-714
        • Maggard-Gibbons M.
        The use of report cards and outcome measurements to improve the safety of surgical care: the American College of Surgeons National Surgical Quality Improvement Program.
        BMJ Qual Saf. 2014; 23: 589-599
        • Hollenbeak C.S.
        • Boltz M.M.
        • Wang L.
        • et al.
        Cost-effectiveness of the National Surgical Quality Improvement Program.
        Ann Surg. 2011; 254: 619-624
        • Lawson E.H.
        • Hall B.L.
        • Louie R.
        • et al.
        Association between occurrence of a postoperative complication and readmission: implications for quality improvement and cost savings.
        Ann Surg. 2013; 258: 10-18
        • Huffman K.M.
        • Cohen M.E.
        • Ko C.Y.
        • Hall B.L.
        A comprehensive evaluation of statistical reliability in ACS NSQIP profiling models.
        Ann Surg. 2015; 261: 1108-1113
      1. User Guide for the 2016 ACS NSQIP participant use data file (PUF).
        (October)2017