Abstract
Background
Meaningful reporting of quality metrics relies on detecting a statistical difference
when a true difference in performance exists. Larger cohorts and longer time frames
can produce higher rates of statistical differences. However, older data are less
relevant when attempting to enact change in the clinical setting. The selection of
time frames must reflect a balance between being too small (type II errors) and too
long (stale data). We explored the use of power analysis to optimize time frame selection
for trauma quality reporting.
Methods
Using data from 22 Level III trauma centers, we tested for differences in 4 outcomes
within 4 cohorts of patients. With bootstrapping, we calculated the power for rejecting
the null hypothesis that no difference exists amongst the centers for different time
frames. From the entire sample for each site, we simulated randomly generated datasets.
Each simulated dataset was tested for whether a difference was observed from the average.
Power was calculated as the percentage of simulated datasets where a difference was
observed. This process was repeated for each outcome.
Results
The power calculations for the 4 cohorts revealed that the optimal time frame for
Level III trauma centers to assess whether a single site’s outcomes are different
from the overall average was 2 years based on an 80% cutoff.
Conclusion
Power analysis with simulated datasets allows testing of different time frames to
assess outcome differences. This type of analysis allows selection of an optimal time
frame for benchmarking of Level III trauma center data.
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to SurgeryAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Benchmarking trauma centers on mortality alone does not reflect quality of care: implications for pay-for-performance.J Trauma Acute Care Surg. 2014; 76: 1184-1191
- Reliability adjustment: a necessity for trauma center ranking and benchmarking.J Trauma Acute Care Surg. 2013; 75: 166-172
- Ranking hospital performance based on individual indicators: can we increase reliability by creating composite indicators?.BMC Med Res Methodol. 2019; 19: 131
- Reliability of risk-adjusted outcomes for profiling hospital surgical quality.JAMA Surg. 2014; 149: 467-474
- Methodology and analytic rationale for the American College of Surgeons Trauma Quality Improvement Program.J Am Coll Surg. 2013; 216: 147-157
- Pull back the curtain: external data validation is an essential element of quality improvement benchmark reporting.J Trauma Acute Care Surg. 2020; 89: 199-207
- Reduction in venous thromboembolism events: trauma performance improvement and loop closure through participation in a state-wide quality collaborative.J Am Coll Surg. 2015; 221: 661-668
- The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 “negative” trials.N Engl J Med. 1978; 299: 690-694
- Power outage-inadequate surgeon performance measures leave patients in the dark.JAMA Surg. 2016; 151: 599-600
- Surgical mortality as an indicator of hospital quality: the problem with small sample size.JAMA. 2004; 292: 847-851
- The Michigan Trauma Quality Improvement Program: results from a collaborative quality initiative.J Trauma Acute Care Surg. 2017; 82: 867-876
- Regional collaborative quality improvement for trauma reduces complications and costs.J Trauma Acute Care Surg. 2015; 78 (discussion85–7): 78-85
- Bootstrapping within the Multilevel/Hierarchical Linear Modeling Framework: A Primer for Use with SAS and SPLUS.Vol. 30. Mult Linear Regres Viewpoints, 2004: 23-34http://www.glmj.org/archives/MLRV_2004_30_1.pdf#page=24Date accessed: July 2, 2022
- Resampling Methods of Estimation.in: Wright J.D. International Encyclopedia of the Social & Behavioral Sciences. Elsevier, London2015: 492-495
- American College of Surgeons Trauma Quality Improvement Program (ACS TQIP).ACS TQIP Benchmark Report: Fall 2021. American College of Surgeons TQIP, 2021: 5
- Trauma quality improvement.Crit Care Clin. 2017; 33: 193-212
- Association of hospital participation in a regional trauma quality improvement collaborative with patient outcomes.JAMA Surg. 2018; 153: 747-756
- Evaluation of the ProPublica Surgeon Scorecard “adjusted complication rate” measure specifications.Ann Surg. 2016; 264: 566-574
- A methodological critique of the ProPublica Surgeon Scorecard.Rand Health Q. 2016; 5: 1
- Comparing publicly reported surgical outcomes with quality measures from a statewide improvement collaborative.JAMA Surg. 2016; 151: 680-682
- Extending trauma quality improvement beyond trauma centers: hospital variation in outcomes among nontrauma hospitals.Ann Surg. 2022; 275: 406-413
Article info
Publication history
Published online: July 07, 2022
Accepted:
May 30,
2022
Identification
Copyright
© 2022 Elsevier Inc. All rights reserved.
