Central Surgical Association| Volume 128, ISSUE 4, P613-622, October 2000

Download started.


Evaluating surgical competency with the American Board of Surgery In-Training Examination, skill testing, and intraoperative assessment


      Background. Evaluation of surgical competency should include assessment of knowledge, technical skill, and judgment. The purpose of this study was to determine the relationship between the American Board of Surgery In-Training Examination (ABSITE), skill testing, and intraoperative assessment. Methods. Postgraduate year 2 (PGY-2) and postgraduate year 3 (PGY-3) surgery residents (n = 33) were tested by means of (1) the ABSITE, (2) skill testing on a laparoscopic video-trainer, and (3) intra-operative global assessments during laparoscopic cholecystectomy. The Pearson correlation was used to determine the correlation between the ABSITE, skill testing, and intraoperative assessments. For the comparison of PGY-2 and PGY-3 resident performance, Wilcoxon rank sum tests were used. Results. The ABSITE scores did not correlate with skill testing or intraoperative assessments (not significant). Skill testing correlated with the intraoperative composite score and with 4 of 8 operative performance criteria (P <.05). The ABSITE scores and skill testing were not different for PGY-2 and PGY-3 residents (not significant). Intraoperative assessments were better in 5 of 8 criteria and the composite score for PGY-3 versus PGY-2 residents (P <.05), which demonstrated construct validity. Conclusions. The ABSITE measures knowledge but does not correlate with technical skill or operative performance. Residency programs should use multiple assessment instruments to evaluate competency. There may be a role for both skill testing and intraoperative assessment in the evaluation of surgical competency. (Surgery 2000;128:613-22)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Surgery
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Anderson CI
        • Ilgenfritz FM
        • Dean R.
        External regulation, resident performance, and quality improvement.
        Curr Surg. 1994; 51: 447-450
        • Schwartz RW
        • Sloan DA
        • Griffen WO
        • Bland KI
        • Rhodes RS
        • Strodel WE.
        The necessity, practicality, and feasibility of modern educational and evaluative methods for residency training: economic and governing body perspectives, the 1994 Association for Academic Surgery panel on education.
        Curr Surg. 1997; 54: 261-269
        • Bridges M
        • Diamond D.
        The financial impact of teaching surgical residents in the operating room.
        Am J Surg. 1999; 177: 28-32
        • Martin M
        • Vashisht B
        • Frezza E
        • Ferone T
        • Lopez B
        • Pahuja M
        • et al.
        Competency-based instruction in critical invasive skills improves both resident performance and patient safety.
        Surgery. 1998; 124: 313-317
        • Grosse ME
        • Cruft GE
        • Blaisdell FW.
        The American Board of Surgery in-training examination.
        Arch Surg. 1980; 115: 654-657
        • Schwartz RW
        • Donnelly MB
        • Sloan DA
        • Johnson SB
        • Strodel WE.
        The relationship between faculty ward evaluations, OSCE, and ABSITE as measures of surgical intern performance.
        Am J Surg. 1995; 169: 414-417
        • Schwartz RW
        • Witzke DB
        • Donnelly MB
        • Stratton T
        • Blue AV
        • Sloan DA
        Assessing residents' clinical performance: cumulative results of a four-year study with the Objective Structured Clinical Examination.
        Surgery. 1998; 124: 307-312
        • Anastakis DJ
        • Regehr G
        • Reznick RK
        • Cusimano M
        • Murnaghan J
        • Brown M
        • et al.
        Assessment of technical skills transfer from the bench training model to the human model.
        Am J Surg. 1999; 177: 167-170
        • Martin JA
        • Regehr G
        • Reznick R
        • MacRae H
        • Murnaghan J
        • Hutchison C
        • et al.
        Objective structured assessment of technical skill (OSATS) for surgical residents.
        Br J Surg. 1997; 84: 273-278
        • Reznick R
        • Regehr G
        • MacRae H
        • Martin J
        Testing technical skill via an innovative “bench Station” examination.
        Am J Surg. 1996; 173: 226-230
        • Winckel CP
        • Reznick RK
        • Cohen R
        • Taylor B.
        Reliability and construct validity of a structured technical skills assessment form.
        Am J Surg. 1994; 167: 423-427
        • Scott DJ
        • Bergen PC
        • Euhus DM
        • Guo W
        • Jeyarajah DR
        • Laycock R
        • et al.
        Intense laparoscopic skills training improves operative performance of surgery residents.
        in: American College of Surgeons Surgical Forum. L. 1999: 670-671
        • Rosser JC
        • Rosser LE
        • Savalgi RS.
        Objective evaluation of a laparoscopic surgical skill program for residents and senior surgeons.
        Arch Surg. 1998; 133: 657-661
        • Jones DB
        • Brewer JD
        • Soper NJ.
        The influence of three-dimensional video systems on laparoscopic task performance.
        Surg Laparosc Endosc. 1996; 6: 191-197
        • Risucci DA
        • Tortolani AJ.
        A methodological framework for the design of research on the evaluation of residents.
        Acad Med. 1990; 65: 36-41
        • Newble DI.
        Assessing clinical competence at the undergraduate level.
        Med Educ. 1992; 26: 504-511
        • Biester TW.
        The American Board of Surgery In-training Examination as a predictor of success on the Qualifying Examination.
        Curr Surg. 1987; 44: 194-198
        • Maxim BR
        • Dielman TE.
        Dimensionality, internal consistency, and interrater reliability of clinical performance ratings.
        Med Educ. 1987; 21: 130-137
        • Scott DJ
        • Bergen PC
        • Euhus DM
        • Guo W
        • Jeyarajah DR
        • Laycock R
        • et al.
        Comparison of edited videotape to direct observation to assess operative performance after laparoscopic skills training.
        J Surg Res. 1999; 86 ([abstract]): 291
        • Derossis AM
        • Fried GM
        • Abrahamowicz M
        • Sigman HH
        • Barkun JS
        • Meakins JL.
        Development of a model for training and evaluation of laparoscopic skills.
        Am J Surg. 1998; 175: 482-487
        • Macmillan AIM
        • Cuschieri A.
        Assessment of innate ability and skills for endoscopic manipulations by the advanced Dundee endoscopic psychomotor tester: predictive and concurrent validity.
        Am J Surg. 1999; 177: 274-277
        • Scott DJ
        • Young WN
        • Tesfay ST
        • Jones DB.
        Medical students learn more than residents during structured laparoscopic skills training.
        Surg Endosc. 2000; 14 ([abstract]): S217
      1. Bergen PC, Littlefield JH, O'Keefe GE, Rege RV, Anthony TA, Kim LT, et al. Identification of high-risk residents. J Surg Res. In press.