SIMPLifying Urology Residency Operative Assessments: A Pilot Study in Urology Training

Robert S Wang, Stephanie Daignault-Newton, Sapan N Ambani, Khaled Hafez, Brian C George, Kate H Kraft

Journal of Urology, October 2021

Abstract

Purpose: The Society for Improving Medical Professional Learning (SIMPL) app is an innovative, convenient and validated smartphone-based tool to evaluate residents’ operative performance. In this study, we describe the initial implementation of SIMPL in our program’s pediatric urology rotation-the first among urology residencies-and provide preliminary data on its adoption by residents and faculty.

Materials and methods: Residents and faculty in our pediatric urology division submitted SIMPL evaluations following surgical cases from August 2019 to July 2020. Evaluations consisted of ratings in 3 domains: resident autonomy, resident operative performance and patient-related case complexity. An online survey was also used to gauge attitudes towards SIMPL, describe patterns of use and solicit feedback on areas for improvement.

Results: Eight residents and 6 faculty submitted 141 evaluations, with 76.6% of evaluated cases having both faculty and resident ratings. Verbal feedback was included in 94.2%. Faculty-resident agreement ranged from 68.6% to 75.2% (kappa=0.47 to 0.61). Faculty rated postgraduate year (PGY)-4 residents as more autonomous (p=0.040) and higher performing (p=0.028) than PGY-3 residents. All participants agreed that SIMPL was easy to use and compared favorably to existing avenues of feedback. Barriers to implementation included lack of reminders for evaluations and evaluation fatigue.

Conclusions: The SIMPL application improved both frequency and quality of resident operative feedback. Among participants, SIMPL was preferred over the existing feedback system at our institution.

PMID: 34032501