Examining the Impact of Using the SIMPL Application on Feedback in Surgical Education

Kirsten Gunderson, BS, Sarah Sullivan, PhD, Charles Warner-Hillard, MSPH, Ryan Thompson, MD, Jacob A. Greenberg, MD, EdM, Eugene F. Foley, MD, Hee Soo Jung, MD

Journal of Surgical Education, 2018

Abstract

Objective

The System for Improving and Measuring Procedural Learning (SIMPL) smartphone application allows physicians to provide dictated feedback to surgical residents. The impact of this novel feedback medium on the quality of feedback is unknown. Our objective was to compare the delivery and quality of best-case operative performance feedback given via SIMPL to feedback given in-person.

Design

We collected operative performance feedback given both in-person and via SIMPL from surgeons to residents over 6 weeks. Feedback transcripts were coded using Verbal Response Modes speech acts taxonomy to compare the delivery of feedback. We evaluated quality of feedback using a validated resident survey and third-party assessment form.

Setting

University of Wisconsin School of Medicine and Public Health, a large academic medical institution.

Participants

Four surgical attendings and 9 general surgery residents.

Results

Nineteen SIMPL and 18 in-person feedback encounters were evaluated. Feedback via SIMPL was more directive (containing thoughts, perceptions, evaluations of resident behavior, or advice) and contained more presumptuous utterances (in which the physician reflected on and assessed resident performance or offered suggestions for improvement) than in-person feedback (p = 0.01)

The resident survey showed no significant difference between the quality of feedback given via SIMPL and in-person (p = 0.07). The mean score was 47.74 (SD = 3.00) for SIMPL feedback and 45.33 (SD = 4.77) for in-person feedback, with a total possible score of 50.

Third-party assessment showed no significant difference between the quality of feedback given via SIMPL and in-person (p = 0.486). The mean score was 23.40 (SD = 3.75) for SIMPL feedback and 22.25 (SD = 5.94) for in-person feedback, with a total possible score of 30.

Conclusions

Although feedback given via SIMPL was more direct and based on the attendings’ perspectives, the quality of the feedback did not differ significantly. Use of the dictation feature of SIMPL to deliver resident operative performance feedback is a reasonable alternative to in-person feedback.

Full text link: https://www.jsurged.org/article/S1931-7204(18)30210-1/fulltext(18)30210-1/fulltext