Electronic Thesis and Dissertation Repository

Thesis Format

Monograph

Degree

Master of Science

Program

Surgery

Supervisor

Grant, Aaron

2nd Supervisor

Ott, Michael

Co-Supervisor

3rd Supervisor

Ross, Douglas

Co-Supervisor

Abstract

Introduction: A central tenet of competency-based medical education is formative assessment of trainees. There are no assessments examining resident competence on-call, despite this being a significant component of resident training and characterized by less supervision compared to daytime.

Methods: A national survey was conducted to evaluate the state of assessment in Canadian Plastic and Reconstructive Surgery programs. An on-call assessment tool was developed based on a consensus group and was piloted over six months. Validity of the tool was examined through qualitative and quantitative methods.

Results: There were 63 tools completed across ten residents and seven staff physicians. Tool reliability was 0.67 and scores were significantly correlated to year of training. Staff and residents considered the tool useful, feasible and acceptable.

Conclusions: The on-call assessment tool has multiple sources of validity evidence to support its purpose of assessing surgical resident competence on-call. Further research is required to assess tool generalizability.

Summary for Lay Audience

The training of a surgeon is complex. In Canada, resident physicians become independent practicing surgeons by being immersed in a five-year work-based curriculum during which they receive didactic teaching as well as supervised hands-on experience. Assessment of knowledge and performance is important in determining whether a trainee is progressing as expected and to provide feedback to enhance future performance. The current residency curriculum focuses on frequent, low-stakes assessment of trainees in the workplace. However, currently there are few assessments during the on-call period when supervisors are often not present, and residents function with greater autonomy.

We surveyed residents and program directors from all Canadian Plastic & Reconstructive Surgery training programs and confirmed that there is a lack of assessment on-call. Residents and program directors believed a more formal way of assessment would be beneficial.

A tool was developed with input from surgeons experienced in medical education that could be used to assess resident performance on-call. This tool was piloted in the Division of Plastic & Reconstructive Surgery in London, Ontario. Ten residents were assessed by seven staff physicians across 63 instances. The tool was able to differentiate between residents of advancing training level. More occasions of scoring will be needed to improve reliability of the tool.

We interviewed four residents and three staff physicians who participated in the pilot to better understand the utility and impact of the tool. Analysis of the interview transcripts revealed there was a positive impact on the amount of feedback given as well as standardization of the feedback process. In addition, the pilot results suggested potential refinements that could be made to improve the practicality of the assessment. However, there was general agreement the tool design was acceptable and useful.

Overall, this thesis offers a better understanding of the landscape of feedback and assessment on-call for surgical trainees. It also provides an assessment tool that can be used to facilitate feedback and learning on-call. Further work should be done to see if this tool is more broadly applicable.

Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.

Download

Share

COinS