Editor's Note: The online version of this article contains the survey tool used in the study.

Introduction

The Liaison Committee on Medical Education and the Accreditation Council for Graduate Medical Education have identified teaching of medical students as an important part of residency.1,2 Studies show that medical students look to residents as their primary source of knowledge about patient care,3 with 30% of their overall knowledge attributed to coming from residents.4 Attendings report residents have a vital role in teaching,5 and that residents have a better understanding of medical practices if they teach other learners.6 Literature assessing students' opinions of their instructors is mixed, with 1 study reporting that up to a third of medical students had negative opinions of their residents' teaching performances.7

The best way to encourage and improve teaching from residents is still being evaluated, and little evidence is available.8,9 Residents' attitudes toward teaching could be 1 factor, although we could not find any literature assessing this, and the literature on feedback from students to residents also is sparse. Several small studies have shown a benefit to instituting a formal feedback mechanism10,11; however, we are unaware of large-scale studies in this area. The literature assessing validity evidence of student feedback of residents is also sparse, although there are several studies showing that students can reliably and validly rate faculty lecturers,12–18 as well as studies that show that student feedback affects faculty lecturers in a positive way.19–21

In this study we aim to broaden our understanding of feedback by assessing residents' awareness of student feedback, how feedback is received, and residents' attitudes toward student feedback. We also identify attitudes about teaching more commonly found in residents who seek feedback.

Methods

The Oregon Health & Science University (OHSU) is an academic tertiary referral hospital with 24 residency programs and the only medical school in the state of Oregon. OHSU uses several ways for residents to get student medical feedback, including an online system, at least twice yearly faculty advisor meetings, and informal opportunities for face-to-face feedback during rotations.

A preliminary survey was developed by K.K.T. and J.K. with input from local graduate medical education administrators. J.K. is involved in student curriculum development at OHSU and heads the neurology clerkship. A final survey was developed after 11 fellows completed the survey and provided feedback on ease of completion and comprehension of questions.

Surveys were sent to all 604 residents at our institution in December 2012 using an online-based question service (SurveyMonkey). Replies were linked to residents' e-mail addresses to ensure there were no duplicate responses. Weekly reminders were sent out for 3 weeks. Participation was voluntary; however, small prizes were offered to randomly selected respondents to encourage participation. Data were collected, deidentified, and tabulated by a member of our local graduate medical education office before being sent to the primary investigators for analysis.

The study was approved by the Institutional Review Board at OHSU.

Statistical analysis combined review of open-ended questions by authors on individual questions alongside contingency table assessments and rank correlations of fundamentally intriguing question pairs. Responses were evaluated using Pearson χ² test to determine whether the goodness-of-fit for the count distributions was significantly different from chance. The distributions were considered with both the original 5-level response (strongly disagree, disagree, neutral, agree, strongly agree) as well as a reduced 3-level scale (strongly disagree/disagree, neutral, strongly agree/agree; survey instrument provided as online supplemental material).

Question pairs were developed a priori for deeper examination of associations and trends. Assessment consisted of contingency table row and column independence using Fisher exact test (using reduced response scales comparing “agree” with “disagree”) and rank correlation using Kendall τ (tau, using the full 5-level response to calculate rank correlation). Fisher exact test determined the odds ratio of agreeing with one question in the pair given prior agreement with the other question. Kendall τ gave a more comprehensive assessment of association between questions using the full response scale. Repeated testing was controlled with Bonferroni corrections for each series of test modalities. Statistical analysis used R 2.15,22 with additional utility from the ggplot2,23 exact2×2,24 and Kendall25 packages.

Results

Of 604 residents, 335 responded (55%). One respondent (0.3%) was unable to be linked to a residency program due to a computer error.

The majority of residents (71.6%, 239 of 334) received some kind of feedback from medical students (table). Despite the requirement that advisors discuss student feedback with residents, the majority (72.2%, 242 of 335) stated this had not happened, and almost half of residents (44.2%, 148 of 335) were not aware of any formal feedback methods in their program. Residents who had received feedback reported a wide range of methods encountered (table).

TABLE Demographics of Respondents and Feedback

View Fullscreen

Almost all respondents agreed or strongly agreed that teaching is an important part of their job (93.1%, 312 of 335), and a large majority felt that teaching is a priority (73.4%, 246 of 335), and that student feedback should be collected (67.1%, 225 of 335). However, less than half agreed or strongly agreed that teaching affects their current role (47.2%, 158 of 335) or future job prospects (30.2%, 101 of 335), with a large proportion neither agreeing nor disagreeing (32.8%, 110 of 335 [current job]; 41.2%, 138 of 335 [future positions]). Although only a quarter of residents solicit feedback always or often, more than half would like feedback always or often (57.3%, 192 of 335).

Most of those who received student feedback (n  =  234) agreed or strongly agreed that feedback is useful (63.7%, 149 of 234) and has made them a better physician (53.8%, 126 of 234), and a large majority of those who have not received feedback agreed or strongly agreed that they would have liked feedback (69.9%, 65 of 93) and that feedback would have a beneficial impact on their performance (75.8%, 69 of 91). For those receiving feedback, the odds of agreeing with one statement were 152 times higher if they agreed with the other (95% CI 28–1574, P < .001), whereas the odds of joint agreement for residents who had not received feedback were 207 times higher (95% CI 13–13 300; P < .001; figure 1). Despite prior receipt of feedback, the vast majority of residents desired feedback sometimes, often, or always (93%, 137 of 147 [had prior feedback]; 81%, 77 of 95 [had no prior feedback]), whereas a minority desired it seldom or never. The odds of desiring student input often or always were 5.8 times higher if the resident had previously received feedback (95% CI 2–15; P < .001; figure 1).

View Figure

• Download as Powerpoint
• Download Figure

A moderate, significant positive association was observed between the desired frequency and soliciting feedback (Kendall τ: 0.27, P < .001), with the odds of soliciting feedback often or always 9.6 times higher if feedback was desired at the same frequency (95% CI 2–89, P < .001). However, of the 192 respondents who would like student feedback often or always, 31% reported they directly solicit it seldom or never (figure 1).

For residents who had not received feedback, there was little consensus on the effect of feedback on position, with 38% (36 of 95) agreeing it affected their current role and only 30% (28 of 95) believing feedback affected future prospects. Similarly, there was no significant association with the notion that feedback would have improved medical skill (figure 2). Conversely, of those who had received feedback, most believed student feedback affected their current position (51%, 122 of 239), although fewer believed it had a role in any future job placements (32%, 75 of 239). Unlike those who had not received feedback, having the opinion that feedback improved medical skill had significantly greater odds of agreeing that feedback had an impact on both current residency position (OR, 21; 95% CI 6–82; P < .001) and future job prospects (OR, 23; 95% CI 6–136; P < .001; figure 2).

View Figure

• Download as Powerpoint
• Download Figure

Reported barriers to feedback fell into 3 categories: student apprehension, time constraints, or perceived lack of a formal system. For residents aware of their program's feedback system (n  =  164), and those unaware (n  =  126), the largest proportion attributed the barriers to student apprehension (48%, 78 of 164 [aware]; 38%, 48 of 126 [unaware]). A belief that time constraints were the largest hurdle was more prominent for residents aware of their feedback systems (25%, 41 of 164 [aware]; 14%, 18 of 126 [unaware]); however, there was no significant difference in the odds of citing either barrier over the other based on feedback awareness. Of note, the likelihood of citing “lack of formal systems” was much greater if a resident was unaware of his or her department's formal feedback mechanisms (χ²  =  22.9, P < .001).

Discussion

This study finds that residents at a large academic institution viewed student feedback favorably, would like more of it, and felt it made them better physicians, regardless of whether they had or had not received feedback. Unfortunately, many residents were not receiving student feedback on their teaching.

Several small studies examined the effectiveness of feedback on resident teaching. One showed resident teaching skills improved with a formal teaching and feedback course,10 while another found improved resident teaching performance as well as an increased perception of the importance of teaching after implementation of a student feedback program.11 Because our study found that residents had favorable attitudes toward student feedback and felt it had made them better physicians, student feedback could be used to improve resident teaching performance, thereby improving both residents' and students' experiences.

Despite residents' positive attitudes toward feedback, many respondents in our study did not receive student feedback, and many were not aware of feedback mechanisms in place. While residents postulated several main reasons why students do not give feedback, the problem seems to be a multifactorial combination of residents' lack of awareness of available feedback systems, and resident advisors who do not adhere to feedback provision policies during resident meetings. Other factors likely include residents' perceptions that feedback does not affect their current or future prospects, or some degree of apprehension, because most residents did not endorse seeking out feedback despite their interest in it. Regardless of the underlying causes, a disconnect exists between what educators and residents would like to have happen and what is happening.

Our study has several limitations. Respondents may have misinterpreted questions, because our survey has not previously been used or validated; however, we attempted to mitigate this with feedback from fellows on our preliminary survey. Although OHSU is a large institution with diverse residencies, it is 1 institution and the findings may not generalize to other institutions or regions. Also, as in all surveys, participation was voluntary, which may have introduced selection bias, and there are the common risks in survey-based research of social desirability bias and recall bias.

Further research could investigate what interventions improve feedback mechanisms and increase resident awareness of those systems. Companion evaluation of students' perceptions of barriers to feedback could help clarify these discrepancies. We hope that this study will spawn future projects, including a larger survey involving institutions across the United States.

Conclusion

Our findings showed that residents are willing participants in the education of medical students, and it supports the use of student feedback as a way to improve the educational experience for residents and students.

Residents who did receive feedback from students were more likely to have actively solicited it. Overall, residents believe that feedback from students on their teaching abilities would benefit their clinical and teaching performance.