Study Design and Conceptual Framework

We conducted a qualitative study using the constant comparative method associated with grounded theory.¹⁹ This entails a systematic approach to discover themes and develop an explanation of a social phenomenon or process grounded in the data. Consistent with grounded theory, we had no a priori hypotheses, and used social cognitive theory (SCT)²⁰ as a sensitizing framework²¹ from which to examine the data. The SCT recognizes the role played by the clinical environment, relationships with supervisors and patients, and individual knowledge and belief systems in learning, and it has been used to understand learning from multisource feedback.^16,17 We used SCT to direct us to important features of the data from which we could build categories and themes.²¹

Participants

Between June and July 2016, we recruited a stratified, purposive²² sample of pediatrics residents affiliated with the University of Chicago, Phoenix Children's Hospital, and Stanford University. Eligible residents had participated in a randomized controlled trial from June 2015 to June 2016 in which they received written patient and family feedback and either discussed this feedback with a faculty clinical advisor (intervention) or reviewed it on their own (control). Intervention-group residents also had their advisor observe 2 patient encounters and solicit feedback from these patients or families on the resident's behalf to discuss during their meeting. All feedback was gathered using the Communication Assessment Tool (CAT).²³ The CAT asks respondents to rate 14 dimensions of a physician's communication and interpersonal skills using a 5-point scale (1, poor, to 5, excellent) and has evidence of validity when used with physicians.²³ We modified the CAT to include 2 open-ended questions for details regarding residents' skills: (1) “What did you like about this resident's communication?” and (2) “How can this resident improve?” Feedback was collected in person by research assistants using a mobile device or in paper form.

Our focus groups included only residents who received completed CAT forms from 3 or more patient or family encounters. To explore how feedback influenced learning, we held separate focus groups with residents in the intervention and control groups. An aggregated written report of patient and family feedback was e-mailed to each resident prior to focus group recruitment to prompt recall.

Data Collection and Instrument

Eligible residents (n = 92 for all sites) were e-mailed invitations to participate in a 60-minute focus group. For consistency at each site, we assigned 1 nonphysician facilitator who was not involved in the study to lead the focus group. Facilitators were blinded to the groups (control versus intervention). Prior to the focus groups, facilitators participated in a 90-minute training session led by 1 author (A.L.B.). Facilitators were encouraged to use probes to enable deeper reflection and exploration of the topics discussed.

The semistructured interview guide (provided as online supplemental material) was developed from a literature search on multisource feedback and reviewed for content by the research team. Prior to implementation, we pilot-tested the guide with 4 noneligible residents for clarity and flow. The final guide included 5 open-ended questions to elicit residents' perspectives on patient feedback and to explore factors that facilitate learning from this feedback.

This study was approved by the Institutional Review Boards at the University of Chicago, Phoenix Children's Hospital, and Stanford University. Verbal consent was obtained from all residents prior to the focus groups.

Data Analysis

Focus groups were audiorecorded and transcribed verbatim. Data were deidentified and uploaded into Dedoose 7.1 (Sociocultural Research Consultants, Los Angeles, CA) for analysis. We used constant comparison and an iterative process of open, axial, and selective coding to uncover themes from the focus groups.¹⁹ Two authors (A.L.B. and N.O.) read the first 3 transcripts to independently generate an inductive list of codes. We refined the codes, and combined them into a single coding structure that was used to independently analyze the remaining transcripts, meeting after each analysis to discuss, add, or remove codes. We determined that codes were saturated after analysis of the fifth transcript and applied this coding structure to the final 2 transcripts without further discussion.

We used SCT to direct our attention to specific interpersonal and environmental factors that affected feedback delivery and learning. We examined the codes through the lens of the learner as well as the relational, environmental, and clinical factors (ie, axial and selective coding); we then discussed these relationships to combine codes and refine categories to generate the final list of themes.

To assess coding reliability, we calculated the Cohen kappa on a subset of transcripts after finalizing our coding structure (k = 0.92; ie, after analysis of the fifth transcript).²² We also used Dedoose to quantitatively compare code applications across control and intervention groups and sites. We observed that each code was discussed in all focus groups. We shared our codes, categories, and representative quotations with the research team and asked for their perspectives on the accuracy of the themes. As a final step, a subset of participants from all institutions reviewed and commented on the themes. All agreed that we accurately conceptualized the process of feedback consumption and illustration by faculty.

Patient and Family Feedback Is Unique and Important but May Lack Specificity and Individualization

Residents reported that patient and family feedback offered a unique and important perspective on their communication and interpersonal skills compared with feedback from faculty and peers. Yet many indicated that patient feedback lacked specificity and individualization, which some attributed to the feedback collection tool, and they commented that numeric Likert-scale ratings were difficult to interpret and translate into learning goals. They felt ratings “lacked meaning” and struggled to identify specific skills to improve on to “move from good to excellent” on a given item. In addition, residents commented that open-ended or verbal feedback also lacked the specificity they needed for learning.

Discussion About Feedback With a Trusted Faculty Member Is Helpful for Self-Reflection and Learning

Residents in both study arms reported that discussing patient feedback with a trusted faculty member was helpful for self-reflection and learning, particularly in light of the challenges they had in interpreting the feedback they received. Many residents noted that these discussions facilitated their recall of patient encounters, which helped them consider ways to transform ratings or vague comments into specific learning goals. Some residents believed trust could only be developed through longitudinal relationships with faculty, while others believed trust could be built during a single discussion. Residents in the control arm, who did not participate in the feedback discussion with an advisor, frequently spoke about sharing patient feedback with their supervising physician as a way to better understand patients' perspectives and identify areas for behavioral change.

Faculty Employ 5 Key Strategies to Promote Resident Openness to and Acceptance of Patient Feedback

Residents identified 5 key strategies faculty used during these discussions to promote trainee openness to and acceptance of patient and family feedback (table 3). These were to (1) situate feedback in the context of the clinical encounter; (2) help residents identify and overcome emotional responses to negative feedback; (3) develop individualized, specific goals from feedback provided; (4) frame feedback in the context of lifelong learning; and (5) hold residents accountable for behavior change. Collectively, these strategies achieved important goals. For example, when faculty situated the feedback in the context of a clinical encounter, residents could more easily reflect on their own behavior and how it affected patients' experiences and the feedback received. This motivated trainees to take “greater ownership” of the feedback and recognize its relevance to their practice.

table 3 Five Resident-Identified Strategies for Faculty to Promote Resident Openness to and Learning From Patient Feedback

View Fullscreen

Many residents described reacting to negative ratings or comments with frustration, surprise, or disavowal, and they commented that discussion with faculty ameliorated these reactions and helped them remain open to the feedback. Residents also indicated that faculty played a critical role in clarifying the value of patient feedback for their lifelong development and in helping them set and stay accountable for learning goals.

Faculty Observation of Patient Encounters Improved Resident Perceptions of Feedback Credibility

Residents reported that the credibility and utility of patient and family feedback improved when faculty observed patient encounters, solicited written and verbal feedback from the patient or family on the resident's behalf, and shared this feedback with the resident. Residents who participated in the intervention arm of the study commented about the value of this experience. Residents in the control arm spoke of the perceived benefits of having a “third-party faculty member” participate in this process.