High-Value Consults: A Curriculum to Promote Point-of-Care, Evidence-Based Recommendations
ABSTRACT
Background
In an era when value-based care is paramount, teaching trainees to explicitly communicate the evidence behind recommendations fosters high-value care (HVC) in the consultation process.
Objective
To implement an HVC consult curriculum highlighting the need for clear consult questions, evidence-based recommendations to improve consult teaching, clinical decision-making, and the educational value of consults.
Methods
A pilot curriculum was implemented for residents on cardiology consult electives utilizing faculty and fellows as evidence-based medicine (EBM) coaches. The curriculum included an online module, an EBM teaching point template, EBM presentations on rounds, and “coach” feedback on notes.
Results
A total of 15 residents and 4 fellows on cardiology consults participated, and 87% (13 of 15) of residents on consults felt the curriculum was educationally valuable. A total of 80% (72 of 90) of residents on general medicine rotations responded to the survey, and 25 of 72 residents (35%) had a consult with the EBM template. General medicine teams felt the EBM teaching points affected clinical decision-making (48%, 12 of 25) and favored dissemination of the curriculum (90%, 72 of 80). Checklist-guided chart review showed a 22% improvement in evidence-based summaries behind recommendations (7 of 36 precurriculum to 70 of 146 charts postcurriculum, P = .015).
Conclusions
The HVC consult curriculum during a cardiology elective was perceived by residents to influence clinical decision-making and evidence-based recommendations, and was found to be educationally valuable on both parties in the consult process.
Introduction
The interface between generalists and specialists is an opportunity to enhance high-value care (HVC). In an era when value-based care is paramount, it is essential to teach trainees to be deliberate about the quality of consult questions, recommendations, and their foundation in evidence-based medicine (EBM).1 High-value care is a complex concept that involves applying evidence to determine value relative to the risk associated with a clinical decision, in terms of both cost and benefit. When a consulting team is asked to see a patient, interdisciplinary communication is an added challenge to the already difficult task of HVC.2,3
The primary team requesting the consult should be able to weigh the risks and benefits of recommendations from the specialist(s) and ensure streamlined communication to the patient. Knowing the evidence behind these recommendations can ensure an HVC decision is made. Prior studies showed a positive impact on directly placing evidence into charts2–5 to aid in the education and utilization of HVC. The American College of Physicians developed online modules targeting fellows in the consultation process.3 Beyond this, we did not find point-of-care curricular innovations in the literature utilizing the consult electives for overarching HVC or point-of-care EBM skills for residents.2,6,7
We developed, implemented, and evaluated a resident-driven HVC consult curriculum. We hypothesized that participation in the curriculum would (1) improve the quality of consult notes written by residents, as confirmed by increased use of evidence in the recommendations provided; (2) enhance clinical decision-making; and (3) increase the perceived educational value to the primary team placing the consult.
Methods
Settings and Participants
At the University of Pittsburgh Medical Center, we developed an HVC consult curriculum focusing on communication of evidence in notes and on rounds to promote evidence-based recommendations. We piloted the curriculum to 15 residents and 4 fellows who rotated on the cardiology service from October 2015 to February 2016.
Intervention
We developed a multimodal curriculum comprising 4 components that can be incorporated into different teaching styles. These components focus on self-directed learning behaviors, with feedback from cardiology faculty and fellows who served as “coaches.”
Interactive Virtual Patient Introductory Module
Residents participated in a 15-minute module reviewing an approach to EBM based on Knowles' principles of self-directed learning. We modified the traditional 5 A's (assess, ask, acquire, appraise, apply) to a new framework called the IDEAS framework (Identify a knowledge gap, Define a clinical question, Examine the evidence, Apply, and Summarize out loud; figure 1). Fellows and faculty reviewed a parallel module that was geared toward understanding residents' expectations, and used the IDEAS framework to coach learners.
Template for EBM Teaching Points
To streamline teaching points, an electronic health record (EHR) EBM template (dot phrase) was created so that whenever “. EBM” was typed, the following template appeared:
-
Evidence and Educational Points:
-
Clinical Question:
-
Teaching Point: Include how this applies to this patient and a summary of the evidence
-
Resource: List your resource, or if this is an expert opinion
Our intervention increased the visibility of the teaching points for the primary team and cardiology faculty or fellow giving feedback. Expectations were set to include a teaching point within the initial consult note for every new consult seen by a resident. Faculty and fellows were asked to discuss the teaching points with the residents to assess quality and application of the evidence to the consult.
EBM Pearl Presentation
Each learner was expected to do biweekly 2-minute EBM presentations on rounds, including a clinical question, summary of the evidence in a 30-second sound bite, application to the patient, and citation of source used.
Consult Note Feedback Checklist
Our subspecialty education committee (made up of educational leaders from each medicine specialty) developed a checklist reviewing components of a high-quality assessment and plan for a consult note.
Outcomes
We conducted a literature search for point-of-care EBM consult practices and found no clear consensus on a measure with evidence of validity. An expert panel of regional EBM medical educators reviewed and pilot tested our survey for clarity of questions and answers. General feedback on content and clarity of the survey was also provided, and the survey was modified. The survey was designed to assess use of the template, barriers to communication of evidence-based recommendations, educational value, and the impact on clinical management from residents' perspectives. We debriefed with residents on the consult service at the end of each block.
All residents on general medicine inpatient teams were given anonymous surveys at the end of their rotation (after 4 months on service). They were asked to evaluate the concept of the overall EBM template. A subset of medicine residents had consults that included the EBM teaching pearl. They were surveyed about the educational value of the EBM template, and how it changed their clinical management.
A precurriculum and postcurriculum chart review using a checklist to evaluate the evidence-based content of the note was conducted to assess the impact of the curriculum. The checklist included use of a citation, summary of the evidence, and application of evidence to the patient, as well as a grading system of the quality of the consult question. Precurriculum charts for the 3 months prior to implementation of the curriculum, and postcurriculum charts for the pilot period from October to February were downloaded from the EHR on weekdays. All consult notes written by residents were reviewed using the checklist by 2 independent reviewers (D.R.N. and A.K.), with a calculated percent agreement of 87%. Consult questions were categorized into low and high quality based on criteria developed by Conley et al.8
This curriculum was approved by the University of Pittsburgh Medical Center Quality Review Committee.
Statistical Analysis
All survey information was entered into REDCap (Vanderbilt University, Nashville, TN) and analyzed with STATA 14 (StataCorp LLC, College Station, TX). Chi-square and Fisher's exact tests were used for mean comparisons between groups for the survey analysis and chart review. Statistical analysis and design were supervised by the University of Pittsburgh Data Center under the guidance of a master statistician.
Results
Postcurriculum Survey of Resident and Fellow Attitudes and Behaviors
All 15 residents and 4 fellows on cardiology consults participated in the curriculum and responded to the postcurriculum survey. A total of 72 of 90 residents (80%) on general medicine rotations responded to the postcurriculum survey; of these, 25 of 72 responses (35%) had a consult with the EBM template. A total of 80% (12 of 15) of the residents on the consult service agreed that writing teaching points was educationally valuable for themselves. They discussed the EBM pearls with the team requesting the consult, and 73% (11 of 15) agreed it was educationally valuable to the primary team. From the primary team perspective, 89% (64 of 72) wanted other consult services to use the template. Of the 25 residents with an EBM template in a consult note, 48% (12 of 25) felt it influenced clinical decision-making. Postcurricular attitudes, barriers, and reported behaviors for cardiology consult residents, fellows, and general medicine teams are displayed in Tables 1 and 2.
Citation: Journal of Graduate Medical Education 9, 5; 10.4300/JGME-D-17-00177.1
Citation: Journal of Graduate Medical Education 9, 5; 10.4300/JGME-D-17-00177.1
Cardiology Consult Small Group Debriefing Comments
During debriefing sessions, participants discussed the challenge of writing answerable questions, which required more time, a potential barrier to using the EBM template. They also mentioned the challenge of balancing expert opinion when it did not agree with the evidence. When an attending disagreed with an EBM teaching pearl, many times the resident would just delete this portion from his or her note. Residents found it valuable when attending physicians discussed why they disagreed with the evidence. Having the teaching points in the note reminded attendings and fellows to provide feedback on the evidence they were using as it became a part of the EHR.
Checklist-Guided Chart Review
The checklist-guided chart review gauged whether residents were including evidence within their notes prior to this curriculum. We assessed whether the EBM template in the EHR was necessary, or whether emphasizing the importance of evidence alone was enough to catalyze behavior change to include evidence within the consult note. A total of 36 historical charts prior to the initiation of the curriculum and 148 postcurricular charts were reviewed. Residents were asked to use the EBM template, but they often forgot to or chose not to utilize it. We compared evidence-based content of postcurriculum notes with and without the template. Within the historical control of 36 charts, 25% (n = 9) of the notes included a summary and application of the evidence, and 17% (n = 6) included a citation. Of the 148 postcurriculum charts reviewed, 54 notes (36%) included the EBM template. A summary and application of the evidence were included in 47% (n = 70) of the notes, and 42% (n = 62) included a citation (P = .015). On closer analysis, 50 of 70 charts (71%) with evidence-based summaries and patient application used the EBM template. Of the 78 charts with no evidence-based content, 81% (n = 63) of these were categorized as low-quality questions (n = 184, P = .0001).
Discussion
We found the HVC consult curriculum was perceived by residents to affect clinical decisions, to increase communication of evidence, and to be educationally valuable. As the EHR template increased EBM in notes, structured changes through templates may be more effective than solely teaching about the importance of evidence-based notes.
Utilizing evidence as the foundation of an HVC consult may provide multiple opportunities to increase residents' comfort with presenting recommendations to the primary team in a way that promotes value. Residents asked more questions about the evidence and indications for studies and procedures. The identification of evidence in notes and on rounds gave “coaches” insight into the quality of sources that residents were using in clinical recommendations. The most challenging aspect of implementing the curriculum was faculty development and buy-in. From an information technology perspective, it was fairly easy to create a new template that residents can use to include the EBM pearl.
Since completion of this study, we have expanded the curriculum to all subspecialties. We did 30-minute presentations at subspecialty faculty meetings, and we presented the curriculum with the IDEAS framework. The online module helped with sustainability, and it was easy to e-mail reminders of the curriculum for each block.
Nationally, as nearly 75% of residents go into a subspecialty fellowship, developing HVC consult skills in training can promote better utilization of consultants.9 We developed an HVC consult curriculum model that combines components of HVC with EBM, and overlays the complexity of shared decision-making with interdisciplinary communication (figure 2). Using these core components, we offer examples of curricular targets that span multiple aspects of the HVC consult equation that we plan on using to further enhance this curriculum.
There are limitations to this study. We conducted a single institution pilot on the cardiology consult service, and the sample was small and limited to cardiology. Although the survey was modified after testing, additional validity evidence would be needed to ensure participants interpreted each question as we intended. The chart checklist reviewers were not blinded to the research question, which could have allowed for biased review. Assessing clinical impact was done only through resident perceptions that the EBM pearl influenced clinical management.
Conclusion
Our data showed that residents perceived an HVC consult curriculum during a cardiology elective to influence clinical decision-making and evidence-based recommendations for both parties in the consult process, and found it to be educationally valuable.
IDEAS (Identify a knowledge gap, Define a clinical question, Examine the evidence, Apply, and Summarize out loud) Teaching Framework
Model for High-Value Consults Curricula
Author Notes
Funding: This research was funded by a University of Pittsburgh Medical Center Division of General Internal Medicine Data Center Grant.
Conflict of interest: The authors declare they have no competing interests.
This project has been presented as an oral presentation at the Society of General Internal Medicine Annual Meeting, Hollywood, Florida, May 11–14, 2016.
