Surveys are frequently used to collect data in graduate medical education (GME) settings.1 However, if a GME survey is not rigorously designed, the quality of the results is likely to be lower than desirable. In a recent editorial we introduced a framework for developing survey instruments.1 This systematic approach is intended to improve the quality of GME surveys and increase the likelihood of collecting survey data with evidence of reliability and validity. In this article we illustrate how researchers in medical education may operationalize this framework with examples from a survey we developed during the recent
Skip to main content
Issues
Issues
Select Issue
FROM THE EDITOR
MD, MPH,
MD,
MD, MPH, and
PhD
PhD
Editing a journal is all about language. Authors' thoughts and experiences—their message—must be translated into text, revised, and edited to convey to the readers the concepts intended by the authors. Some concepts are complex and need powerful language to be expressed.
A competency-based education movement is dominating graduate medical education reform around the world in the 21st century. This movement can be viewed as a struggle to formulate the values of the medical profession in words that enable a new perspective on curricula, training, and assessment. Without new language, this innovation would not have been possible. The 7 roles of
From the Editors
Many of the articles in this issue focus on competency-based education and the roll-out of the Milestones in the 7 specialties entering the Next Accreditation System in July 2013.
In the editorial section, ten Cate discusses the power of language and the value of entrustable professional activities (EPAs) in competency-based education (pp. 6). In a second editorial Magee and colleagues present the second in a series of articles on survey design, focusing on the application of a survey design framework to assess the integration of 2 internal medicine programs (pp. 1).
Perspectives
Cianciolo and Kegg discuss the
PERSPECTIVES
PhD and
MD, FAAEM
Never trust anything that can think for itself if you can't see where it keeps its brain.
JK Rowling, Harry Potter and the Chamber of Secrets
The judicious entrustment of professional activities is a challenging process, fraught with uncertainty. Yet, it is the essence of clinical teaching and practice-based learning and improvement. Effective clinical teaching and supervision require an evidence-based approach to entrustment that should exemplify a broader, organizational approach to patient safety and quality improvement. Yet, relatively little is known about how supervisors use information gleaned from direct observations of trainee competence in entrustment decision making. A deeper
MA,
DO,
MSLIS, JD, and
MD
Abstract
While video is a powerful teaching and learning tool because it can influence knowledge, skills, and attitude formation effectively and reach learners with various learning and communication styles, there are pedagogical, technical, and copyright considerations. Instructors must know sources of appropriate videos, select effective video segments, apply various strategies for incorporating video triggers into the overall educational process, refine the message, overcome technological obstacles, and comply with copyright laws. One might ask, “Is using video triggers to improve your teaching worth it?” “Yes!” Numerous studies demonstrate that using video in many medical education settings supports and enhances learning and offers a bigger advantage in contrast with traditional methods.
REVIEW
MD, MPH(c),
MD, MSPH,
MD, and
MD
Abstract
Background
Resident duty hour limits have been a point of debate among educators, administrators, and policymakers alike since the Libby Zion case in 1984. Advocates for duty hour limits in the surgical subspecialties cite improvements in patient safety, whereas opponents claim that limiting resident duty hours jeopardizes resident education and preparedness for independent surgical practice.
Methods
Using orthopaedic surgery as an example, we describe the historical context of the implementation of the duty hour standards, provide a review of the literature presenting data that both supports and refutes continued restrictions, and outline suggestions for policy going forward that prioritize patient safety while maintaining an enhanced environment for resident education.
Results
Although patient safety markers have improved in some studies since the implementation of duty hour limits, it is unclear whether this is due to changes in residency training or external factors. The literature is mixed regarding academic performance and trainee readiness during and after residency.
Conclusion
Although excessive duty hours and resident fatigue may have historically contributed to errors in the delivery of patient care, those are certainly not the only concerns. An overall “culture of safety,” which includes pinpointing systematic improvements, identifying potential sources of error, raising performance standards and safety expectations, and implementing multiple layers of protection against medical errors, can continue to augment safety barriers and improve patient care. This can be achieved within a more flexible educational environment that protects resident education and ensures optimal training for the next generation of physicians and surgeons.
ORIGINAL RESEARCH
MB, MMSc,
MD, and
MD, MMSc
Abstract
Background
The first year of graduate medical education is an important period in the professional development of physicians. Disruptive behavior interferes with safe and effective clinical practice.
Objective
To determine the frequency and nature of disruptive behavior perceived by interns and attending physicians in a teaching hospital environment.
Method
All 516 interns at Partners HealthCare (Boston, MA) during the 2010 and 2011 academic years were eligible to complete an anonymous questionnaire. A convenience nonrandom sample of 40 attending physicians also participated.
Results
A total of 394 of 516 eligible interns (76.4%) participated. Attendings and interns each reported that their team members generally behaved professionally (87.5% versus 80.4%, respectively). A significantly greater proportion of attendings than interns felt respected at work (90.0% versus 71.5% respectively; P = .01). Disruptive behavior was experienced by 93% of interns; 54% reported that they experienced it once a month or more. Interns reported disruptive behavior significantly more frequently than attending physicians, including increased reports of condescending behavior (odds ratio [OR], 5.46 for interns compared with attendings; P < .001), exclusion from decision making (OR, 6.97; P < .001), and berating (OR, 4.84; P = .02). Inappropriate jokes, abusive language, and gender bias were also reported, and they were not significantly more frequent among interns than attending physicians. Interns most frequently identified nurses as the source of disruption, and were significantly more likely than faculty to identify nurses as the source of disruptive behavior (OR, 10.40; P < .001). Attendings reported other physicians as the most frequent source of disruption.
Conclusions
Although interns generally feel respected at work, they frequently experience disruptive behavior. Interns described more disruptive behaviors than a convenience sample of attending physicians at our institution.
MD, PhD,
MD, MS,
PharmD,
MD, MS, and
MD, MS
Abstract
Background
Resident duty hour limits, new requirements for supervision, and an enhanced focus on patient safety have shown mixed effects on resident quality of life, patient safety, and resident competency. Few studies have assessed how recent graduates feel these changes have affected their education.
Objective
We assessed recent graduates' perceptions about the effects of duty hour and supervision requirements on their education.
Methods
We conducted a cross-sectional survey of graduates from South Carolina Area Health Education Consortium-affiliated family medicine residency programs from 2005 to 2009 by using logistic regression to determine associations between participant characteristics and survey responses.
Results
Graduates (N = 136) completed the survey with a 51.3% response rate. Nearly all (96%) reported that residency prepared them for their current work hours; 97% reported they felt adequately supervised; 81% worked fewer hours in practice than in residency; 20% believed the limits had restricted their clinical experience; and 3% felt duty hour limits were more important than supervision. Graduates who practiced in a mid-sized communities were more likely to report duty hour limits restricted their clinical experience than individuals practicing in communities of <10 000 (OR = 6.30; 95% CI, 1.38–28.72).
Conclusions
Most graduates who responded to the survey felt supervision was equally or more important than limits on resident duty hours. However, 20% of respondents felt that the duty hour standards limited their education. The duty hour and supervision requirements challenge educators to ensure quality education.
MD, MEd,
MD,
,
, and
DO
Abstract
Background
Patient- and family-centered care (PFCC) approaches to care are important in enhancing the patient-centeredness of the health care experience, yet little is known about the effectiveness of formal approaches for teaching patient-centeredness in residency.
Intervention
We developed and implemented a PFCC curriculum and assessed its impact on residents' self-perceptions of patient-centered behavior and practices.
Methods
We used a quasi-experimental, nonrandomized approach with a pretest-posttest design. An experimental group of 24 interns filled out the Patient Practitioner Orientation Scale (PPOS) before residency, and a control group of 18 graduating residents who had not been exposed to a PFCC curriculum also completed the PPOS. After 2 years of residency and exposure to a PFCC curriculum, interns in the experimental group repeated the PPOS. We compared mean total and subscale PPOS scores.
Results
There was no difference in baseline total or subscale PPOS scores between the experimental and control group. The mean total PPOS score for the experimental group after exposure to the curriculum was 4.55 (P = .45), reflecting no change in patient-centeredness. The 17 female interns in the intervention group were more patient centered (4.8 ± 0.36) than the 6 male interns (4.2 ± 0.38) (P = .005), scoring significantly higher (4.6 ± 0.39 versus 4.0 ± 0.38) in the sharing domain (P = .001).
Conclusion
Interns' exposure to a PFCC curriculum did not result in a change in their perceived patient-centeredness. Most pediatrics residents at our children's hospital perceive themselves as patient and family centered at the start of residency and remain so throughout.
MD, FACEP, FAAEM,
MD, MPH, FACEP,
MD, FACEP, FAAEM,
MD,
MD,
MD,
MD,
MD, and
MD, FACEP, FAAEM
Abstract
Background
Faculty involvement in resident teaching events is beneficial to resident education, yet evidence about the factors that promote faculty attendance at resident didactic conferences is limited.
Objective
To determine whether offering continuing medical education (CME) credits would result in an increase in faculty attendance at weekly emergency medicine conferences and whether faculty would report the availability of CME credit as a motivating factor.
Methods
Our prospective, multi-site, observational study of 5 emergency medicine residency programs collected information on the number of faculty members present at CME and non-CME lectures for 9 months and collected information from faculty on factors influencing decisions to attend resident educational events and from residents on factors influencing their learning experience.
Results
Lectures offering CME credit on average were attended by 5 additional faculty members per hour, compared with conferences that did not offer CME credit (95% confidence interval [CI], 3.9–6.1; P < .001). Faculty reported their desire to “participate in resident education” was the most influential factor prompting them to attend lectures, followed by “explore current trends in emergency medicine” and the lecture's “specific topic.” Faculty also reported that “clinical/administrative duties” and “family responsibilities” negatively affected their ability to attend. Residents reported that the most important positive factor influencing their conference experience was “lectures given by faculty.”
Conclusions
Although faculty reported that CME credit was not an important factor in their decision to attend resident conferences, offering CME credit resulted in significant increases in faculty attendance. Residents reported that “lectures given by faculty” and “faculty attendance” positively affected their learning experience.
MSc,
PhD,
MSc,
PhD, and
PhD
Abstract
Background
There is a need for valid methods to assess the readiness for clinical practice of recently graduated physicians. To develop these methods, it is relevant to know the general features of trainees' performance that facilitate supervisors' trust in their ability to perform critical clinical tasks.
Objective
To discover such essential facets of competence (FOCs), based on the opinion of experienced physician educators.
Methods
We conducted a Delphi study, consisting of 2 rounds, among 18 experienced physician educators in the Netherlands. Mean, standard deviation, level of agreement, and skewness were calculated for the importance of FOCs for making entrustment decisions. The study yielded a list of 25 FOCs.
Results
In the first round, means were between 6.50 and 7.00 on a 7-point Likert scale (SD, 0.42–2.18); in the second round, means ranged from 5.45 to 6.90 (SD, 0.3–2.02). The level of agreement was high for 92% of the FOCs in the first round and 100% of the FOCs in the second round.
Conclusions
Our Delphi study found consensus among experts about FOCs that are important for clinical entrustment decisions.
MD,
MD, MPH,
MD,
MD,
PhD,
MD,
MD, MBA,
MD, and
EdD
Abstract
Background
Entrustable professional activities (EPAs) can form the foundation of competency-based assessment in medical training, focused on performance of discipline-specific core clinical activities.
Objective
To identify EPAs for the Internal Medicine (IM) Educational Milestones to operationalize competency-based assessment of residents using EPAs.
Methods
We used a modified Delphi approach to conduct a 2-step cross-sectional survey of IM educators at a 3-hospital IM residency program; residents also completed a survey. Participants rated the importance and appropriate year of training to reach competence for 30 proposed IM EPAs. Content validity indices identified essential EPAs. We conducted independent sample t tests to determine IM educator-resident agreement and calculated effect sizes. Finally, we determined the effect of different physician roles on ratings.
Results
Thirty-six IM educators participated; 22 completed both surveys. Twelve residents participated. Seventeen EPAs had a content validity index of 100%; 10 additional EPAs exceeded 80%. Educators and residents rated the importance of 27 of 30 EPAs similarly. Residents felt that 10 EPAs could be met at least 1 year earlier than educators had specified.
Conclusions
Internal medicine educators had a stable opinion of EPAs developed through this study, and residents generally agreed. Using this approach, programs could identify EPAs for resident evaluation, building on the initial list created via our study.
MD, MS and
MD, MHPE
Abstract
Background
The 6 competencies defined by the Accreditation Council for Graduate Medical Education provide the framework of assessment for trainees in the US graduate medical education system, but few studies have investigated their impact on remediation.
Methods
We obtained data via an anonymous online survey of pediatrics residency program directors. For the purposes of the survey, remediation was defined as “any form of additional training, supervision, or assistance above that required for a typical resident.” Respondents were asked to quantify 3 groups of residents: (1) residents requiring remediation; (2) residents whose training was extended for remediation purposes; and (3) residents whose training was terminated owing to issues related to remediation. For each group, the proportion of residents with deficiencies in each of the 6 competencies was calculated.
Results
In all 3 groups, deficiencies in medical knowledge and patient care were most common; deficiencies in professionalism and communication were moderately common; and deficiencies in systems-based practice and practice-based learning and improvement were least common. Residents whose training was terminated were more likely to have deficiencies in multiple competencies.
Conclusion
Although medical knowledge and patient care are reported most frequently, deficiencies in any of the 6 competencies can lead to the need for remediation in pediatrics residents. Residents who are terminated are more likely to have deficits in multiple competencies. It will be critical to develop and refine tools to measure achievement in all 6 competencies as the graduate medical education community may be moving further toward individualized training schedules and competency-based, rather than time-based, training.
MD,
PhD,
MD, and
MD
Abstract
Background
Multisource evaluations of residents offer valuable feedback, yet there is little evidence on the best way to collect these data from a range of health care professionals.
Objective
This study evaluated nonphysician staff members' ability to assess internal medicine residents' performance and behavior, and explored whether staff members differed in their perceived ability to participate in resident evaluations.
Methods
We distributed an anonymous survey to nurses, medical assistants, and administrative staff at 6 internal medicine residency continuity clinics. Differences between nurses and other staff members' perceived ability to evaluate resident behavior were examined using independent t tests.
Results
The survey response rate was 82% (61 of 74). A total of 55 respondents (90%) reported that it was important for them to evaluate residents. Participants reported being able to evaluate professional behaviors very well (62% [36 of 58] on the domain of respect to staff; 61% [36 of 59] on attire; and 54% [32 of 59] on communication). Individuals without a clinical background reported being uncomfortable evaluating medical knowledge (60%; 24 of 40) and judgment (55%; 22 of 40), whereas nurses reported being more comfortable evaluating these competencies. Respondents reported that the biggest barrier to evaluation was limited contact (86%; 48 of 56), and a significant amount of feedback was given verbally rather than on written evaluations.
Conclusions
Nonphysician staff members agree it is important to evaluate residents, and they are most comfortable providing feedback on professional behaviors. A significant amount of feedback is provided verbally but not necessarily captured in a formal written evaluation process.
MD, MPP,
MS,
PhD, and
MD
Abstract
Background
Endotracheal intubation (ETI) is an essential skill that emergency medicine residents learn throughout their training.
Objective
To evaluate the effect of implementing a postgraduate year (PGY)-1 anesthesiology rotation on ETI success in the emergency department during PGY-2.
Methods
Residents in the study group completed a 4-week PGY-1 anesthesiology rotation. During the first 6 months of PGY-2, we compared ETI performance of the study group with a control group who did not experience a PGY-1 anesthesiology rotation. Data recorded included date, level of training, first- and second-attempt success, rescue devices used, major adverse events, and intubation scenario. A Pearson χ2 test was used to compare first-attempt success, overall success (≤2 attempts), and adverse events rates between the 2 groups.
Results
Overall success rate for the study groups was 95.7% (111 of 116), compared with 94.5% (137 of 145) for the controls (P = 66). First-attempt success for the study group was 78.4% (91 of 116), compared with 83.4% (121 of 145) for the control group; this was not statistically significant (P = .30). Observed major and minor adverse events were similar: 19.0% for the study group (22 of 116) versus 24.8% (36 of 145) for the control group (P = .26).
Conclusions
The addition of an anesthesiology rotation to the PGY-1 curriculum did not have a significant effect on ETI success or the rate of adverse events during the first 6 months of PGY-2. First-attempt overall success and adverse events of our PGY-2 study group were consistent with previously published studies.
MPH,
MS,
MD,
MD, and
MD
Abstract
Background
The American Board of Internal Medicine approved the use of Practice Improvement Modules (PIMs) to help training programs teach and assess practice-based learning and improvement (PBLI) and systems-based practice (SBP).
Methods
We surveyed individuals who ordered a PIM in a residency or fellowship training program between June 2006 and August 2009. The 43 programs that volunteered to participate completed a 30-minute anonymous online survey.
Results
Program directors or associate program directors led the PIM process in 30 programs (70%). Trainees' degrees of involvement in PIMs were highly variable between programs, and several respondents felt that trainees were either not sufficiently engaged or not engaged with enough consistency. The most common activity for trainee involvement was data collection through patient surveys or chart review, although only 17 programs (40%) provided protected time for this activity. Few trainees participated in higher level activities such as data analysis or identification for areas of improvement or were given leadership roles; yet most respondents reported that completing the PIM helped trainees learn basic principles of QI and develop competence in PBLI and SBP and that PIM completion improved the program's ability to develop QI initiatives and resulted in program or institutional improvements, including sustainable improvement in patient care. Most respondents reported that the outcome warranted the effort to complete PIMs.
Conclusions
PIMs may be a valuable but underused educational experience for trainees as well as training programs. Focusing on particular themes and facets of PIMs may facilitate implementation.
MD,
MD,
MD,
MD, MPH,
MD,
BSN, MS,
MD, MPH, and
MD
Abstract
Background
Many academic hospitals have incorporated family-centered rounds, yet little is known about pediatrics residents' perspectives on the educational impact of these rounds.
Objective
To identify pediatrics residents' knowledge, attitudes, and beliefs about family-centered rounds, including perceived benefits and barriers.
Methods
We conducted focus groups of residents exposed to family-centered rounds at 2 university-affiliated, freestanding children's hospitals. Focus group data were analyzed using grounded theory.
Results
A total of 24 residents participated in 4 focus groups. Residents reported that family-centered rounds enhance education by increasing patient encounters and improving physical exam skills, direct observation, real-time feedback, and attending role modeling; improve parent satisfaction, interpersonal and communication skills, and safety; and reduce length of stay. Physical constraints (large teams and small rooms), lack of uniform approaches to family-centered rounds, variable attending teaching styles, and specific conditions (child abuse, patients on isolation) were cited barriers.
Conclusions
Pediatrics residents report that well-conducted family-centered rounds improve their education and the quality of patient care, including parent satisfaction, communication with families, and patients' length of stay. Standardizing family-centered rounds and reducing attending variability in teaching style might further enhance residents' educational experiences.
MD,
MD,
MD, and
MD, MHA
Abstract
Background
The Accreditation Council for Graduate Medical Education (ACGME) recommends that residents gain broad procedural competence in pediatrics training. There is little recent information regarding practice patterns after graduation.
Objective
We analyzed reported procedures performed in actual practice by graduates of a general pediatrics residency program.
Methods
We conducted an online survey from April 2007 to April 2011 of graduates of a single pediatrics program from a large children's hospital. Eligible participants completed general pediatrics residency training between 1992 and 2006. Graduates were asked about the adequacy of their training for each procedure, as well as the frequency of commonly performed procedures in their practice. As the primary analysis, procedures were divided into emergent and urgent procedures.
Results
Our response rate was 54% (209 of 387). General pediatricians rarely performed emergent procedures, such as endotracheal intubation, intraosseous line placement, thoracostomy, and thoracentesis. Instead, they more commonly performed urgent procedures, such as laceration repair, fracture or dislocation care, bladder catheterization, foreign body removal, and incision and drainage of simple abscesses. Statistically significant differences existed between emergent and urgent procedures (P < .001).
Conclusions
In a single, large, urban, pediatrics residency, 15 years of graduates who practiced general pediatrics after graduation reported they rarely performed emergent procedures, such as endotracheal intubation, but more often performed urgent procedures, such as laceration repair. These results may have implications for ACGME recommendations regarding the amount and type of procedural training required for general pediatrics residents.
MD,
MD, and
PhD, MPH, PStat
Abstract
Background
End-of-residency outpatient handoffs affect at least 1 million patients per year, yet there is no consensus on best practices.
Objective
To explore the use of formal systems for end-of-residency clinic handoffs in internal medicine–pediatrics residency (Med-Peds) programs, and their associated categorical internal medicine and pediatrics programs.
Methods
We surveyed Med-Peds program directors about their programs' system for handing off ambulatory continuity patients.
Results
Our response rate was 85% (67 of 79 programs). Thirty-one programs (46%) reported having a system for end-of-residency handoffs. Of the 30 that offered detailed information, 22 (73%) formally introduced the program to residents, 12 (40%) standardized the handoff, and 14 (47%) used multiple methods for information exchange, with the electronic health record and oral transfer of information (15 of 30, 50%) the most common. Six programs (20%) indicated they did not offer residents protected time to complete end-of-residency handoffs, and 13 programs (43%) did not identify a specific postgraduate year level for residents to whom patients were handed off. Programs were more likely to have a system for end-of-residency handoffs if another categorical program at their institution also had one (P < .001).
Conclusions
Fewer than half of responding Med-Peds programs have outpatient handoff systems in place. Inclusion of end-of-residency handoff information in the electronic health record may represent a best practice that has the potential of enhancing continuity and safety of care for patients in resident continuity clinics.
EDUCATIONAL INNOVATION
PhD,
MD, MBA,
MD, MA,
MD,
MD,
MD, PhD,
MD, and
MD
Abstract
Background
The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project introduced 6 general competencies relevant to medical practice but fell short of its goal to create a robust assessment system that would allow program accreditation based on outcomes. In response, the ACGME, the specialty boards, and other stakeholders collaborated to develop educational milestones, observable steps in residents' professional development that describe progress from entry to graduation and beyond.
Objectives
We summarize the development of the milestones, focusing on 7 specialties, moving to the next accreditation system in July 2013, and offer evidence of their validity.
Methods
Specialty workgroups with broad representation used a 5-level developmental framework and incorporated information from literature reviews, specialty curricula, dialogue with constituents, and pilot testing.
Results
The workgroups produced richly diverse sets of milestones that reflect the community's consideration of attributes of competence relevant to practice in the given specialty. Both their development process and the milestones themselves establish a validity argument, when contemporary views of validity for complex performance assessment are used.
Conclusions
Initial evidence for validity emerges from the development processes and the resulting milestones. Further advancing a validity argument will require research on the use of milestone data in resident assessment and program accreditation.
MD,
MD,
BS,
MD, MPH,
MD, MBA, and
MD
Abstract
Background
Medical error is a major cause of preventable morbidity and mortality. Resident fatigue is likely to be a significant contributor.
Objectives
We calculated and compared predicted fatigue impairment in surgical residents on varying schedules by using the validated Sleep, Activity, Fatigue, and Task Effectiveness model and Fatigue Avoidance Scheduling Tool; we identified specific times of day and rotations during which residents were most affected, instituted countermeasures, and measured the predicted response.
Methods
We compared 4 scheduling patterns: day shift, trauma shift, night shift, and prework hour restriction Q3 call (or every-third-night call). The dependent variables were mean daily effectiveness while at work and the percentage of time residents worked with critical fatigue impairment (defined as an effectiveness score of less than 70 correlated with an increased risk for error and a blood alcohol content of 0.08). Fatigue countermeasures (ie, a 30-minute nap, eliminating 24-hour shifts) were applied to rotations with significant impairment to determine impairment plasticity.
Results
Calculated mean effectiveness scores and percentage of time spent impaired at work were as follows: day shift, 90.3, 0%; trauma shift, 82.0, 7.5%; prework hour restriction Q3 call shift, 80.7, 23%; and night shift, 68.0, 50% (P < .001). Fatigue optimization countermeasures for night shift rotation improved mean daily effectiveness to 87.1 with only 1.9% of time working while impaired (P < .001).
Conclusions
There is a significant potential for fatigue impairment in residents, with work schedule a significant factor. Once targeted, fatigue impairment may be minimized with specific countermeasures. Fatigue optimization tools provide data for targeted scheduling interventions, which reduce fatigue and may mitigate medical error.
PharmD, MMedEd,
MD,
MD,
MD,
MD, MPH, and
MD
Abstract
Background
The Accreditation Council for Graduate Medical Education Outcome Project intended to move residency education toward assessing and documenting resident competence in 6 dimensions of performance important to the practice of medicine. Although the project defined a set of general attributes of a good physician, it did not define the actual activities that a competent physician performs in practice in the given specialty. These descriptions have been called entrustable professional activities (EPAs).
Objective
We sought to develop a list of EPAs for ambulatory practice in family medicine to guide curriculum development and resident assessment.
Methods
We developed an initial list of EPAs over the course of 3 years, and we refined it further by obtaining the opinion of experts using a Delphi Process. The experts participating in this study were recruited from 2 groups of family medicine leaders: organizers and participants in the Preparing the Personal Physician for Practice initiative, and members of the Society of Teachers of Family Medicine Task Force on Competency Assessment. The experts participated in 2 rounds of anonymous, Internet-based surveys.
Results
A total of 22 experts participated, and 21 experts participated in both rounds of the Delphi Process. The Delphi Process reduced the number of competency areas from 91 to 76 areas, with 3 additional competency areas added in round 1.
Conclusions
This list of EPAs developed through our Delphi process can be used as a starting point for family medicine residency programs interested in moving toward a competency-based approach to resident education and assessment.
MD,
MSW, PhD, and
MD
Abstract
Background
Future physicians may not be prepared for the challenges of caring for the growing population of poor patients in this country. Given the potential for a socioeconomic “gulf” between physicians and patients and the lack of curricula that address the specific needs of poor patients, resident knowledge about caring for this underserved population is low.
Intervention
We created a 2-day Resident Academy orientation, before the start of residency training, to improve community knowledge and address resident attitudes toward poor patients through team-based experiential activities. We collected demographic and satisfaction data through anonymous presurvey and postsurvey t tests, and descriptive analysis of the quantitative data were conducted. Qualitative comments from open-ended questions were reviewed, coded, and divided into themes. We also offer information on the cost and replicability of the Academy.
Results
Residents rated most components of the Academy as “very good” or “excellent.” Satisfaction scores were higher among residents in primary care training programs than among residents in nonprimary care programs for most Academy elements. Qualitative data demonstrated an overall positive effect on resident knowledge and attitudes about community resource availability for underserved patients, and the challenges of poor patients to access high-quality health care.
Conclusions
The Resident Academy orientation improved knowledge and attitudes of new residents before the start of residency, and residents were satisfied with the experience. The commitment of institutional leaders is essential for success.
MD, MS (BCHM), MS (Epidemiology),
BS,
BS,
MD, PhD,
MD, MBA, and
MD
Abstract
Background
The transition from internship to residency training may be a stressful time for interns, particularly if it involves a change among programs or institutions after completing a preliminary year.
Objective
We explored whether an e-learning curriculum would increase interns' preparedness for the transition to the first year of clinical anesthesiology training and reduce stress by improving confidence and perceived competence in performing professional responsibilities.
Intervention
We tested a 10-month e-learning program, Successful Transition to Anesthesia Residency Training (START), as a longitudinal intervention to increase interns' self-perceived preparedness to begin anesthesiology residency training in a prospective, observational study and assessed acceptance and sustainability. After a needs assessment, we administered the START modules to 22 interns, once a month, using an integrated learning management and lecture-capture system. We surveyed interns' self-assessed preparedness to begin anesthesiology residency before and after completing the START modules. Interns from the prior year's class, who did not participate in the online curriculum, served as controls.
Results
After participation in the START intervention, self-assessed preparedness to begin residency improved by 72% (P = .02). Interns also felt more connected to, and had improved positive feelings toward, their new residency program and institution.
Conclusion
Participation in our novel 10-month e-learning curriculum and virtual mentorship program improved interns' impression of their residency program and significantly increased interns' subjective assessment of their preparedness to begin anesthesiology residency. This e-learning concept could be more broadly applied and useful to other residency programs.
MD, PhD,
MD,
MD,
MD, MPH,
MD,
MD,
MD,
MD,
MD,
MD,
MD,
MD, and
MD
Abstract
Background
The internal medicine milestones were developed to advance outcomes-based residency training and will play an important role in the next accreditation system.
Innovation
As an element of our program's participation in the internal medicine educational innovations project, we implemented a milestones-based evaluation process in our general medicine and pulmonary-critical care rotations on July 1, 2010.
Measures
Outcomes assessed included survey-rated acceptability to participating faculty, residents, and clinical competency committee members.
Results
Faculty and residents agreed that the milestones promoted a common understanding of what knowledge, skills, and attitudes should be displayed at particular points in residents' professional development and enhanced evaluators' ability to provide specific performance feedback. Most residents and faculty members agreed that the milestones promoted fairness and uniformity in the evaluation process. Clinical competency committee members agreed the milestones improved the quality of information available for deliberations and resulted in more uniform promotion standards. Faculty rated the use of too many milestones per form/tool at a mean of 7.3 (where 1 was minimally problematic, and 10 was maximally problematic) and the potential for evaluator fatigue (mean, 8.2) as the most significant challenges to the use of milestones. Eight of 12 faculty members would recommend milestones in other programs; 4 were uncertain.
Conclusions
Despite logistical challenges, educators and trainees found that milestones promoted a common understanding of what knowledge, skills and attitudes should be displayed at particular stages of training; permitted greater specificity in performance feedback; and enhanced uniformity and fairness in promotion decisions.
BRIEF REPORT
MD,
EdD, and
MD
Abstract
Background
Excessive, sometimes unnecessary consultation challenges the ophthalmology resident's ability to obtain adequate rest and personal time while taking at-home calls. Basic training of nonophthalmologists in assessing and treating common ophthalmic emergencies offers a potential solution to this problem by providing primary caregivers the knowledge base, tools, and confidence to manage them independently.
Objective
We measured ophthalmic consultation requests at a large teaching institution and the impact of an educational intervention on the number and type of requests.
Methods
During a 31-day period in 2009 all requests for urgent ophthalmic consultation were logged and characterized by reason, urgency, and source of the call. A 3-part educational intervention was targeted to residents and attending physicians in the departments of emergency medicine (half-day lecture and practice), anesthesiology (grand rounds lecture), and otolaryngology (guidelines for traumatic orbital fractures). Six months later we conducted a 26-day call log to evaluate the impact.
Results
Pre intervention, we received 63 total and 56 after-hours calls for urgent ophthalmic consultation as compared to 39 total and 35 after-hours calls post intervention. Ophthalmology residents reported seeing more urgent and fewer nonessential consultations in the postintervention period, and there was greater agreement between calling physicians' initial diagnosis and consulting physicians' final diagnosis. No adverse patient outcomes occurred as a result of the change in practices.
Conclusion
Basic education of nonophthalmologists in ophthalmic diagnostic and treatment concepts, through relatively brief educational interventions, was associated with a decrease in total consultation requests and more appropriate consultations in this pilot study at a single institution.
MD,
MD,
BS,
MD,
MD,
MD,
MHA,
MD, and
MD, PhD
Abstract
Background
Increased documentation and charting requirements are challenging for residents, given duty hour limits. Use of mobile electronic devices may help residents complete these tasks efficiently.
Objective
To collect initial data on usage rates, information technology (IT) support requirements, and resident use of iPads during training.
Methods
In this pilot study, we provided 12 residents/fellows from various specialties at the University of Virginia with an iPad with IT support. The system used a virtual private network with access to the institution's electronic health record. Participants were allowed to develop their own methods and systems for personalized iPad use, and after 9 months they provided data on the utility of the iPad. Feedback from the IT team also was obtained.
Results
Average iPad use was 2.1 h/d (range, 0.5–6 h/d). The average self-reported reduction in administrative work due to the iPad was 2.7 h/wk (range, 0–9 h/wk). A total of 75% (9 of 12) of the users would recommend universal adoption among residents and fellows. More than 90% (11 of 12) of users reported the iPad would improve communication for coordination of care. A total of 68% (8 of 12) of users reported that an iPad facilitated their activities as educators of medical students and junior residents. Residents cited slow data entry into the electronic health record and hospital areas lacking Wi-Fi connectivity as potential drawbacks to iPad use. The IT team reported minimal support time for device setup, maintenance, and upgrades, and limited security risks.
Conclusions
The iPad may contribute to increased clinical efficiency, reduced hours spent on administrative tasks, and enhanced educational opportunities for residents, with minimal IT support.
MD, FACP,
DO, and
MD, MPH
Abstract
Background
Despite the rising prevalence of diabetes, there is a paucity of diabetes curricula in residency training. The multidisciplinary diabetes team approach is underused in residency education.
Objective
To assess the feasibility of an innovative multidisciplinary resident diabetes clinic (MRDC) in enhancing (1) resident diabetes knowledge via a Diabetes Awareness Questionnaire, and (2) subsequent process and patient outcomes in patients with diabetes via a Diabetes Practice Behavior Checklist.
Methods
From October 2008 to February 2010, 14 internal medicine residents managed patients with uncontrolled diabetes in a weekly half-day MRDC for 1 month (total 4–5 half-day sessions/resident), with a collaborative team of internists, diabetes educators, an endocrinologist, and a pharmacist. The curriculum included didactic sessions, required readings, and patient-specific case discussions. A 20-question Diabetes Awareness Questionnaire was administered to each resident prerotation and postrotation. Records of 47 patients with diabetes in the residents' own continuity clinics (not the MRDC) were audited 6 months before and after the MRDC for Diabetes Practice Behavior Checklist measures (glycated hemoglobin, blood pressure, low-density lipoprotein cholesterol, retinal referral, foot exam, microalbumin screen). Pre-MRDC and post-MRDC data were compared via paired t test.
Results
The MRDC residents exhibited a modest increase in mean (SD) scores on the Diabetes Awareness Questionnaire (before, 8.2 [2.8]; after, 10.9 [2.8]; P = .02) and a modest mean (SD) performance increase in overall process outcomes from the Diabetes Practice Behavior Checklist (before, 74% [18%]; after, 84% [18%]; P = .004). No improvements occurred in patient outcomes.
Conclusions
Multidisciplinary diabetes teaching may be useful in fostering certain resident knowledge and performance measures but may not alter clinical outcomes. Further large-scale, longitudinal studies are needed to understand the effect of our curriculum on residents' diabetes knowledge and future practice behavior.
MD,
MD,
MD,
PhD, and
MD
Abstract
Background
The Accreditation Council for Graduate Medical Education requires residency programs to ensure safe patient handovers and to document resident competency in handover communication, yet there are few evidence-based curricula teaching resident handover skills.
Objective
We assessed the immediate and sustained impact of a brief educational intervention on pediatrics intern handover skills.
Methods
Interns at a freestanding children's hospital participated in an intervention that included a 1-hour educational workshop on components of high-quality handovers, as well as implementation of a standardized handover format. The format, SAFETIPS, includes patient information, current diagnosis and assessment, patient acuity, a focused plan, a baseline exam, a to-do list, anticipatory guidance, and potential pointers and pitfalls. Important communication behaviors, such as paraphrasing key information, were addressed. Quality of intern handovers was evaluated using a simulated encounter 2 weeks before, 2 weeks after, and 7 months after the workshop. Two trained, blinded, independent observers scored the videotaped encounters.
Results
All 27 interns rotating at the Children's Hospital consented to participate in the study, and 20 attended the workshop. We included all participant data in the analysis, regardless of workshop attendance. Following the intervention, intern reporting of patient acuity improved from 13% to 92% (P < .001), and gains were maintained 7 months later. Rates of key communication behaviors, such as paraphrasing critical information, did not improve.
Conclusions
A brief educational workshop promoting standardized handovers improved the inclusion of essential information during intern handovers, and these improvements were sustained over time. The intervention did not improve key communication behaviors.
RIP OUT
MD, MS,
MD, MS,
MD, MS,
MD, MSc, and
MD, MPH
The Challenge
Busy clinician-educators are charged with developing a national reputation to advance their careers and achieve academic promotion. Fortunately, scholarly work now includes not only peer-reviewed publications but also the development of curriculum materials, assessment tools, and presentation of educational workshops at regional or national meetings.
PhD
The Challenge
The entrustable professional activity (EPA) concept allows faculty to make competency-based decisions on the level of supervision required by trainees. Competency-based education targets standardized levels of proficiency to guarantee that all learners have a sufficient level of proficiency at the completion of training.1–6 Collectively, the competencies (ACGME or CanMEDS) constitute a framework that describes the qualities of professionals. Such a framework provides generalized descriptions to guide learners, their supervisors, and institutions in teaching and assessment. However, these frameworks must translate to the world of medical practice. EPAs were conceived to facilitate this translation, addressing
MD and
MD, MHPE
The Challenge
Institutional policies on resident supervision are guided by the Accreditation Council for Graduate Medical Education (ACGME) and Residency Review Committee Program Requirements. These standards address the availability and quality of clinical supervisors, and highlight the importance of affording learners a graduated level of responsibility throughout training to develop them into competent physicians ready for unsupervised practice.
Although many surgical and procedural-based specialties have long included explicit language addressing clinical supervision, this has not been the case for medical specialties. In their 5-year review of the impact of the initial 2003 duty hour regulations, the Institute of Medicine report
COMMENTARIES
MD, MSc
In this issue Lobo et al1 report the results of a pilot study surveying residents' use of iPads. This study of 12 residents found there was high self-reported use of the devices, with an average of 2.1 hours per day. Using their iPads, residents could access electronic health records along with a range of educational resources. Most residents perceived that iPads should be distributed to all residents and fellows, and that this would improve communication and efficiency.
To put this pilot study in context, there has been a rapid uptake in the use of smartphones and tablets in medical
MD, MPH
Poverty is potentially the greatest predictor of poor health in the United States. Disease-specific and all-cause mortality rates increase in a consistent gradient with decreasing socioeconomic position.1,2 Poverty is associated with a 5-year loss in life expectancy,3 and the loss of quality-adjusted life years for those living with incomes in the bottom one-third of the population is significantly higher than are losses associated with smoking and obesity.4 Although access to health care is one piece of a complex set of mechanisms contributing to health inequity, health care contributes only a small fraction
TO THE EDITOR
MD and
PhD
Arthrocentesis is a safe and effective procedure used by internists for both diagnosis and therapy of musculoskeletal problems. We commend Preisner et al for their recent publication regarding long-term retention of knee and shoulder aspiration and injection skills in internal medicine residents.1 Retention of skills after educational interventions (including simulation) is rarely studied and worthy of further investigation. In their study, postgraduate year-1 residents completed joint aspiration and injection simulation-based training and a posttest. Six to 30 months later, residents were randomly assigned to receive web-based review of the material and skills testing or skills testing alone. Aspiration and
ON TEACHING
MD
I heard the clang of handcuffs jerked against the metal sidebar of a stretcher and gave a knowing look to the officer guarding the door. I entered to meet my next patient, only to be halted by his verbal assault of crude and irate obscenities. I had slept well, and it was early in my day but I immediately felt fatigued. A man in his twenties, Mr Pence was handcuffed to the stretcher and being questioned by a detective. My attention fell to his right leg where a single bullet wound marred an otherwise healthy-looking physique. As I reached out
NEWS AND VIEWS
MD and
PhD
Introduction
As the Accreditation Council for Graduate Medical Education (ACGME) transitions to the Next Accreditation System (NAS), there is value in exploring the elements of the current accreditation process for opportunities for improvement. A key function of the accreditation process, both in the current system and in the NAS, is to assess compliance with the ACGME program and institutional requirements.
In the current accreditation system, the ACGME's review of individual residency education programs is based on the Program Information Form prepared prior to the site visit. In the NAS, the review will be based on data annually submitted to the
The Journal of Graduate Medical Education is deeply grateful to the individuals who served as reviewers and provided critiques and helpful suggestions to authors for the 4 issues published in 2012. Reviewers shown bolded reviewed a significant number of articles, and the Journal greatly appreciates the contribution of their time, energy, and expertise.
Eva Aagaard, MD
D. Clay Ackerly
Meenakshy Aiyer, MD
Ralitsa Akins, MD, PhD
Amer Aldeen, MD
Robert Allen, MD
Ronald Amedee, MD
Douglas Ander, MD
Kathryn Andolsek, MD, MPH
Felix Ankel, MD
Paul Aronowitz, MD
Carol Aschenbrener
Stewart Babbott, MD
Lubna Baig, MBBS, MPH, FCPS,
Rouge on the Lips of Silence
Yuki TeramotoMD, PhD
Evaluating Methodology for Increasing Diversity in US Residency Training Programs: A Scoping Review
Lauren BoeckermannMD, MPH,
Dennis MenjivarBS,
Emily E. HagnMD,
Melissa PreviteraMLIS,
Samuel H. CheshierMD, PhD,
Hilary C. McCraryMD, MPH, and
Jeremiah A. AltMD, PhD
Scoping Review of Simulation-Based Training for Social Determinants of Health Within Residency Programs
Nathan TunellMS,
Sehr KhanMD,
Ashley PavlicMD, MA, and
Taylor SonnenbergMD, MSGH
Career Outcomes Among Graduates of 2 Urban Health Primary Care Training Programs
Preetham BachinaBS,
Ariel G. VilidnitskyMD,
Katherine ShawMD,
Monica MixMD, MPH,
Talia Robledo-GilMD, and
Leonard FeldmanMD, FACP, FAAP, MHM
Trends in MedEdPORTAL Faculty Development Resources for Clinician Educators
Amna AneesMD, FACP,
Craig NoronhaMD,
Aashish DidwaniaMD, and
Karen FriedmanMD, MS
The Effect of Paging Reminders on Fellowship Conference Attendance: A Multi-Program Randomized Crossover Study
Joshua SmithMD,
Lorenzo ZaffiriMD,
Julie ClaryMD, MBA,
Tyler DavisMD, and
Gabriel T. BossletMD, MA
A Values Affirmation Intervention to Improve Female Residents' Surgical Performance
Arghavan SallesMD, PhD,
Claudia M. MuellerPhD, MD, and
Geoffrey L. CohenPhD
Improving Residents' Safe Opioid Prescribing for Chronic Pain Using an Objective Structured Clinical Examination
Daniel P. AlfordMD, MPH,
Brittany L. CarneyMS,
Belle Brett EDD,
Sharon J. ParishMD, and
Angela H. JacksonMD
Integrating a Resident-Driven Longitudinal Quality Improvement Curriculum Within an Ambulatory Block Schedule
Aleksey TentlerMD,
Mirela FeurdeanMD,
Steven KellerPhD, and
Neil KothariMD
Skills for Interviewing Adolescent Patients: Sustainability of Structured Feedback in Undergraduate Education on Performance in Residency
Nadim JoukhadarBSc,
Genna BourgetMD,
Sarah ManosMD,
Karen MannPhD,
Jill HatchettePhD, and
Kim BlakeMD
eISSN: 1949-8357
ISSN: 1949-8349
Powered by PubFactory