ABSTRACT

Background

Resident handoff communication skills are essential components of medical education training. There are no previous systematic reviews of feedback and evaluation tools for physician handoffs.

Objective

We performed a systematic review of articles focused on inpatient handoff feedback or assessment tools.

Methods

The authors conducted a systematic review of English-language literature published from January 1, 2008, to May 13, 2015 on handoff feedback or assessment tools used in undergraduate or graduate medical education. All articles were reviewed by 2 independent abstractors. Included articles were assessed using a quality scoring system.

Results

A total of 26 articles with 32 tools met inclusion criteria, including 3 focused on feedback, 8 on assessment, and 15 on both feedback and assessment. All tools were used in an inpatient setting. Feedback and/or assessment improved the content or organization measures of handoff, while process and professionalism measures were less reliably improved. The Handoff Clinical Evaluation Exercise or a similar tool was used most frequently. Of included studies, 23% (6 of 26) were validity evidence studies, and 31% (8 of 26) of articles included a tool with behavioral anchors. A total of 35% (9 of 26) of studies used simulation or standardized patient encounters.

Conclusions

A number of feedback and assessment tools for physician handoffs in several specialties have been studied. Limited research has been done on the studied tools. These tools may assist medical educators in assessing trainees' handoff skills.

ABSTRACT

Background

Prior reviews of geriatrics curricula for internal medicine (IM) and family medicine (FM) residents have not evaluated study quality or assessed learning objectives or specific IM or FM competencies.

Objective

This review of geriatrics curricula for IM and FM residents seeks to answer 3 questions: (1) What types of learning outcomes were measured? (2) How were learning outcomes measured? and (3) What was the quality of the studies?

Methods

We evaluated geriatrics curricula that reported learning objectives or competencies, teaching methods, and learning outcomes, and those that used a comparative design. We searched PubMed and 4 other data sets from 2003–2015, and assessed learning outcomes, outcome measures, and the quality of studies using the Medical Education Research Study Quality Instrument (MERSQI) and Best Evidence Medical Education (BEME) methods.

Results

Fourteen studies met inclusion criteria. Most curricula were intended for IM residents in the inpatient setting; only 1 was solely dedicated to FM residents. Median duration was 1 month, and minimum geriatrics competencies covered were 4. Learning outcomes ranged from Kirkpatrick levels 1 to 3. Studies that reported effect size showed a considerable impact on attitudes and knowledge, mainly via pretests and posttests. The mean MERSQI score was 10.5 (range, 8.5–13) on a scale of 5 (lowest quality) to 18 (highest quality).

Conclusions

Few geriatrics curricula for IM and FM residents that included learning outcome assessments were published recently. Overall, changes in attitudes and knowledge were sizeable, but reporting was limited to low to moderate Kirkpatrick levels. Study quality was moderate.

ABSTRACT

Background

The Family Medicine (FM) Milestones are a framework designed to assess development of residents in key dimensions of physician competency. Residency programs use the milestones in semiannual reviews of resident performance from entry toward graduation.

Objective

To examine the functioning and reliability of the FM Milestones and to determine whether they measure the amount of a latent trait (eg, knowledge or ability) possessed by a resident or simply indicate where a resident falls along the training sequence.

Methods

This study utilized the Rasch Partial Credit model to examine academic year 2014–2015 ratings for 10 563 residents from 476 residency programs (postgraduate year [PGY] 1 = 3639; PGY-2 = 3562; PGY-3 = 3351; PGY-4 = 11).

Results

Reliability was exceptionally high at 0.99. Mean scores were 3.2 (SD = 1.3) for PGY-1; 5.0 (SD = 1.3) for PGY-2; 6.7 (SD = 1.2) for PGY-3; and 7.4 (SD = 1.0) for PGY-4. Keyform analysis showed a rating on 1 item was likely to be similar for all other items.

Conclusions

Our findings suggest that FM Milestones seem to largely function as intended. Lack of spread in item difficulty and lack of variation in category probabilities show that FM Milestones do not measure the amount of a latent trait possessed by a resident, but rather describe where a resident falls along the training sequence. High reliability indicates residents are being rated in a stable manner as they progress through residency, and individual residents deviating from this rating structure warrant consideration by program leaders.

The Journal of Graduate Medical Education is deeply grateful to the individuals who served as reviewers and provided critiques and helpful suggestions to authors for the 5 issues published in 2016. Reviewers shown bolded reviewed a significant number of articles, and the Journal greatly appreciates the contribution of their time, energy, and expertise.

Eva Aagaard, MD

Christopher Aakre, MD

Karen Adams, MD

Anoop Agrawal, MD

Meenakshy Aiyer, MD

Ralitsa Akins, MD, PhD

Elizabeth Allen, MD

Suzanne Allen, MD, MPH

Ronald Amedee, MD

Gowri Anandarajah, MD

Douglas Ander, MD

Dorothy Andriole, MD

Tanya Anim, MD

Louise Arnold,