Utilization of an Educational Prescription to Improve Performance on the Internal Medicine In-Training Examination

Setting and Problem

With a 3-year rolling average American Board of Internal Medicine Certifying Examination (ABIM-CE) pass rate of 80%, the University of Kansas Internal Medicine Residency Training Program set out to improve its board pass rate. The intervention involved targeting residents deemed at risk for ABIM-CE failure based on performance on the Internal Medicine In-Training Examination (IM-ITE). The IM-ITE is a test administered annually by the American College of Physicians. More than 20000 internal medicine residents in more than 10 countries take the IM-ITE. Postgraduate year (PGY)–based comparative data on test performance are provided to residency program directors to assist programs and individual residents to recognize gaps in medical knowledge. Additionally, with individual score reports, educational objectives are released so that the individual resident can target specific missed concepts in preparation for the ABIM-CE.

Intervention

In addition to a robust and deliberate board preparation plan for all residents, our program developed a defined educational prescription for PGY-2 residents who scored below the 30th percentile compared with peers on the IM-ITE. This threshold was chosen because of data correlating IM-ITE performance with ABIM-CE performance. PGY-2 residents with deficiencies in medical knowledge based on IM-ITE performance were placed on an educational prescription, which entailed the following requirements:

1. Completion of a learning style assessment.

2. Meeting with a learning specialist.

3. Meeting with a faculty member to review an evidence-based approach to the remediation of medical knowledge.

4. Expectations for weekly completion of IM-ITE educational objectives (minimum of 5) and Medical Knowledge Self-Assessment Program questions (minimum of 20).

5. At minimum, a monthly check-in with a program director advisor to track compliance and progress.

Outcomes to Date

In academic year 2013–2014, 6 PGY-2 residents were placed on an educational prescription because of suboptimal performance on the IM-ITE. Residents were notified of the educational prescription requirements by formal letter from the program leadership team. Residents were informed that noncompliance with the educational prescription could result in serious repercussions, including notation of deficiency in the competency of medical knowledge on the milestone report shared with the Accreditation Council for Graduate Medical Education. A timeline for completion of required meetings was established. Regular communication with the resident's program director advisor was established to track progress.

All 6 residents on an educational prescription improved their percentile rank compared with their peers on the IM-ITE. Five residents improved their percentile rank to > 30%, resulting in them moving off the educational prescription plan (table).

TABLE Change in Individual Resident Internal Medicine In-Training Examination Performance With Implementation of an Educational Prescription

View Fullscreen

Despite some initial resistance when notified of their placement on an educational prescription, the residents subsequently embraced the expectation to improve their medical knowledge. Postimplementation feedback from the residents highlighted the importance of the prescription's structured accountability as the most valuable aspect of the intervention.

In light of the outstanding improvement by each of these residents compared with their peers, we plan to continue to apply this educational prescription for each PGY-2 who has performed below the 30th percentile rank compared with his or her peers. Additionally, because of our outlined success, we plan to expand the educational prescription to include PGY-3 residents for this upcoming academic year. We will track ABIM-CE outcomes related to these residents to ensure that our ultimate goal of success on the ABIM-CE is achieved with implementation of this educational prescription.