The Dermatology Milestone Project

Introduction

The educational Milestones represent a developmental framework from less to more advanced for each of the 6 Accreditation Council for Graduate Medical Education (ACGME) competencies. They are outcome-based descriptors and targets for resident performance as the resident moves from entry into residency through graduation. In the Next Accreditation System (NAS), dermatology resident achievement of the educational Milestones also will be 1 of several indicators of the educational effectiveness of dermatology residency programs, and the Dermatology Residency Review Committee (RRC) will examine Milestone performance data to determine whether residents overall are progressing.

Milestone Development History

The Dermatology Milestones represent a joint initiative of the ACGME and the American Board of Dermatology (ABD). The development of the Milestones was led by the Dermatology Milestone Working Group (DMWG), whose volunteer members represented key stakeholder organizations, including the Association of Professors of Dermatology (APD), the ABD, and the Dermatology RRC. The members of the DMWG are listed in box 1. Milestone Working Group members were selected by organization, subspecialty, age, sex, and geographic diversity. Members represented medical dermatology, pediatric dermatology, procedural dermatology, and dermatopathology, and included current and past program directors and department chairs. An advisory board to the working group allowed for broader input from stakeholders. The members of the advisory group are listed in box 2.

The first draft of the Dermatology Milestones was developed at the APD Annual Meeting in the fall of 2010. The cochair of the Dermatology Milestone Working Group, Daniel Loo, MD, chaired the APD Milestone Ad Hoc Committee to create Milestones for each of the 6 competencies. This committee included Dr Loo (patient care); Amy Chang, MD (patient care: procedural dermatology); Joslyn Kirby, MD (medical knowledge); Misha Rosenbach, MD (interpersonal and communication skills); Erik Stratman, MD (practice-based learning and improvement); Roy Colven, MD (systems-based practice); and Amit Garg, MD (professionalism). Feedback was obtained from the APD membership through focus sessions on the individual competencies, plenary sessions for general comments, and postmeeting surveys. Additional work after the meeting resulted in a Milestone draft the DMWG used to begin its work.

In November 2010, representatives from the RRC and the ABD attended a meeting in Chicago at which the ACGME introduced Milestones to all RRCs and certification boards, and reported on Milestone development in progress. The DMWG was asked to aim for 30 to 40 subcompetency areas for the Milestones, distributed roughly equally across the 6 competencies. The working group conducted its work via face-to-face meetings in October 2011, and May and September 2012; individual and small-group work; conference calls; and e-mail. The draft Milestones underwent numerous revisions after receiving feedback from the stakeholder community through open forums and surveys.

At its October 2011 meeting, the working group focused on the development of draft Milestones for patient care (PC) and medical knowledge (MK), using work from the APD Milestone Ad Hoc Committee as the initial resource. The working group developed PC and MK subcompetencies that corresponded to the major subareas of dermatology as defined in the RRC's program requirements. Four members of the working group volunteered to produce working Milestone drafts for professionalism, interpersonal and communication skills, practice-based learning and improvement, and systems-based practice. Resources used in Milestone development included the work of the ACGME Expert Panel on Milestones draft for professionalism, interpersonal and communication skills, practice-based learning and improvement, and systems-based practice; draft Milestones from other specialties (internal medicine, pediatrics, ophthalmology, obstetrics and gynecology, general surgery, urology); an ACGME sample Milestone report format; and ACGME International Milestones used in Singapore. In March 2012, the working group solicited input on the PC and MK draft Milestones from the advisory board and made subsequent revisions.

The DMWG's meeting in May 2012 focused on refining the PC and MK Milestones and reviewing, discussing, and editing the Milestone drafts for the other 4 competencies. The working group refined the definitions of the Milestone levels, identifying each level as beginning resident (0 to 3 months), junior resident (4 to 18 months), senior resident (19 to 36 months), graduating resident, and practitioner. The working group ultimately decided to use levels 1 through 5 instead, and to omit identifying labels and time ranges for the 5 levels to allow more flexibility for programs. The 5-level framework was derived from the Dreyfus model of acquisition of expertise. The Bloom taxonomy was used to assist the working group with verb choices for cognitive levels in the revision of the Milestones. The working group divided into 6 teams of 2 to further refine the Milestone drafts for each of the 6 competencies.

Before the September 2012 APD meeting, a survey was sent to the APD membership asking for feedback on the Dermatology Milestone draft, and at the meeting the DMWG chair and cochair provided updates on Dermatology Milestone development, and the DMWG hosted an interactive session to obtain feedback from APD members. Nicole Owens, MD, chair of the Dermatology RRC, fielded questions related to APD member concerns regarding the Milestones, including the anticipated use of the Milestones from the RRC perspective. Immediately after the meeting, the working group reviewed and discussed APD survey results and the feedback obtained at the meeting and used it to produce a revised version of the Milestone draft.

In March 2013, the draft Milestones underwent alpha testing in 1 volunteer dermatology program and detailed feedback was received. This feedback showed evidence that higher Milestone performance levels correlated with more senior residents with stratification of postgraduate year groups. In April 2013, the DMWG received and incorporated detailed feedback from the ABD on the professionalism Milestones. Four programs volunteered for beta testing of the draft Milestones in the summer of 2013 and completed a questionnaire that included free-text comments. In September 2013, feedback from alpha and beta testing and input from the ABD was used to further revise the draft Milestones. The revised draft was sent to all DMWG members for further input with subsequent revision and was then vetted by the executive leadership of the ABD, resulting in the implementation version of the Dermatology Milestones.

General Features of the Dermatology Milestones

The implementation version of the Dermatology Milestones contains 28 subcompetency areas across the 6 competencies, well below the 30 to 40 suggested by the ACGME. They include 7 subcompetency areas for PC, 5 for MK, 4 for systems-based practice, 4 for practice-based learning and improvement, 3 for professionalism, and 5 for interpersonal and communication skills. Each subcompetency was labeled with the initials for the competency followed by a number (eg, PC1, PC2). The implementation Milestones emphasize observable and assessable Milestones; allow for realistic progression of Milestones; use standard language across Milestones; eliminate redundancy; and include several Milestones in the Level 5 (practitioner) column that could be achieved by high-performing residents. Key words are bolded to make the Milestones user-friendly.

Envisioned Practical Use in Evaluating Residents

Longitudinal, multiyear data will need to be collected to establish Milestone validity, utility, and practicality, and to modify the Milestones as needed after broader use. Dermatology programs will form Clinical Competency Committees to assess resident Milestone performance. At the 2013 APD Annual Meeting, Dr Erik Stratman chaired an ad hoc committee to create assessment tools designed to feed directly into specific Milestone subcompetencies (box 3). Many of these tools are mini–clinical evaluation exercises (mini-CEXs) that emphasize direct observation of resident performance followed by 3 to 5 minutes for feedback. Each item of these assessment tools maps directly to 1 of the subcompetencies. The scores for each item can be directly transcribed onto a Milestone scorecard, with the aggregate of scores within 1 subcompetency determining the resident's level (1–5).

Use of the Milestones

The Dermatology RRC will track semiannual residents' aggregate data on the Milestones, along with other key indicators in the NAS to assess program educational effectiveness. Dermatology resident achievement of the Milestones is 1 measure in assisting programs to identify residents who may need remediation. Program Milestone performance also may be used to make changes in the academic curriculum, as needed. Resident and program performance on the Milestones will allow dermatology-specific normative data to be generated over time. At the completion of training, Milestone performance reporting will assist in documenting that trainees have demonstrated sufficient competence to enter practice without direct supervision, which will contribute to specialty accountability to the public. Future investigation may include the correlation of Milestone data with board certification rates and/or other existing validated frameworks.