Evaluating Methodology for Increasing Diversity in US Residency Training Programs: A Scoping Review
ABSTRACT
Background There remains limited understanding of effective strategies to increase diversity, equity, and inclusion within residency programs—highlighting the need for a comprehensive review of current interventions.
Objective To synthesize literature regarding interventions to increase the representation of populations underrepresented in medicine (URiM) within US residency programs.
Methods A scoping review of studies published from January 2000 to July 2023 was conducted. Data were extracted from PubMed, Embase, Cochrane Library, and Scopus. URiM was defined by race, ethnicity, and gender. Studies were included if an intervention was implemented by a graduate medical education residency program but were excluded if they did not describe a defined intervention or if they were published outside of the United States. Study interventions were categorized into 5 areas: applicant factors, selection measures, application screening, interviews, and post-interview communication.
Results Initial search captured 2683 titles and abstracts; 257 full-text articles were reviewed, with 27 eligible articles meeting inclusion criteria. Eligible articles were categorized as: applicant–12 (44%), selection–8 (30%), screening–17 (63%), interview–8 (30%), and post-interview–4 (15%). Many articles addressed multiple interventions that positively impacted URiM composition, making it difficult to isolate the effect of individual interventions. Common, effective interventions included holistic reviews, clerkships, and standardized interviews.
Conclusions This review demonstrates that interventions aimed at increasing diversity in residency programs vary in their approaches, but consistent evaluation and evidence of effectiveness are lacking from the current literature.
Introduction
Initiatives aimed at increasing the representation of individuals underrepresented in medicine (URiM) within graduate medical education (GME) are crucial to cultivating a diverse physician workforce. This lack of representation not only raises concerns of fairness but also impacts the quality of patient care.1-3 A growing body of evidence demonstrates that when patients are treated by physicians who share similar backgrounds, outcomes improve due to better communication, trust, and cultural understanding.4-7 In an increasingly diverse and multiracial society, equitable representation in medicine is essential to reflect and address the needs of all communities.5
While many institutions have implemented diversity initiatives, substantial gaps remain across most medical specialties—particularly in competitive fields such as surgery, orthopedics, and cardiology.8-12 Barriers such as limited mentorship, narrow definitions of merit based on test scores, and unequal access to research opportunities disproportionately affect individuals from underrepresented racial, ethnic, and gender groups.13-17 Existing reviews have tended to focus on single specialties or narrow demographic groups, offering program directors limited guidance on how to intervene earlier in the pipeline.
This scoping review synthesizes published interventions designed to increase racial, ethnic, and gender diversity in residency programs. By identifying patterns across specialties and highlighting common practices, this review aims to provide GME leadership with a comprehensive and practical summary of approaches to support diversity, equity, and inclusion (DEI) within recruitment practices.
Methods
We conducted a scoping literature review to evaluate measures taken to promote racial, ethnic, and gender diversity among residents of US residency programs within any medical or surgical specialty. A scoping review was selected because, upon examining the literature, we determined a systematic review or meta-analysis was not feasible due to heterogeneous data and the inclusion of multiple interventions within a single study.
The literature search was conducted in July 2023 and was limited to studies published from January 2000 to July 2023, using the following databases: PubMed, Embase, Cochrane Library, and Scopus. Phase 1 involved screening titles and abstracts for relevance to the topic of interest. Phase 2 involved an evaluation of the full text to ensure studies met our inclusion criteria. The search methods were reported using relevant items of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis for Scoping Reviews (PRISMA-ScR) checklist.18 Major Medical Subject Heading (MeSH) terms used for the systematic search created by Associate Librarian, M.P., included DEI, reduce bias, mitigate bias, minimize bias, minority groups, cultural diversity, internship, residency, residency education, residency training, resident education, resident training, graduate medical education, medical internship, medical resident, residency program, and residency internship. This process was completed with the assistance of review software (Covidence).
Eligibility Criteria
Studies were included if they integrated a clear intervention(s) to increase racial, ethnic, or gender diversity in residency recruitment. Studies were eliminated if no specific action item was integrated to increase diversity (eg, a review of social media posts mentioning diversity initiatives). Studies published outside of the United States or those related to medical student or fellowship recruitment were excluded.
Title/Abstract and Full-Text Review
Author roles and qualifications at the time of literature review were as follows: L.M.B. was a second-year otolaryngology resident, H.C.M. was an assistant professor, D.M. was a first-year medical student, M.P. was an associated librarian, S.H.C. was an associate professor, E.E.H. was an associate professor, and J.A.A., the senior author, was a professor who has contributed to several subspecialty consensus statement publications. After removing duplicates, 3 authors (H.C.M., L.M.B., D.M.) independently reviewed the titles and abstracts. The full-text review was completed by L.M.B. and H.C.M. E.E.H. and J.A.A. completed a calibration exercise to review 5% of the abstracts and titles with >90% agreement that these studies met the criteria for full-text review. Upon full-text review, any remaining discrepant opinions between H.C.M. and L.M.B. regarding whether an article met all eligibility requirements for inclusion were resolved by E.E.H. and J.A.A.
As demonstrated in Figure 1, 5 categories were used to organize articles following the processes of applying to residency: applicant factors, selection measures, application screening, interviews, and post-interview communication. These were developed by the author group to compartmentalize interventions based on the chronological order of the residency application process. Applicant factors included increasing awareness of programs to a diverse applicant pool through outreach efforts, mentorship pathways, and away rotations. Program selection included institutional-level interventions, in which programs focused on internal factors such as bias training, diversity of faculty, or intentional mission statements. Application screening referred to interventions aimed at the residency review process: blinding aspects of applications, formalized rubrics, and holistic reviews. The interview category involved standardized interview questions or increasing URiM faculty visibility on interview day. Post-interview interventions included additional communication with applicants to emphasize commitment to diversity. Most of the articles addressed multiple interventions within a single article.
Citation: Journal of Graduate Medical Education 17, 5; 10.4300/JGME-D-24-00740.1
After analyzing each article, H.C.M. and L.M.B. assigned a quality assessment using the Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence.19 The study was institutional review board exempt.
Results
Study Characteristics
After removing 27 duplicates, the initial search captured 2683 titles and abstracts. A full-text review of 257 articles determined the set of articles for final review. A total of 27 articles were ultimately included based on the specified criteria (Table and Figure 2).
Citation: Journal of Graduate Medical Education 17, 5; 10.4300/JGME-D-24-00740.1
The final 27 articles were divided into the categories listed above and noted in Figure 1, based on the interventions addressed. The number of articles included in each category is found in Figure 3. Some were included in multiple categories, making the total percentage greater than 100%. Numerous specialties were represented in this scoping search. Interventions took place between 2005 and 2022. A summary of interventions and quality assessment is expanded in the Table.
Citation: Journal of Graduate Medical Education 17, 5; 10.4300/JGME-D-24-00740.1
Applicants
The most common theme throughout this category was the successful increase in residency diversity following the implementation of funded visiting clerkship rotations. Of the 27 articles meeting final inclusion criteria, 8 (30%) supported medical student clerkships to increase clinical or research exposure to a specialty.20-27 Importantly, these programs were typically funded by the hosting departments, and most included stipends to cover living expenses. Funded rotations, outreach efforts, and scholarships successfully increased recruitment and matching rates of URiM candidates to programs.
Programs hosted outreach events to reach a more diverse applicant pool. Most outreach to URiM residents and medical students was offered at local, regional, and national meetings.22,27-29 Several programs provided early mentorship to URiM students through interactions with current URiM residents informally or through the Alliance of Minority Physicians.20 Lall et al reached potential applicants by forming relationships with historically black colleges and universities.30 Other outreach avenues included GME-wide recruitment programs in the form of virtual URiM recruitment diversity brunches.31 Outreach events or points of contact with applicants have demonstrated effectiveness in increasing the number of URiM applications programs receive.
Selection
The focus of the selection interventions revolved around intentional efforts by departments to prioritize diversity efforts. Strategies within this category included highlighting a program’s commitment to diversity throughout the residency application process, particularly during interviews.26,32-34 The required buy-in by the department was explicitly mentioned as a part of the strategy.26,28,35 Several programs dedicated representatives of the institution’s diversity committee to work alongside the GME and individual departments to develop specific diversity plans.36,37 Diversity plans consisted of revising mission statements and creating diversity task forces. Anti-bias training for those individuals involved in the recruitment and selection process of residents was prioritized.20,33,37 Programs committed to ongoing evaluation of their diversity efforts and their outcomes through these taskforces.29,34,37 Ultimately, these selection interventions underscored a deliberate and sustained commitment by departments to prioritize diversity throughout the residency recruitment process.
Screening
The most commonly reported change among programs was the adoption of a holistic application review process.21,22,27,30,32,34,38-42 While the specifics of these “holistic residency application reviews” were not always clearly defined, the overarching goal was to align the selection process with each institution’s mission and values. Studies that focused exclusively on implementing holistic review models reported increases in the number of URiM applicants who were interviewed and matched.38,40-42 To support reviewers in applying holistic principles, several programs developed structured scoring systems that emphasized desirable applicant attributes—ranging from academic performance to personal experiences.30,33,41,43 For example, one program introduced a “distance traveled” question, which prompted reviewers to consider applicants’ contextual lived experiences and challenges; faculty reported gaining new insights that influenced their decisions.39
To reduce implicit bias, one study removed applicant names and photographs during review. Interestingly, URiM applicants had higher interview rates in the unblinded condition, suggesting that full anonymization may not always benefit diversity goals.44 Other strategies to improve equity in the review process included replacing United States Medical Licensing Examination Step 1 cutoffs with alternative assessments, such as situational judgment tests and scoring algorithms tailored to institutional priorities.45 Two additional studies reduced the weight of Step 1 scores in interview decisions by using Gestalt-based evaluations or treating the score as pass/fail prior to the national shift.28,35,46
Interview
Despite receiving interview offers, URiM students may still face disadvantages during the interview phase that affect their chances of matching. This review found several programs that initiated changes to the interview encounter to reduce bias by asking a standardized question among all applicants.29,32 One program blinded the interviewers to the interviewee’s application to provide a more unbiased assessment.29 The largest focus of this section, however, was to increase the visibility of the program’s existing diversity by having purposeful recruitment of the URiM resident candidates by URiM faculty during the interview day.22,26,29,32-35,44 These targeted interview day strategies—particularly efforts to increase URiM visibility—were associated with higher URiM representation in matched cohorts.
Post-Interview
Three studies included an additional outreach component to applicants in the form of post-interview email correspondence.20,29,34 There was no study to examine the isolated effect of post-interview communication, as these efforts were studied in conjunction with other interventions. Another follow-up opportunity involved hosting a funded second-look event to alleviate any additional concerns students may have regarding a specific program.26 This extra point of contact was important for applicants to feel confident about the suitability of a program, especially regarding mentorship and acceptance of URiM status.
Limitations of the Literature
The current literature aimed at increasing URiM populations in medical residency programs is heterogeneous, thus limiting the ability to provide summary data, recommendations, or meta-analysis regarding the effectiveness of a particular intervention. Another limitation of the current literature is the frequent combination of multiple interventions within a single article. This limitation makes the degree of impact of a single change within an institution challenging to assess in isolation.
Discussion
The review identified several recurring strategies across the residency selection continuum aimed at increasing URiM representation. These included applicant-focused interventions such as funded clerkships and mentorship; selection and screening reforms like holistic reviews and showcasing departments’ diversity efforts; interview modifications to reduce bias and enhance representation; and post-interview efforts such as tailored communication and second-look events to reinforce program commitment to diversity. The literature suggests that a comprehensive, longitudinal approach is necessary to meaningfully support URiM applicants across every phase of the GME selection process.
Efforts to improve diversity in GME reflect broader trends seen in medical school admissions and other professional fields. For example, holistic application reviews have been adopted across GME and undergraduate medical education to broaden access for URiM groups.47 Similar strategies have emerged in allied health professions, including pharmacy and physical therapy, where these interventions are employed to engage underrepresented students and diversify the workforce.48,49 These parallels highlight a growing recognition that equity-focused admissions practices are essential for building inclusive, representative institutions aligned with the communities they serve.
Compared to earlier reviews that were limited by specialty scope, timeframe, or narrow definitions of diversity, this study offers a broader cross-specialty synthesis and identifies interventions associated with increased URiM representation across varied institutional contexts.50-52 However, institutional differences in resources, leadership commitment, and overall culture strongly affect the feasibility and impact of these interventions. Programs with limited URiM faculty or minimal institutional support may find it difficult to implement mentorship or outreach strategies without placing an undue burden on existing URiM trainees and faculty—a phenomenon known as the “minority tax.”53-55 Varying levels of diversity across specialties can also shape a program’s prioritization of DEI efforts. By aggregating and contextualizing these findings, this review offers GME programs a foundation to critically evaluate and adapt diversity strategies that are both achievable and aligned with their institutional contexts.
Several limitations of the study should be considered. The nature of a scoping review prioritizes mapping the existing literature over evaluating the quality or effectiveness of interventions, restricting our ability to draw evidence-based conclusions or make definitive recommendations. This methodology is especially constrained when the studies found in our literature review varied widely in design, scope, and reported outcomes. As a result, we cannot assess which recurring strategies are most effective or generalizable across programs. Additionally, potential author biases may have shaped the interpretation of this review. Our lived experiences and identities—including those who identify with groups underrepresented in medicine—may have influenced how we framed findings or interpreted gaps. Alternative explanations for the reported increases in URiM representation—such as concurrent national or institutional policy changes—could not be assessed based on the available data. Future research using systematic review methods or longitudinal outcome evaluations would help determine the relative impact of interventions and guide evidence-based best practices.
Further research should examine how diversity-focused interventions can be adapted to fit the specific needs of various US GME programs and evaluate the long-term outcomes of URiM trainees who match through these enhanced processes. This is especially important as recent legislative changes at the state and national level threaten the sustainability and legality of DEI efforts through bans, funding cuts, and institutional rollbacks.56 Identifying mechanisms to reduce the minority tax while preserving meaningful engagement from URiM trainees and faculty remains a critical area for future study.
Conclusions
This scoping review summarizes interventions designed to improve racial, ethnic, and gender diversity in US residency programs. Across specialties, common themes included offering structured clerkship opportunities, adopting holistic application review processes, modifying interview practices to reduce bias, and actively demonstrating a program’s commitment to diversity through intentional messaging and outreach, visible role models, and formalized diversity efforts. The interventions described in this study were frequently associated with increased URiM representation among matched residency applicants offering GME programs a concise overview of evidence-based strategies to guide DEI efforts.
Categories of Interventions Used in the Residency Application Process to Increase URiM Representation
PRISMA Study Flow Diagram
Number of Studies With Interventions Targeting Specific Time Points in the Residency Application Process (Figure 1)
Author Notes
