Reimagining Residency Selection: Part 2—A Practical Guide to Interviewing in the Post-COVID-19 Era

MD,
MD,
MD,
MD,
MD,
MD, MCR, and
MD, MHPE
Online Publication Date: 01 Oct 2020
Page Range: 545 – 549
DOI: 10.4300/JGME-D-20-00911.1
Save
Download PDF

Even in the best of times, interview season can be daunting for both program leadership and prospective applicants. In 2020, amid the COVID-19 pandemic, national and specialty level organizations called for virtual-only interview processes for the 2021 graduate medical education (GME) selection period. This marks a landmark change in the way GME selection will occur and may irrevocably change the process going forward. In this article, we explore the recent literature from medical education, social sciences, and business fields to inform residency interview practices, with special attention on digital innovations that may replace or augment traditional interview day processes.

Interviewing: Before and After COVID-19

Residency programs devote an immense amount of time, effort, and resources to residency selection and recruitment. Information gleaned from a typical interview day is one of the most important factors to program directors when determining applicant ranking.1 The interview experience also highly influences applicant decision-making.2,3

The traditional interview process presents many challenges for applicants and residency programs. Program interview dates often do not take into account the interview dates of other programs. This lack of coordination makes medical students' travel planning difficult and often forces applicants to choose between program interviews. Other drawbacks to the current process include the financial burden and loss of time from medical school training.4,5 According to a national survey of more than 1000 graduating medical students, the average number of in-person interviews for applicants was 12.3, and the cost ranged from $1,000 to $5,000 for travel, lodging, and other expenses.6 The financial burden was greater for those applying to more competitive specialties and for those in the couples match.

Pre-COVID-19, some improvements to the interview system were undertaken. Online scheduling services give applicants and programs the ability to schedule, reschedule, and coordinate interviews independently. Some programs allow students to interview if they are participating in a visiting rotation rather than requiring a second visit for the formal interview. Other programs have attempted geographic coordination in which programs in close proximity organize their interview dates in succession. In 1994, the Canadian urology residency training programs designed a “residency fair”: a single site was used for a 1-day in-person opportunity to interview at multiple programs.7 These initiatives also reduce the environmental impact of thousands of miles of travel annually.

Although virtual interviewing holds promise to address many of the problems of the current system, little is known about the effects of a universal virtual interview process for GME. In May 2020, the Coalition for Physician Accountability recommended that all GME interviews be conducted online for the 2020–2021 academic year. These mandated changes present an unusual opportunity for residency selection reform.

Considerations for Virtual Interviews

Rather than adapting existing processes to digital platforms, program directors designing virtual interview processes should determine the goals of the interview process, and then intentionally design virtual experiences to achieve those goals. By performing a local needs assessment, programs can identify areas where previous processes may have gaps. These gaps may occur in gathering, organizing, and sorting information regarding applicants; effectively highlighting program strengths and unique characteristics; or ensuring that the interview process is efficient and equitable. To harness the strengths of technology, specific considerations include incorporating asynchronous longitudinal experiences, increasing interactive elements, selecting a format for formal interviews, and reenvisioning the program “brand.”

Incorporating Asynchronous Longitudinal Processes

Traditional components of the typical interview day have included a tour of the facilities, the program overview, faculty interviews, and a question-and-answer session with the residents that often occurs over a meal.4 Without the constraints of a time-bound in-person interview process, the interview experience may occur asynchronously and longitudinally. The separate elements of the traditional interview day may be spaced and delivered using a variety of media in order to increase knowledge transfer, make the program more memorable, and allow for a more holistic representation of the program and applicant. Program information can be conveyed prior to interviews with accurate, informative electronic manuals, videos of educational offerings, and typical faculty-resident interactions. Digital facility tours can ensure interviewees become familiar with personnel, academic and clinical opportunities, and culture. The program welcome, whether prerecorded or live streamed, should include an orientation to the virtual interview process in addition to any highlights about the program.8 With program orientation occurring in advance, interviews may focus less on delivering information and more on assessing applicant strengths and characteristics. Virtual “second look” visits may reinforce key program features and allow applicants opportunities to ask additional questions prior to rank list formulation. Previously, second look visits have generated controversy due to applicants' perceived pressure to attend one, despite the financial and logistical burden of travel.911 However, virtual second looks may provide similar benefits without the same challenges.

Increasing Interactive Elements

Opportunities for applicants and program faculty and residents to engage with one another have played an important role in rank list decision-making for both applicants and program directors, and must be established for the virtual interview process as well.1,3,6 Increasing interactivity in structured and unstructured virtual interview activities can help applicants assess program culture and personality, and help programs assess important applicant characteristics such as maturity, professionalism, and communication skills.

In the traditional format, informal interactions among applicants, residents, and faculty typically took place over meals. Programs can consider hosting virtual social hours with residents or coffee chats with the program director and other program leadership, either concurrently with the formal interview or longitudinally throughout the recruitment season. Breakout functions can allow for smaller group interactions. Like a good dinner party, grouping people with similar interests, when possible, may lead to richer discussions and a heightened experience for everyone. Creative strategies can be employed to support engagement, such as ice breaker activities, trivia games, or progressive dinners where applicants rotate through different breakout sessions. Options to create uniformity of experiences can include theming the event by topic or type of food. Depending on time zones and funding, it may be possible for a program to arrange multiple food deliveries to participants using meal delivery apps. A system that allows participants to RSVP may be prudent to assess level of interest and ensure manageable numbers so that meaningful interaction is possible. Programs can consider incorporating other interactive opportunities for applicants using social media, virtual community of practice platforms (eg, WhatsApp or Slack),12 or blogs13 throughout the interview time period.14

Selecting a Format for Formal Interviews

Digital interview platforms may offer logistical advantages over in-person interviews. Programs may choose to conduct interviews without video, in order to minimize the chance of bias. The digital format may also be harnessed to incorporate alternative strategies that have been used to assess interpersonal or communication skills, such as the multiple mini-interview1518 or group interviews, or to increase efficiency with parallel tracks of simultaneous interviews with digitally timed transitions. The same strategies recommended for in-person interviews should be considered for the virtual format: explicit written descriptions of the desired traits in an applicant; standardized questions to every applicant; provision of behavior-specific anchors for rating scales for interviewers; use of a scoring rubric to improve interrater and intra-rater scoring; use of multiple observers rather than a single interviewer; training of interviewers in format, scoring, and unethical and illegal question rules; and interviewer blinded to other application data to minimize bias.19,20

Reenvisioning Program Branding

As we discussed in our accompanying article on virtual recruitment, developing and conveying the program brand, or the set of associations that define that program and differentiate it from others, is a crucial component of any selection process.13,14 Although this identity will be conveyed in recruitment activities and media, it is important to consider how the interview experience (virtual or otherwise) also creates and conveys brand identity to applicants. Traditional interview processes may have evolved without a specific intention to convey the brand, and the transition to virtual interviews provides an opportunity for programs to explicitly consider their mission, vision, and values, and to develop a brand identity. Digital or virtual reality tours can overcome barriers of time and space to highlight geographically distant clinical sites, facilities, and aspects of the city and region, such as neighborhoods, restaurants, parks, and landmarks, although these techniques are not without limitations.21

The supplemental material of this article outlines strategies for implementing virtual interview processes and provides an overview of key considerations for virtual interviews.

Recognizing that changes in the application process may heighten anxiety for prospective applicants, programs should provide clear guidance to interviewees. Programs may provide applicants with an overview of the steps needed during the interview experience as well as concrete suggestions for how to navigate the virtual process. A sample applicant checklist is presented in box 1, which may be adapted to provide the same content to program interviewers. Box 2 contains a checklist for programs regarding technology considerations.

Potential Disadvantages to Virtual Interviews

Programs may find that communicating culture via virtual interviews is difficult. The digital format may also affect an interviewer's ability to connect with applicants. If aspects of the interview day that are important to applicants are not explicitly included virtual processes, such as portraying interactions among residents, faculty, and staff, applicants will not experience this key aspect of a program. Noncognitive and technical skills may be more difficult to assess over video. “Video fatigue” may become a problem as applicants spend prolonged periods of time in front of device screens. Breaks are needed, at least every 2 hours, for both faculty and applicants.

Conclusions

This transformation to virtual interviews may allow us to reconsider how our present systems perpetuate sociocultural biases. Race, gender, ethnicity, skin color, body habitus, and socioeconomic status have been shown to have profound effects on job selection.2225 In the current social climate, it is incumbent on program leaders to consider their own processes to minimize bias—both at a personal level for their interviewers, but also at a systemic level within the systems we use. Rather than rinsing and repeating our procedures from years past, programs can view the COVID-19 era as an opportunity to redesign interviews for the better.

Virtual processes may offer logistical advantages in organizing and scheduling interviews, and may reduce time, effort, and financial burden, but will ultimately need to be balanced against the difficult-to-replace personal aspects that in-person experiences offer. Programs should consider incorporating asynchronous longitudinal experiences, increasing interactive elements, selecting a format for formal interviews, and reenvisioning program branding while designing virtual processes that both address gaps of the traditional system and future program and applicant needs in the post-COVID-19 era.

References

  • 1
    National Resident Matching Program. Results of the 2018 NRMP Program Director Survey. https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf. Accessed August 12, 2020.
  • 2
    Schmidt LE,
    Cooper CA,
    Guo WA.
    Factors influencing US medical students' decision to pursue surgery. J Surg Res. 2016;203(
    1
    ):6474. doi:10.1016/j.jss.2016.03.054.
  • 3
    Love JN,
    Howell JM,
    Hegarty CB,
    McLaughlin SA,
    Coates WC,
    Hopson LR,
    et al.
    Factors that influence medical student selection of an emergency medicine residency program: implications for training programs. Acad Emerg Med. 2012;19(
    4
    ):455460. doi:10.1111/j.1553-2712.2012.01323.x.
  • 4
    Shah SK,
    Arora S,
    Skipper B,
    Kalishman S,
    Timm TC,
    Smith AY.
    Randomized evaluation of a web based interview process for urology resident selection. J Urol. 2012;187(
    4
    ):13801384. doi:10.1016/j.juro.2011.11.108.
  • 5
    Brummond A,
    Sefcik S,
    Halvorsen AJ,
    Chaudhry S,
    Arora V,
    Adams M,
    et al.
    Resident recruitment costs: a national survey of internal medicine program directors. Am J Med. 2013;126(
    7
    ):646653. doi:10.1016/j.amjmed.2013.03.018.
  • 6
    Association of American Medical Colleges.
    Fried JG.
    Cost of Applying to Residency Questionnaire Report. https://www.aamc.org/system/files/c/2/430902-costofapplyingtoresidency.pdf. Accessed August 12, 2020.
  • 7
    Grober ED,
    Matsumoto ED,
    Jewett MAS,
    Chin JL.
    The Canadian Urology Fair: a model for minimizing the financial and academic costs of the residency selection process. Can J Surg. 2003;46(
    6
    ):458462.
  • 8
    Association of American Medical Colleges. Virtual Interviews: Tips for Interviewers. https://www.aamc.org/system/files/2020-05/Virtual_Interview_Tips_for_Interviewers_05142020_0.pdf. Accessed August 12, 2020.
  • 9
    Nagarkar PA,
    Janis JE.
    Fixing the Match: a survey of resident behaviors. Plast Reconstr Surg. 2013;132(
    3
    ):711719. doi:10.1097/PRS.0b013e31829ad2bb.
  • 10
    Sebesta EM,
    Lipsky MJ,
    Nunez M,
    Cooper KL,
    Badalato GM.
    The National Resident Matching Program code of conduct: what is the perceived degree of compliance during the urology match process? Urology. 2018;122:3743. doi:10.1016/j.urology.2018.04.046.
  • 11
    Nikonow TN,
    Lyon TD,
    Jackman SV,
    Averch TD.
    Survey of applicant experience and cost in the urology Match: opportunities for reform. J Urol. 2015;194(
    4
    ):10631067. doi:10.1016/j.juro.2015.04.074.
  • 12
    Sarkisian SA,
    Kagel A.
    The use of Slack for medical residency development and recruiting. BMJ Health Care Inform. 2018;25(
    3
    ):194195. doi:10.14236/jhi.v25i3.1019.
  • 13
    Shappell E,
    Shakeri N,
    Fant A,
    Branzetti J,
    Gisondi M,
    Babcock C,
    et al.
    Branding and recruitment: a primer for residency program leadership. J Grad Med Educ. 2018;10(
    3
    ):249252. doi:10.4300/JGME-D-17-00602.1.
  • 14
    Haas MRC,
    He S,
    Sternberg K,
    Jordan J,
    Deiorio NM,
    Chan TM,
    et al.
    Reimagining residency selection: part 1—a practical guide to recruitment in the post-COVID-19 era. J Grad Med Educ. 2020;12(
    5
    ):xxxxxx.
  • 15
    Dore KL,
    Kreuger S,
    Ladhani M,
    Rolfson S,
    Kurtz D,
    Kulasegaram K,
    et al.
    The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions. Acad Med. 2010;85(
    10 suppl
    ):6063. doi:10.1097/ACM.0b013e3181ed442b.
  • 16
    Eva KW,
    Rosenfeld J,
    Reiter HI,
    Norman GR.
    An admissions OSCE: the multiple mini-interview. Med Educ. 2004;38(
    3
    ):314326. doi:10.1046/j.1365-2923.2004.01776.x.
  • 17
    Joshi NK,
    Yarris LM,
    Doty CI,
    Lin M.
    Social media responses to the Annals of Emergency Medicine residents' perspective article on multiple mini-interviews. Ann Emerg Med. 2014;64(
    3
    ):3205. doi:10.1016/j.annemergmed.2014.07.024.
  • 18
    Phillips AW,
    Garmel GM.
    Does the multiple mini-interview address stakeholder needs? An applicant's perspective. Ann Emerg Med. 2014;64(
    3
    ):316319. doi:10.1016/j.annemergmed.2014.01.021.
  • 19
    Stephenson-Famy A,
    Houmard BS,
    Oberoi S,
    Manyak A,
    Chiang S,
    Kim S.
    Use of the interview in resident candidate selection: a review of the literature. J Grad Med Educ. 2015;7(
    4
    ):539548. doi:10.4300/JGME-D-14-00236.1.
  • 20
    Gordon MJ,
    Lincoln JA.
    Family practice resident selection: value of the interview. J Fam Pr. 1976;3(
    2
    ):175177.
  • 21
    Zertuche J-P,
    Connors J,
    Scheinman A,
    Kothari N,
    Wong K.
    Using virtual reality as a replacement for hospital tours during residency interviews. Med Educ Online. 2020;25(
    1
    ):1777066. doi:10.1080/10872981.2020.1777066.
  • 22
    Kutcher EJ,
    Bragger JD.
    Selection interviews of overweight job applicants: can structure reduce the bias? J Appl Soc Psychol. 2004;34(
    10
    ):19932022. doi:10.1111/j.1559-1816.2004.tb02688.x.
  • 23
    Harrison MS,
    Thomas KM.
    The hidden prejudice in selection: a research investigation on skin color bias. J Appl Soc Psychol. 2009;39(
    1
    ):134168. doi:10.1111/j.1559-1816.2008.00433.x.
  • 24
    Edmond MB,
    Deschenes JL,
    Eckler M,
    Wenzel RP.
    Racial bias in using USMLE Step 1 Scores to grant internal medicine residency interviews. Acad Med. 2001;76(
    12
    ):12531256. doi:10.1097/00001888-200112000-00021.
  • 25
    Desbiens NA,
    Vidaillet HJ.
    Discrimination against international medical graduates in the United States residency program selection process. BMC Med Educ. 2010;10(
    1
    ):5. doi:10.1186/1472-6920-10-5.
Copyright: 2020
pdf

Author Notes

Editor's Note: The online version of this article contains strategies for implementing virtual interview processes and an overview of key considerations for virtual interviews.

Corresponding author: Teresa M. Chan, MD, MHPE, McMaster University, Department of Medicine, 237 Barton Street East, McMaster Clinics, Suite 255, Hamilton, Ontario L8L 2X2, Canada, 905.906.8981, teresa.chan@medportal.ca
  • Download PDF