To the Editor: Addressing Limitations of a Standardized Letter of Recommendation

We would like to thank the authors of the recent article “A Call to Action for Standardizing Letters of Recommendation”¹ for their thoughtful summary of the current evidence for standardized letters for residency applications and their call for adoption across more specialties. While we agree that standardized letters overall demonstrate an improvement over the narrative letter of recommendation and that their adoption will likely lead to improvements in the residency selection process, the standardized letter should not be viewed as a panacea for discriminatory practices. Respectfully, their article does not address the negative aspects of standardized letters—particularly the structural bias in this evaluation tool. Further, without acknowledging and addressing the downside of tools currently in use, we lose an opportunity to improve these tools as we seek an equitable future in medical education.

Specifically, the authors mention the need to address bias, “which may affect UIM (underrepresented in medicine) applicants.” This statement is unassailable; however, changing to a standardized letter alone will not mitigate racial/ethnic bias. Our recent article provided evidence that the emergency medicine standardized letter of evaluation (EM SLOE) demonstrates differences in rankings by race/ethnicity.² Additionally, studies have highlighted gender bias in EM SLOE comparative rankings as well as in the narrative portion of the letter.^3,4 Calls for all specialties to create a SLOE need to acknowledge these limitations. Conceding the limitations of the SLOE is vital so that those creating new standardized letters for their specialties can work to mitigate existing bias. Our article details an approach that may help to mitigate bias in new standardized assessment tools.²

Further, the authors here do acknowledge that standardized letters do not stop authors from “inflating students' qualifications,” but stop short of acknowledging the limited validity evidence currently supporting the EM SLOE.⁵ Fully appreciating the limitations of this tool will help future users design a standardized letter that has better value in measuring what program directors value in a residency applicant.

We agree with the authors that more specialties should look to implement a standardized letter to improve the residency application process. However, it is important to learn from the past so that we may create the best possible tools to promote equity within medical education.