Introduction

To become a licensed physician in the United States, one must hold either a doctor of osteopathic medicine (DO) degree or a doctor of medicine (MD) degree, pass the respective licensing examinations, and complete residency training. The DOs and MDs each have their own licensing pathways: the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) for DOs,1 and the United States Medical Licensing Examination (USMLE) for MDs.2 Residency programs consider the scores on USMLE Step 1 and/or COMLEX-USA Level 1 to be one of the most important factors in the selection process of medical students into US residency programs.3–5

Although not required, many osteopathic medical students often take both COMLEX-USA Level 1 and the USMLE Step 1. Because both the USMLE and the COMLEX-USA aim to test a minimal knowledge competency required for a physician to practice medicine in the United States,1,2 it is reasonable to assume that the 2 examinations correlate with each other. To our knowledge, there are 4 previously published studies, each establishing a correlation between the scores of the 2 examinations.6–9 The study by Slocum and Louder7 also provides a formula predicting the scores between the 2 examinations. The study has a sample size of 155 and is based on a single class from a college of osteopathic medical school. Another study by Sarko et al6 also provides a conversion formula, using a sample size of 90 with examination results again from a single class. The National Board of Osteopathic Medical Examiners, the official body that creates and administers the COMLEX-USA, cautions against the direct comparison of the 2 examinations, noting that the COMLEX-USA Level 1 is designed to test the distinct osteopathic philosophy and medical practices.10,11 It is unknown whether Accreditation Council for Graduate Medical Education (ACGME) program directors use formulas to convert osteopathic medical student residency candidates' COMLEX-USA scores into comparable USMLE scores, but it is likely that some program directors use a conversion formula to determine the strength of an applicant.

The USMLE Step 1 or COMLEX-USA Level 1 scores can influence applicants' ability to match into particular programs, thus the validity of the formula may be crucial. The goal of this study was to further investigate the correlation between the 2 examinations and to assess the predictability of USMLE scores from COMLEX-USA scores. The study also assessed the accuracy of the existing formulas in predicting USMLE scores from COMLEX-USA scores for those who took both examinations at Western University of Health Sciences, College of Osteopathic Medicine of the Pacific (COMP), using a larger sample size and data from multiple years.

Methods

Results

We identified 1440 first attempt COMLEX-USA Level 1 examinations taken by COMP students between 2006 and 2012. Of those 1440 students, 1016 (71%) also took USMLE Step 1. The table shows the percentage of students, by graduating class, who took COMLEX-USA Level 1 and USMLE Step 1.

TABLE Numbers and Percentages by Graduating Class of Students Who Took Both the Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 1 and the United States Medical Licensing Examination (USMLE) STEP 1

View Fullscreen

The figure presents the scatterplot of the paired USMLE Step 1 and COMLEX-USA Level 1 three-digit scores. We observed a positive correlation with a Pearson coefficient of 0.834. The regression analysis yielded the following formulae: USMLE Step 1  =  0.239 × COMLEX-USA Level 1 + 82.563 (R²  =  0.69577) or COMLEX-USA Level 1  =  2.909 × USMLE Step 1 − 80.592 (R²  =  0.69577).

View Figure

• Download as Powerpoint
• Download Figure

The mean ± SD scores for the examinations were 524.48 ± 73.885 and 208.02 ± 21.188 for COMLEX-USA Level 1 and USMLE Step 1, respectively. Estimated USMLE Step 1 scores from COMLEX-USA Level 1 scores using the Slocum and Louder7 formula (USMLE Step 1  =  67.97 + 0.24 × COMLEX-USA Level 1) and the Sarko et al6 formula (USMLE-1  =  105.8 + 0.18 COMLEX-1) were calculated. The mean ± SD difference between the actual USMLE Step 1 and the estimated USMLE Step 1 scores by the Slocum and Louder7 formula was 14.17 ± 11.69 (P < .001) and by the Sarko et al6 formula was 7.80 ± 12.48 (P < .001).

Discussion

The COMLEX-USA Level 1 and USMLE Step 1 scores are often used by program directors as one of the deciding factors when ranking residency applicants.3–5 With increasing numbers of osteopathic medical students entering ACGME-accredited residencies,12 there is a strong possibility that program directors will encounter more students with only COMLEX-USA Level 1 scores. If they are unfamiliar with the COMLEX-USA Level 1, their main option is to use an existing formula to convert those scores into a more familiar USMLE Step 1 score. Our study results caution against that, advocating for each USMLE Step 1/COMLEX-USA Level 1 score to be evaluated independently. It is our hope that our findings will help program directors of ACGME-accredited programs evaluate candidates more accurately when considering matching them into their residency program. The formula derived by Louder and Slocum7 underestimated the USMLE Step 1 scores when compared with the actual USMLE Step 1 scores, by a statistically significant average of 14.16 points (6.8% of the average score). Additionally, the formula derived by Sarko et al6 underestimates the USMLE Step 1 scores when compared with the actual USMLE Step 1 scores by a statistically significant average of 7.80 points. If a program director were to use either of those formulae to convert the score of an osteopathic medical student with only a COMLEX-USA Level 1 score to a USMLE Step 1 score, the applicant could be put at a disadvantage. The reason for the differences in the conversion of COMLEX-USA Level 1 to USMLE Step 1 may be due to many factors such as variation in the curricula for different osteopathic medical schools, changes in the examination content of both COMLEX-USA Level 1 and USMLE Step 1, and the larger sample size used in our study. However, similar to the results of the previous studies, our results did show a positive linear correlation between the USMLE Step 1 and COMLEX-USA Level 1 scores. We performed a subanalysis matching the results from the Slocum and Louder7 study to the results from the 2006 COMP students (the same year) and compared the scores. A statistically significant difference of 8.96 was found, demonstrating that the change in examination content cannot fully explain the difference in conversion of COMLEX-USA Level 1 to USMLE Step 1. Unfortunately, we were unable to do a similar subanalysis with data from the Sarko et al6 study because the required data fell outside of our population range.

A limitation of our study is that it was based on the scores of a single osteopathic medical school and thus might not be generalized to the entire osteopathic medical student population. Additionally, the reasons for students taking the COMLEX-USA Level 1 only, or both the COMLEX-USA Level 1 and USMLE Step 1, were not investigated in this study. This may also have a bearing in a student's examination preparation, which may influence the results of the study. There is a potential for a self-selection bias because stronger candidates may opt to take both the USMLE Step 1 and the COMLEX-USA Level 1, thereby biasing the results. That possibility could be investigated in a future study to determine whether there is a correlation in students' preclinical grade point average and those who took both the COMLEX-USA Level 1 and USMLE Step 1 versus those who only took the COMLEX-USA Level 1. We plan to use this study as a springboard for further investigation to include the involvement of different osteopathic medical schools in the future. The USMLE and COMLEX-USA are primarily designed as licensing examinations, and the repurposing for secondary use by programs/program directors for resident selection was not their original intention.1,2 Finally, the COMLEX-USA Level 1 and USMLE Step 1 are distinct examinations. Although there are some similarities, there are also some major differences in the content tested between the 2 examinations, with the focus on osteopathic practice and principles in the COMLEX-USA being the largest difference.

Conclusion

Program directors of ACGME-accredited programs should use caution when using any formula to derive and calculate USMLE Step 1 scores from COMLEX-USA Level 1 scores. Although a strong correlation does exist between the 2 examinations, program directors are encouraged to familiarize themselves with, and evaluate the content of, USMLE Step 1 and COMLEX-USA Level 1 independently.