Introduction

Medical education has been shifting from traditional, lecture-based teaching to approaches that promote higher-order thinking and active learning.¹ One particular approach, the flipped classroom (FC), has become increasingly popular in higher education, including medical education.² In this educational pedagogy, foundational content materials are studied by the learner independently through preclassroom activities, such as reading an article or textbook chapter, watching a multimedia presentation, or listening to a lecture in advance of classroom time. The classroom is reserved for applying foundational content knowledge in small group discussions involving clinical cases or more generic problem-solving. Conversely, in the traditional classroom, foundational content material is transmitted to learners through lectures that require review and reinforcement through study after class.^3,4

Advances in technology and the search for more effective approaches to teaching seem to be driving the shift to FC.⁵ Active, self-directed learning—a necessary component of the FC learning model—is consistent with the needs of resident learners.⁴ Additionally, this model is consistent with social, behavioral, and constructivist learning theories.^6,7 Group collaboration encourages modeling, scaffolding, and feedback that engage learners and facilitate the integration of new knowledge with old knowledge.⁸ Compared to the traditional lecture, the FC promotes higher levels of cognitive processing as defined by Bloom's taxonomy.^8–10 As a result, increasing numbers of educators have adopted the FC across a variety of educational settings.^11–19

Literature on the FC has proliferated rapidly across health sciences education since its inception in 2007.²⁰ Numerous articles originated from pharmacy, nursing, or veterinary medicine education programs.^{3,7,12,21–29} Both a recent systematic review and a meta-analysis covering the FC in medical education yielded very few articles on graduate medical education (GME).^3,30

The GME setting is more challenging than the undergraduate medical education (UME) setting. Resident learners may be less motivated by grades and more motivated by learning that facilitates patient care. Additionally, the GME learner's primary role is care provider while the UME learner's primary role is student. Given the marked differences in learning environments between UME and GME, and the FC's increasing popularity with residency programs,¹⁹ a review specific to the effectiveness of the FC method in GME was needed. The goal of this systematic review is to profile the use of and assess the quality of the research literature on FC methods used in GME.

Methods

This study conforms to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for systematic reviews and was performed in accordance with best practice guidelines.³¹

A health sciences librarian (S.J.S.) performed comprehensive searches of multiple databases, including PubMed, CINAHL Plus, Embase, Web of Science Core Collection, and ERIC, on July 27, 2017. Search alerts were used to gather new records through July 27, 2018. Major search terms and strategies are provided as online supplemental material. To identify additional manuscripts, we also searched MedEdPORTAL and reviewed bibliographies of included studies.

Results

Search Strategy

The figure illustrates the systematic review process. The initial search yielded 2562 articles. After removing duplicates, 2123 studies were screened using titles and abstracts. Articles were excluded based on criteria outlined earlier, which resulted in 116 articles for full-text review. Of these 116, an additional 94 were excluded because they did not meet criteria. The final list of articles that met criteria for quality review included 22 articles.^8,38–58

View Figure

• Download as Powerpoint
• Download Figure

Education Content

The 22 studies included in the final analysis are presented in tables 1 and 2. All were published in the past 5 years (2014–2018), most (86%) in the past 3 years. Across all studies, approximately 985 learners were involved with a flipped classroom intervention. Thirteen medical specialties were represented, including anesthesiology,^46,49,50,57 emergency medicine,^{8,38,44,52,58} internal medicine,³⁹ internal medicine–pediatrics,⁴³ neurology,⁴⁷ neurological surgery,⁴¹ obstetrics and gynecology,^55,56 pathology,⁴² pediatrics,^40,43,51 psychiatry,⁴⁵ radiology,⁵³ thoracic surgery,⁴⁸ and surgical intensive care.⁵⁴ All study designs were quantitative; however, a few gathered feedback through open-ended survey items.

table 1 Flipped Classroom (FC) Methodology in Graduate Medical Education

View Fullscreen

table 2 Summary of Study Outcomes and Analysis

View Fullscreen

Authors of reviewed studies offered numerous reasons for incorporating the FC into their residency programs. Some suggested that using the FC was a solution to scheduling issues—either saving time in already saturated didactic schedules or providing schedule flexibility.^{38,39,41,48,52,53,56} Others suggested that the FC method was selected to improve didactic instruction, motivate or engage learners, promote active learning, or prepare for more advanced levels of content material.^{38–42,44,46,47,49,54}

Residency programs adopted the FC method in 1 of 3 ways. Most commonly, authors described “occasional” use of the FC for a single lesson, usually to emphasize an important topic such as quality improvement,^39,43 resident-as-teacher skills,⁴⁰ using clinical guidelines,⁵¹ echocardiography or electro-encephalograph interpretation,^47,54,57 or pediatric advanced life-support.⁵² Other programs have replaced their entire didactic curriculum with FC sessions.^{41,44,48,50,55} Three articles describe the use of FC for portions of their didactic curriculum: intern orientation,^38,49,53 reinforcement of important basic science principles,^42,45,46 and weekly didactics during 1- or 2-month rotations.^39,47,54,55 As proof-of-concept projects, Haspel and Lockhart designed FC instruction for delivery at national conferences.^42,45

Concerns about the FC surfaced in some of the literature, particularly around learner compliance with self-directed learning. Rose and colleagues discovered that residents did not accurately report time spent viewing video lectures in preparation for FC activities.⁵² Residents in the Young et al study suggested that time for preclass preparation was a weakness of the FC format.⁵⁸ Without adequate preclass preparation, learners are unable to effectively participate in applying content knowledge during the in-class meeting. Consequently, several studies involved the production of innovative methods for delivering preclass content such as podcast lectures, interactive electronic modules, or multimedia textbooks. These materials were designed specifically to persuade learners into completing the self-directed learning required for constructive participation during in-class activities.^{41,45,47,49,50,52,57}

Discussion

This systematic review of FC application in GME yielded several important findings. While still in its early stages, research on the FC in GME has increased substantially over the past 5 years, with 17 of 22 articles being published since 2016. We discovered use of the FC in at least 13 types of residency programs, suggesting that the teaching method can be broadly adopted across a variety of GME settings. Reasons for using the FC and specific techniques on how it was applied vary greatly across programs, suggesting that it is not a one-method-fits-all intervention. We found that the FC has been applied to entire residency programs or, on a more limited scale, within rotations or to cover specific topics. The method has also been used across institutions to prepare learners for specialized topics, or to capitalize on educator expertise.

We attribute the rise of interest in the FC in GME to medical educators' desire to find better teaching methods, recognizing that this method is suitable for residents and can be adapted to the GME setting. The self-directed learning component gives residents more control, allowing them to learn content at their own pace and during times outside of their clinical responsibilities. Consistent with social learning theory, the FC provides a venue for collaborative learning where learners publicly demonstrate their application of knowledge during small group discussion of patient cases and problems.^7,8 Learner enthusiasm for the FC method and documented improvements in knowledge and skills make the FC an attractive alternative to the traditional lecture-based education model.

Medical educators have suggested that their interest in the FC is associated with making more effective use of instructional time, providing more structure for self-directed learning outside of class, or motivating residents to spend more time outside of clinical responsibilities engaged in study. The FC requires learners to complete self-directed learning activities to participate during in-class sessions. This was a concern among graduate medical educators because residents have clinical responsibilities that compete for their time. We believe many studies created more interesting methods for delivering preclassroom content because of this concern.

F. Chen and colleagues' earlier systematic review of the FC approach in medical education found that the “majority of literature has been carried out in UME.”³ Additionally, their review suggested that research up to that point “lacked strong evidence for the effectiveness of FCs.”³ A subsequent comprehensive meta-analysis by K.S. Chen et al covered a relatively small number of studies involving medical education and only one that involved residency education.³⁰ This group of authors tentatively suggested an advantage of the FC over lecture-based methods; however, they expressed concern about this interpretation due to extreme diversity in methods. Our systematic review focused solely on the use of the FC in ACGME-accredited residency programs. Like K.S. Chen, we confronted diversity in research methods, but also diversity in how the FC was applied and the content covered. In contrast with F. Chen, and comparable to K.S. Chen, we found that learners generally find the FC approach acceptable and evidence supports that the FC is as good as a traditional didactic approach (introducing topics through lecture or during a face-to-face meeting followed by readings and study of educational materials).

This systematic review of the FC literature has some limitations. While we employed what we think is a comprehensive search strategy with the help of an experienced medical librarian, it is possible that we did not include more esoteric terms that refer to methods associated with FCs. For example, we did not use the term “problem-based learning,” which is a method that typically describes a comprehensive approach to education, but could be associated with the FC format on a more limited basis. In addition, we made the decision to exclude non-ACGME residency programs and US fellowship programs. This decision was based on the variability of education structures found outside the United States and variability in size and purpose of fellowships. These decisions may have restricted the generalizability of our findings.

The assessment of higher-level outcomes, such as changes in learner behaviors or patient outcomes, remains a challenge in medical education.³⁶ These outcomes are difficult to assess because they rarely can be attributed to a single educational intervention. Despite these challenges, we were heartened to find an increasing number of studies that generated Kirkpatrick levels 2a, 2b, and 3 outcomes, many studies that employed objective measures, and an increasing number of studies that employed more rigorous research designs. Future research on the FC in GME should focus on higher-level outcomes such as changes in behaviors, clinical practice, and patient outcomes.³³

Conclusion

The FC pedagogical approach in GME has been implemented in a variety of ways and studied with a variety of methods, which has yielded variable results. Using MERSQI scores, studies evaluating the efficacy of the FC were somewhat less rigorous than typical medical education research studies; however, if pilot and proof-of-concept studies are eliminated, the average MERSQI score was comparable to that of other medical education research studies. The studies that evaluated resident satisfaction or efficacy concluded that residents held generally positive opinions about the FC. For studies that evaluated learning outcomes, the results were mixed: slightly more than half of the studies using a control group for comparison found positive learning results. Future studies of FC in GME should include higher-level outcomes (changes in knowledge, behaviors, or patient outcomes) and assessment of preclassroom assignment completion.