A Toolkit for Medical Education Scholarship

When I recall my early days of attempting to assess a brand-new curriculum, I see a clinician educator stumbling in the dark. The picture is not attractive—although I do miss the wrinkle-free skin and fluid gait. These days many junior educators are better prepared to conduct work worthy of dissemination, through conferences, courses, and medical scholar programs.^1,2 They also have access to a critical ingredient: mentors. Yet, the Journal of Graduate Medical Education (JGME) receives many “near miss” papers that lack key elements of scholarship (box). We often review potentially exciting studies that were planned without referencing prior work, or without the consideration of internal or external validity (ie, whether the findings are credible or might apply outside the study context).

This editorial considers the scholarship of discovery and the scholarship of teaching (table 1),^3,4 sometimes viewed as 2 ends of a research continuum.⁵ For medical educators with limited resources, this article proposes strategies for success based on evidence and the accumulated wisdom of JGME editors (table 2).

table 1 Types of Education Scholarship, after Glassick³ and Boyer⁴

View Fullscreen

table 2 Strategies for Starting Scholarship

View Fullscreen

Read

Other than perhaps a limited number of podcasts and videos, there is no way around it: you must read. Choose a medical education journal, or 1 to 2 topics, and read every month. Most medical education journals include all levels of physician and health professional training; some concentrate on a single area, such as JGME. Most peer-reviewed medical education journals are indexed on PubMed and accessible through institutions, including local public libraries linked to state universities. Others, including JGME, offer a large number of open-access articles at publication. Read articles for content, methods, and manuscript organization and format; these differ from journal to journal, and among article categories. For example, perspectives, reviews, original research, and personal essays have very different formats and styles.

The commitment to reading is not easy, as most clinician educators struggle to keep up in their clinical field. There are aggregated sources, such as online DR MERL (Dependable Reviews of Medical Education Research Literature)⁶ and KeyLIME podcasts (Key Literature in Medical Education),⁷ that summarize interesting new work. These are useful, but they do not allow for a close review of organization, format, or writing style. Rather than searching for articles, busy clinician educators can set up automatic system alerts, such as a journal's electronic table of contents or PubMed topic alerts, to “push” information automatically provided to you. Read closely at least 1 article per month—and do not skip the methods! If you assemble a group of interested faculty or trainees into a medical education journal club, this activity can accelerate learning for all members: more on this below.

Determine Your Focus

Finding an area on which to focus will require an assessment of the opportunities at your institution that best fit your interests and skills, both current and those that you envision acquiring. Are you considering administration (a role as program director, course director, or dean for graduate or undergraduate education) or educational program innovation (eg, curriculum design, assessment systems, or faculty development) or research (quantitative or qualitative studies to understand teaching and learning)? This is not an exhaustive list of the paths available to clinician educators.^8,9 As you cannot simultaneously acquire expertise in all of these areas, focus is essential.

For example, for educators imagining an administrative career, invitations to join institutional committees or participate in quality improvement projects may present opportunities, whereas for others interested in research, these activities may act as “time bandits.”¹⁰ Serving as a course director usually requires substantial time, but it may allow individuals interested in educational innovations the opportunity to create and evaluate new interventions.

Create a Personal Infrastructure

Structure is your friend: you will accomplish more when you have explicit plans and deadlines. Key elements include time to accomplish scholarly work, timelines, and deliverables. In addition, you may need to take a human subjects research course, such as the online CITI course,^11,12 even if your Institutional Review Board designates educational or quality improvement projects as exempt from review.

First, negotiate with your chair or supervisor for protected time—even 5% time, or 2 hours weekly (if we actually worked 40-hour weeks), is extremely useful. Negotiation means starting with an understanding of the educational needs of the department, and how your work can begin to solve this need. A realistic approach is critical: you cannot create a new fellowship, nor can you plan, conduct, and publish new research with 5% dedicated time, over 1 year. These projects will require more protected time. However, in the same time period with 5% protected time, you could develop a community of educators, create a medical education journal club, organize grand rounds or workshops for faculty development, or plan and start an educational research project.

Second, think strategically: start with a small pilot project that can grow into larger, longitudinal, multidisciplinary, or multi-institutional projects. Consider projects that may have multiple aspects for future exploration. Develop or enhance projects that ally with the needs of your physician undergraduate, graduate, or other health professional education needs.

Third, plan prospectively: retrospective evaluation, or retrofitting scholarship after the course/workshop/assessment has been conducted, may spoil the potential value of your work.

Fourth, a specific timeline is essential, although it may need to be revised. A timeline promotes using time productively and also reassures supervisors. Schedule scholarship activities directly into your calendar, and ask yourself: What will you do by when? It is best to agree to modest goals, and then exceed them.

Develop a Network of Collaborators and Mentors

You cannot do this scholarship alone.⁹ In addition to gaining support and sharing different skill sets, working in a group of collaborators promotes discipline and meeting deadlines. Most of us are less willing to disappoint colleagues than ourselves. Collaborators are not always found at your site, but may be identified through regional or national meetings (particularly interest groups that meet during meetings) or social media (eg, Twitter hashtags, blogs, and LinkedIn). Not everyone who attends an interest group will commit to group work, but this strategy can unearth gold: colleagues who become lifelong collaborators and friends. If you cannot attend away meetings, many interest groups have ongoing e-mail communications or periodic conference calls for the purpose of group work, such as developing workshops, national surveys, multi-institutional curriculum innovations, and review papers.

If there is a potential mentor with education scholarship experience at your site, negotiating and obtaining protected time with this individual can fast-track your development. The negotiation must include mentor deliverables: what will be done and when. For example, the mentor will: meet with you weekly or bimonthly; review your abstracts, posters, Institutional Review Board proposals, and papers before submission; regularly review progress toward education track promotion criteria; identify new career opportunities; and regularly discuss work assignments in light of your career goals. The mentor also commits to the deliverables identified for the mentee. Beyond these, a key ingredient for a successful mentor-mentee relationship is trust, which takes time.

If it is difficult to identify a local mentor, many national societies have developed formal mentoring programs. Also, some professional organizations encourage early submissions, and will provide constructive critiques by an experienced scholar at an upcoming meeting.¹⁵ If no senior mentor can be found, then peer mentoring communities, developed through collaborator networks, may fill some of this void. A committed peer group can provide a valuable audience for sounding new ideas, practicing oral presentations, and critiquing papers before submission.

Learn to Critique Scholarship and to Write

After regularly reading 1 or 2 medical education journals, sign up to be a reviewer. If you have no particular area of expertise, choose an area that matches institutional need and personal interest. If you have no reviewing experience, it is best to do this as part of a group review, with other faculty and trainees, or with a partner. Many journals, including JGME, encourage group reviews; see several JGME articles that describe this process.^16–18 Novice reviewers will acquire skills if at least 1 member of the group has experience reviewing, yet this is not always possible. In this case a group can review published new or classic medical education articles¹⁹ in a journal club–type conference.

For a general introduction to education scholarship, the Medical Education Research Certificate (MERC) program, a series of 9 workshops presented by the Association of American Medical Colleges (AAMC), is another option.¹⁴ These workshops are offered for a fee at the AAMC national meeting and at other education meetings. If you are unable to attend a medical education meeting, then suggest that your institution bring MERC on site, such as 1 workshop per year. In addition, many specialty societies offer relevant courses and workshops.

What if you do not have access to an experienced mentor? A peer network is essential: these are colleagues who will make time to read your writing and provide honest feedback. If peers find your writing confusing, it is likely that reviewers also will. Like any skill, writing well takes practice. Mentors can point out unclear writing and missing information, and suggest specific edits. Good writers are not born; they are created through the blood, sweat, and tears of continued practice.²⁰

Think Prospectively, Not Retrospectively

Glassick's criteria for scholarship³ include reviewing the literature and matching methods to the aim of your project, in the context of prior work. This requires planning for scholarship before the new curriculum, assessment tool, faculty development workshop, or wellness intervention. A common, often fatal flaw in studies submitted to JGME is the attempt to retrofit assessments or outcomes to an already delivered educational activity or survey. Major flaws arise in this situation, such as (1) no power analysis was performed; (2) outcome measures were those of convenience rather than chosen to match the intervention; (3) open-ended responses were relabeled as qualitative research without rigorous methods; (4) home-grown surveys did not undergo testing; and (5) no analysis of nonresponders or nonparticipants was performed. Understanding and using Glassick's criteria when planning scholarly work can prevent these problems.³

In addition, quality indices have been developed to assess research. Use of these indices at the beginning of an educational project or research study may identify potential problems. The Medical Education Research Study Quality Instrument (MERSQI),^21,22 Newcastle-Ottawa Scale-Education,²³ and Best Evidence in Medical Education (BEME)²⁴ indices for quantitative studies, and Côté and Turgeon²⁵ for qualitative studies are guides. Few if any studies will score perfectly on these indices. Review articles on the topic of quality are another resource for thinking before doing.^5,26

Bottom line: for each educational initiative you undertake, consider Glassick's scholarship criteria³ and how the project measures on a relevant quality index. Do this even when you do not plan to disseminate the project to a wider audience.

Write a Review Paper

While teaching, you will frequently encounter medical education questions that, after a brief PubMed or Google Scholar search, reveal no obvious answers. Also, at the beginning of any project, a literature search is essential. If you find no recent or readable review articles on a topic, consider writing a narrative or systematic review paper. Medical education reviews are difficult to conduct and write because definitions, methods, and outcomes often vary enormously; summarizing study findings can be a vexing process. However, even when best practices cannot be recommended, a good review paper is helpful to other educators in pointing out gaps and key next steps. Of note for those without funding, reviews are best done by a group, and may not require funding.

Pass It On

When you have developed expertise and a track record, help more junior colleagues. Develop a community of education scholars at your institution or in your local area with ongoing collaboration and peer mentoring. Encourage junior educators to become medical education journal reviewers through suggesting an appropriate journal, and assist them as they acquire skills. Become a mentor and advocate for scholarship. Pass it on.