A Practical Guide to Writing Effective Needs Assessments in Graduate Medical Education

Introduction

Developing a new curriculum without a needs assessment is like sailing a ship without a compass—ultimately, it can lead to an unfocused or poorly aligned curriculum. A needs assessment is a deliberate process to identify a curriculum’s goals, objectives, content, and educational strategies. Performing a needs assessment (both general and targeted) is a meaningful step in curriculum development.¹ A needs assessment, to guide development and implementation, can produce an effective graduate medical education (GME) curriculum. In addition, a needs assessment can help program directors and trainees identify curricular gaps and tailor new learning experiences to fill them. In this Editorial, we describe the needs assessment process and explicate several different approaches that are available to educators.

Do You Always Need to Perform a Needs Assessment?

Determining when to conduct a needs assessment is fairly straightforward: It should occur whenever a new or revised educational activity is planned.² Ideally, a needs assessment should be conducted before curricular change occurs. Given its essential nature, conducting a needs assessment is considered a best practice, whether through a brief literature review, review of existing data (eg, end-of-rotation resident and faculty surveys), or a more comprehensive process.¹

What Are the Different Approaches to Needs Assessments?

Determining the best type of needs assessment for a given curriculum development project is dependent on several factors. A needs assessment can be broad or focused, based in part on the depth of the educator’s current understanding of the content and the type of information needed to create the curriculum. In some situations, a curriculum already exists in the public domain (eg, MedEdPORTAL), and a literature review may provide sufficient information to create the new learning experience. In other situations, new data must be collected to inform the curriculum development process. Sensitive topics, such as workplace violence among trainees, may require interviews or focus groups to explore more in-depth discussions of emotions and behaviors surrounding personal experiences. Other curricular areas, such as prioritizing multiple topics and approaches, may benefit from surveying interest holders anonymously.

When conducting a needs assessment, it is essential to consider the scope and potential impact of the topic. For instance, a small, optional teaching conference on heart failure would likely require a less intensive needs assessment than a new, longitudinal patient safety curriculum. The resources available and timeline for the curriculum will also influence the process, as will the stakes of the curriculum. A large curriculum prioritized by the institution may require a more rigorous exploration using, for example, literature review, individual interviews, and multiple focus groups. In some cases, depending on the methodology, institutional guidelines, and whether you intend to disseminate the work, institutional review board (IRB) approval may be required. When in doubt it is best to obtain an IRB ethics determination before beginning the process.

A needs assessment can be performed using a variety of methods including literature review, interpersonal interactions, review of existing outcomes, consensus processes, or integration methods. These approaches can be categorized into 2 general types: (1) reviewing existing artifacts, such as literature and existing data, and (2) gathering and analyzing new data. Table 1 summarizes the advantages and disadvantages of the various needs assessment methods.

Table 1 Advantages and Disadvantages of Common Types of Needs Assessments

View Fullscreen

Literature Review

Needs assessments nearly always begin with a literature review to establish a solid foundation, outline the landscape of the topic, and provide insights from prior research.³ The literature review might include databases, textbooks, and other relevant publications (Table 2). It can be helpful to meet with an academic librarian to develop a search strategy and iterate the search process as the work unfolds. In our experience, librarians are extremely helpful but are often underutilized in GME.

Table 2 Examples of Approaches to Literature Reviews for Needs Assessments

View Fullscreen

Another information source is the grey literature, which can be defined as materials and research produced by organizations outside of traditional publishing channels. Grey literature includes reports, white papers, policy briefs, conference proceedings, technical documents, working papers, government publications, and other materials that are not formally peer-reviewed or published in journals.⁴ These materials can often be found using general search engines, such as Google. In some cases, a grey literature review can help support a traditional literature review. Within the grey literature are curriculum archives, which are databases of prior curricula that have been created and are often housed on university websites.

Interpersonal Interactions

Interpersonal interactions typically involve conversations and communications with key participants, including the target trainees.⁶ Mechanisms for encouraging these interactions include public town halls, focus groups, interviews, or informal discussions.

Selecting the specific type of interpersonal interaction will depend on the topic, desired information, and the complexity of the problem to be addressed. Town hall discussions, interviews, and focus groups can be structured to meet the needs of the assessment. For example, more focused questions can guide the conversation whereas inclusion of more open-ended questions can create more flexible conversations. Town halls may be conducted by open invitation, with participants unknown in advance, unlike interviews and focus groups in which specific participants are invited. When there is a concern that speaking in a group might suppress individual perspectives, interviews or individual informal discussions may be best.⁷ Alternatively, a focus group or town hall may fit better if the purpose is to capture emerging ideas or glean insights from interpersonal discussions and interactions. Often it is helpful to record and transcribe these conversations for review and analysis; such methods may require IRB approval.

Surveys

Surveys can include open- or close-ended questions completed by participants. Much has been written about the best ways to design, develop, and administer surveys. In general, surveys can be helpful for reaching many participants and capturing information that participants can reliably provide, such as opinions or personal facts. Readers are encouraged to review previously published recommendations before creating their own survey instrument.^8-13

Review of Existing Artifacts

Reviewing and analyzing existing educational artifacts is crucial for identifying gaps in learning or outcomes. Existing artifacts refer to a curriculum that is already in use. For instance, identifying lower scores on specific examination content or recurring patient safety issues can highlight areas needing improvement. By leveraging existing data, new curricular changes can be more effectively tailored to address these gaps.

Reviewing existing content and curricular artifacts is valuable when revising a curriculum, as it helps to identify commonalities and areas for alignment. For example, if when reviewing emergency medicine resident milestones focused on leadership and emergency stabilization, a gap is identified in leadership, a review of the current leadership curriculum could be utilized as part of the needs assessment. When reviewing prior content, it can be helpful to utilize a strengths, weakness, opportunities, and threats (SWOT) methodology for the current context, as it relates to the proposed curriculum.¹⁴ This can provide a comprehensive overview; however, it can be subjective, time-consuming, and at times oversimplify the problems. To mitigate this, it can be helpful to supplement SWOT with additional sources of information.

Consensus Methods

Consensus methods are systematic ways to establish group (often expert) agreement. Examples of consensus methods include nominal group technique and Delphi methods.¹⁵ Nominal group technique is a structured model that consists of a moderator asking a group questions and determining consensus in a meeting, which is focused on generating immediate data.¹⁶ This can be helpful when discussion and idea generation are needed as part of the consensus process. In one study by Thibault et al, nominal group technique was used to determine how to encourage residents to participate in medical education research.¹⁷ In 3 meetings with residents, the investigators explored themes and developed a framework that integrated residents as participants in education research during training.¹⁷ In contrast, the Delphi method involves the use of multiple (often anonymous) questionnaires over time to determine consensus on a topic where the core elements are largely known, but limited consensus is present.¹⁸

Integration Methods

Integration methods are a different way of combining knowledge across multiple sources of data and information. One example of this is the CLAIM model for clinical practice to foster interprofessional development (CLAIM: Competitive analysis, Literature review with thematic analysis, Ask stakeholders, Internal review by experts, and Mapping of curriculum).¹⁹ The CLAIM model is a multistage process that begins with a needs assessment and then maps objectives to a curriculum. The process involves 3 phases: (1) focus on discovery of new information; (2) ask key participants and experts to review the data; and (3) synthesize and map information to a curriculum.¹⁹

Another example of an integrative method is design thinking. Design thinking uses 5 stages—discovery, interpretation, ideation, experimentation, and evolution²⁰—to put new ideas into action.²¹ It has been used in a broad range of creative and innovative settings. Using design thinking to create a new didactic curriculum for a residency program can be very useful because it can result in more creative solutions.

Summary

Conducting a needs assessment is an essential step in designing an effective GME curriculum. The Journal of Graduate Medical Education encourages authors to conduct a needs assessment prior to implementing new curricular interventions and include this needs assessment as part of their submitted manuscripts, usually in the Methods section. Approaches to a needs assessment will align with the extent of the curricular interventions and may include literature review, interpersonal interactions, data analysis, consensus methods, and integration strategies. The specific process used will depend on the assessment’s goals and desired level of rigor, while factors such as available time, resources, and existing materials will play a major role in the choice of process. Ultimately, a well-executed needs assessment enriches the curriculum, fosters stakeholder buy-in, and lays a solid foundation for future educational initiatives.