Program Evaluation Models: Evaluating Processes and Outcomes in Graduate Medical Education

The Challenge

Program directors, designated intuitional officials (DIOs), and other graduate medical education (GME) educators strive to ensure the quality of their training programs. For example, faculty at annual program evaluation retreats often ask, “Is our training being implemented as we intended?” or “Is our training really working?” They may not recognize that these are 2 different types of questions; both may be answered differently depending on the program evaluation model used. Each model provides a different vantage point to inform the development and implementation of an educational initiative (process evaluation) or from which to judge the value or effectiveness of an educational initiative (outcome evaluation). Familiarity with commonly used models can optimize GME educators' ability to obtain actionable answers to a program's evaluation questions.

What Is Known

Two program evaluation models, appreciative inquiry and the logic model, have been successfully used to frame GME program evaluations (table).

table Key Features, Assumptions, Applications, and Pros and Cons of Evaluation Models

View Fullscreen

How You Can Start TODAY

1. Consider. Sketch out how each program evaluation model “fits” with the program you are evaluating, and with the type of question you are asking. For example, fill a blank table comprised of columns labeled with inputs, activities, outputs, and outcomes to visualize how your program might align with the logic model. Or pilot some appreciative inquiry questions with program participants: Are responses to these questions likely to yield information that will be useful for program evaluation?

2. Begin systematically. Identify a small program or initiative whose implementation or effectiveness has not been evaluated and pick one of the models discussed. Convene a diverse group of stakeholders (eg, faculty, trainees, program coordinators) to discuss feasibility of evaluation and how evaluation data will be utilized. Outline your evaluation plan, then implement.

What You Can Do LONG TERM

1. Familiarize yourself with different program evaluation models. Utilize published resources in medical education and evaluation³ and attend evaluation-oriented workshops/meetings.

2. Get involved in the evaluation community. Take advantage of opportunities with program evaluators at your institution or through your national education organization(s).