Abstract
Background
Graduate medical education is based on an on-the-job training model in which residents provide clinical care under supervision. The traditional method is to offer residents graduated levels of responsibility that will prepare them for independent practice. However, if progressive independence from supervision exceeds residents' progressive professional development, patient outcomes may be at risk. Leaders in graduate medical education have called for “optimal” supervision, yet few studies have conceptually defined what optimal supervision means and whether optimal care is theoretically compatible with progressive independence, nor have they developed a test for progressive independence.
Objective
This research develops theory and analytic models as part of the Resident Supervision Index to quantify the intensity of supervision.
Methods
We introduce an explicit set of assumptions for an ideal patient-centered theory of optimal supervision of resident-provided care. A critical assumption is that informed attending staff will use available resources to optimize patient outcomes first and foremost, with residents gaining clinical competencies by contributing to optimal care. Next, we derive mathematically the consequences of these assumptions as theoretical results.
Results
Under optimal supervision, (1) patient outcome is expected to be no worse than if residents were not involved, (2) supervisors will avoid undersupervising residents (when patients are at increased risk for poor outcomes) or oversupervising residents (when residents miss clinical opportunities to practice care), (3) optimal patient outcomes will be compatible with progressive independence, (4) progressive development can be inferred from progressive independence whenever residents contribute to patient care, and (5) analytic models that test for progressive independence will emphasize adjusting the association between length of graduate medical education training and supervision for case complexity and clinic workload, but not patient health outcomes.
Conclusion
An explicit theoretical framework is critical to measure scientifically progressive independence from supervision using graduate medical education data.
Background
In this article, we introduce the patient-centered theory of optimal supervision. The theory is designed to explain supervision of residents engaged in outpatient care for the purpose of testing for progressive independence in graduate medical education (GME). The theory is part of the Resident Supervision Index (RSI), designed to measure and analyze the intensity of resident supervision in outpatient care settings.
Development of a theoretical framework is critically important to resident supervision. Specifically, theory makes explicit all of the underlying assumptions necessary for statistical inferences from data analyses, provides the basis to define optimal supervision, and offers frameworks to develop guidelines that can help teaching staff balance patient care needs with resident education demands. Theory is used to derive hypotheses, develop analytic models to test those hypotheses, and guide formulation of intensity scores1 and collection of GME data.2
Progressive Independence
Graduate medical education is based on an apprenticeship model of on-the-job training in which resident trainees provide patient care under the supervision of attending physicians. In teaching settings, attending physicians are faced with competing demands. First and foremost, they must ensure that patients receive safe and effective care, while at the same time providing clinical opportunities that help residents develop professionally into self-regulating, autonomous, independently practicing physicians, which is the ultimate goal of GME.3–7
The traditional method for achieving GME goals is to provide residents with progressive independence from supervision. That is, residents are offered graduated levels of responsibility with increasing responsibilities for more complex cases as they progress in their GME training.8–10 In practice, progressive independence from supervision is largely based on the individual's level of training, with responsibilities increasing with each postgraduate year of training completed. Thus, by virtue of promotion from one postgraduate year level to the next, some residents may become more autonomous from supervision than what may be justified based on their clinical competence. Undersupervised residents, whose progressive independence from supervision grows at a rate faster than their progressive professional development in clinical competencies, may put patients at increased risk of poor outcomes. On the other hand, oversupervised care may fail to offer residents the clinical opportunities they need to develop professionally, leaving them unprepared at the end of their training program to enter independent practice. To reconcile patient care with teaching goals, some GME scholars7,11,12 and the Institute of Medicine13 have proposed that attending physicians must offer an “optimal” level of supervision. However, few studies have defined what optimal means, indicate whether optimal supervision is achievable and compatible with progressive independence, or provide insight regarding how to test for the presence of optimal supervision using GME data. Understanding supervision is essential for teaching facilities to effectively function with increasing patient workloads, growing education demands, and dwindling resources.
Resident Supervision Index
The 4-part RSI method is designed to measure and explain the intensity of resident supervision during clinical encounters with outpatients. Designed to coordinate as a single method, these parts include the following: (1) RSI Inventory, a content-valid feasible and reliable instrument administered to attending staff and residents to collect supervision data describing outpatient care encounters2; (2) RSI scores calculated from RSI Inventory responses to quantify the intensity of resident supervision1; (3) RSI theory used to derive testable hypotheses; and, finally, (4) RSI analytic models that drive the analyses of RSI scores to empirically test theory-driven hypotheses.
Theory of GME Supervision
To test for progressive independence, this article describes the RSI theoretical framework, or patient-centered theory of optimal supervision. The theory was designed to explain how attending faculty supervises the time that residents engage in patient care. The theory was not designed to explain information-gathering oversight, didactic training, resident evaluation, or faculty research activities and administrative duties. These elements are left for future research and development.
The theory was represented as a mathematical framework (appendix) to (1) define optimal supervision and (2) calculate scores that measure the intensity of resident supervision during patient care. To test for progressive independence, we also (3) derive specific hypotheses, (4) develop reduced-form models for analyses, and (5) identify conditions when an inverse association between length of GME training and intensity scores (progressive independence) can be used to infer that the clinical performance of a resident has progressed (progressive development).
Methods
We begin with a set of explicit theoretical assumptions about patient-centered optimum supervision of resident-provided care. We then derive the consequences of those assumptions as theoretical results.
The model describes supervision as resource allocation of scarce clinic resources, including residents, so as to maximize the collective health outcomes of all clinic patients seen in the teaching clinic. The model narrowly focuses on the tension facing supervisors between putting time into direct patient care or, alternatively, into supervising residents, who in turn provide patient care. For the sake of simplicity, and without loss of generality, we ignore other activities that can divert staff time away from resident supervision such as research, administration, and other teaching functions. Inspiration came from industrial economics, where market firms produce products by allocating scarce human resources among competing activities to maximize production. Analogously, industrial firms are teaching clinics, products are patient outcomes, human resources are attending staff and residents, and competing activities are staff-provided patient care and staff supervision of resident-provided care. Assumptions are the following:
(1) Optimal Supervision
Attending staff will supervise residents and engage in direct patient care so as to maximize, first and foremost, the mean health outcome for all clinic patients.
(2) Informed Decision
Attending staff makes decisions concerning direct supervision of patient care after becoming informed about the patient's case and the resident's performance. This assumption follows after attending staff has engaged in resident oversight and collected information necessary to make optimal decisions.
(3) Patient Assignment
Patients who present to the teaching clinic are assigned to a resident for resident-provided care or are retained by professional staff for staff-provided care.
(3a) The number of patients assigned to each resident is predetermined based on the complexity of patient cases, as well as clinic protocols, GME program requirements, and government regulations.
(4) Professional Time
(4a) Attending staff will provide care directly to patients or supervise residents engaged in providing patient care.
(4b) The efficacy of residents to provide professional services and improve patient outcomes will depend in part on the following:
(4b1) The resident's clinical experience (eg, competencies, judgment, prior training, and outcomes achieved on prior cases) and
(4b2) The clinical complexities of assigned cases.
(4c) Attending staff directly supervises none, some, or all of the time that residents spend engaged in providing patient care.
(5) Patient Outcomes
Attending staff and residents prioritize their time to make the largest possible improvement in patient outcomes with each additional minute from the following:
(5a) Staff professional time and
(5b) Resident professional time. (However, these contributions will eventually plateau when any further time will not improve patient outcomes and, if continued, may even become harmful.)
(5c) Quality. Professional contributions to outcome will depend on the quality of patient care and effectiveness of resident supervision, which in turn depend on patients, residents, attending physicians, the teaching clinic, and the GME program characteristics.
(6) Resident Learning
Residents learn clinical competencies by engaging in optimally supervised care.
Theoretical Results
The following subsections represent the theoretical consequences of the assumptions, derived mathematically in the appendix.
Optimal Supervision Identified
Optimal supervision occurs when the attending physician supervises residents so that the contribution of an additional minute of direct patient care to the outcome of staff-assigned patients, prioritized based on patient clinical needs, equals the contribution of an additional minute of resident supervision to the outcome of resident-assigned patients, prioritized based on resident supervision needs. That is, supervisors will allocate time among patients and between patient care and supervision so as to achieve the greatest effect on patient outcome.
Theoretical Supervision Intensity Score
Theoretical scores, represented symbolically by RSI, equal the proportion of time when the resident was providing patient care that was also being directly supervised by attending staff. Scores vary between 0 (no supervision) and 1 (the entire time residents were engaged in patient care is directly supervised by staff). The theoretical score is equivalent to the summary patient care encounter score [RSIcare], as operationally defined elsewhere.1
Hypotheses Derived
Progressive Independence
Residents with longer length of GME training will be assigned to more patient care responsibilities.
Complexity Effect
Residents facing more complex cases will face more intensive supervision. The complexity effect comprises 2 components. The efficacy component describes residents who need more supervision to handle more complex clinical cases. The assignment component describes residents who are allowed more time to manage more complex cases. The complexity-effect hypothesis follows whenever the assignment component is negligible.
Workload Effect
Residents providing care in clinics where staff faces greater workloads will be assigned to more patient care responsibilities and thus face less intensive supervision.
Patient Outcomes Effect
Patient outcomes and supervision intensity will covary with clinic workload so that better patient outcomes will be associated with more intensive supervision across clinics with less clinic workload.
Statistical Models
To test for progressive independence under optimal supervision, associations between supervision intensity and length of GME training should be adjusted to control for variation in (covariates) patient case complexity, clinic workload, and characteristics of the patient, resident, attending staff, clinic, and training program that reflect the efficacy of care and resident supervision. The RSI scores are bounded between 0 and 1 and are expected to be heteroskedastic, bimodal, and highly skewed. Thus, associations between scores and covariates are calculated in 2 parts, beginning with logistic regressions to predict whether residents were directly supervised (RSI > 0) or not supervised (RSI = 0), followed by linear regressions using a log linking function (ln[RSI / (1-RSI)]) to assess supervision intensity for those encounters when direct supervision has occurred (RSI > 0). Applied in health economics,14,15 the first part computes the likelihood that any supervision occurs during the encounter, and, if supervised, the second part computes the probability that the resident is supervised at any given encounter moment. By exponentiating regression coefficients, effect sizes for both parts are calculated as odds ratios.
Statistical Inference
Resident professional development can be inferred from tests for progressive independence whenever attending staff are patient-centered optimal supervisors over residents who contribute positively to patient care. Motivation for the latter condition is that residents must be engaged in patient care before a theory based on resource allocation can meaningfully explain supervision behavior.
Discussion
This article presents an explicit theory of patient-centered optimal supervision as part of the RSI method. The RSI is designed to explain the intensity by which residents are supervised during patient care. The theory is called patient centered because supervisors are assumed to allocate scarce clinic resources, including residents, among all patients being seen in the clinic to maximize, first and foremost, the health outcomes of those clinic patients. Supervision is thus described in theory as resource allocation to achieve a single objective (patient outcomes) rather than as trade-offs between 2 objectives (resident education outcomes and patient outcomes).
Several theoretical consequences were derived. First, patient outcomes are expected to be no worse than if residents were uninvolved in care because residents could always be removed from care if removal enhances (maximizes) patient outcomes. Second, residents are expected to be neither undersupervised (patient outcomes suffer) nor oversupervised (residents miss clinical opportunities that would not have harmed patient outcomes) if staff uses resident time and other scarce clinic resources efficiently to optimize patient outcomes. Third, optimal health outcomes for patients and progressive independence for residents are theoretically compatible goals. Progressive professional development can be inferred from an empirical finding of progressive supervision independence whenever an informed attending staff maximizes patient outcomes by engaging residents, who in turn contribute to patient care. Fourth, to test for progressive independence, the association between length of GME training (predictor) and resident supervision (dependent variable) must be computed after “controlling” for other covariates before inferring progressive professional development in resident competencies. These covariates include case complexity, clinic workload, and other resident, staff, patient, and clinic characteristics. These covariates, however, do not theoretically include the patient outcomes of resident supervision. This is important because patient health outcomes are subject to measurement errors16 and endogenity biases.17
Not having to empirically measure outcomes from a theory built on outcome maximization is not without precedent. For example, economists explain product demand for utility-maximizing consumers without actually measuring utility. Alternatively, a dual-objective theory may explain supervision as maximizing 2 simultaneous objectives of patient outcomes and resident education outcomes. Under dual-objectives, staff may face conflicting goals between furthering education opportunities for residents and enhancing care for patients. Understanding how supervisors trade off patient and education outcomes is necessary before professional development could be inferred from a test for progressive independence. For example, it would be incorrect to infer to professional development any increases in clinical responsibility that faculty may have assigned to residents at the expense of poorer outcomes to patients. The patient-centered theory does not present conflicts from patient-resident trade-offs because staff is always assumed to maximize only the single objective of patient outcomes. Thus, variation in patient outcomes results from changing circumstances and from not changing preferences or values for patient care versus education goals.
Ultimately, patient-centered supervision is a behavioral assumption that must be subjected to empirical testing in GME settings. For example, the theory predicts that achieved patient outcomes and actual supervision intensity will be inversely related whenever greater workload demands pressure staff to transfer more care responsibility to residents. The transfer of responsibility is consistent with the empirical literature in which associations between supervision4,18–22 and workload23,24 relative to patient outcomes have been observed. Our single-objective theoretical framework interprets these associations as manifestations of scarce medical resources rather than deliberate trade-offs between patient needs and trainee education goals. The RSI framework is consistent with mandates from GME accrediting bodies, who recognize the relationship between supervision and workload by limiting the number of trainees who can be supervised at any single time by an attending physician. The use of physician time to measure supervision intensity is also supported by existing literature in which attending physicians have been observed to spend more time overseeing the care of less experienced residents25–27 and changing or validating resident-provided treatment plans,28 patient history, diagnostic testing, diagnosis, follow-up care, and medication management.29,30 Supervision time has also been observed to be associated with greater compliance with process-of-care guidelines in emergency departments,18 more correct readings of computed tomography images,22 lower death and complication rates from surgery,19,31 decreased medical errors, and fewer malpractice claims.4 Finally, a critical assumption underlying patient-centered supervision is that residents contribute to clinical workload. Resident contributions to workload have been observed in the literature for surgical32 and medical33 residents. More research is needed to understand how supervision relates to patient health, trainee education outcomes, and clinic workload. For example, our mathematical theory predicts that the strength of associations between supervision intensity and patient outcomes will diminish whenever more experienced residents treat less complex cases. Future studies should consider how the strength of these intensity-outcome associations may vary by clinic, resident, patient, and attending physician characteristics as a means to test a patient-centered orientation among supervisors of resident-provided care.
The patient-centered framework may also serve as a foundation for ethical guidelines to help faculty balance their teaching and clinical care duties. While optimal supervision has been characterized as balancing patient care quality and safety against resident education,12,34 how faculty balance these complex trade-offs is not fully understood. Our theoretical framework also builds on the Institute of Medicine's report that calls on GME to expand the role of supervision to encompass “optimized” patient care and education goals and that recommends “… improvements in the content of residents' work, a patient workload and intensity appropriate to learning, and more frequent consultations between residents and their supervisors.”13 (p. 19) Finally, our theory is consistent with the call for more GME research by the Department of Veterans Affairs Blue Ribbon Panel on VA-Medical School Affiliations.35
The approach used in this study has several limitations. First, the theory focuses on direct supervision over patient care and ignores other complexities with staffing time and faculty demands.36,37 Second, it also focuses only on intensity of supervision and not on the content, appropriateness, timeliness, or quality of supervision. Third, the theory does not consider other activities competing for staff time such as research, service, other teaching, and administrative duties. Fourth, it does not describe how patient cases are assigned to residents, how clinic administration negotiates patient volume, or how staff time is determined.
Conclusions
Patient-centered theory of optimal supervision provides the theoretical basis for a test for progressive independence and lays the foundation to understand associations among supervision intensity, professional development, clinic workload, case complexity, and patient outcomes. Good theory is critical if scientific research is to help properly inform GME policy.
Author Notes
T. Michael Kashner, PhD, JD, is Professor and Associate Chair for Translational Research, Department of Medicine, Loma Linda University Medical School and Director of the Center for Advanced Statistics in Education at the Jerry L. Pettis Memorial VA Medical Center, Loma Linda, CA, Professor of Psychiatry at the University of Texas Southwestern Medical Center at Dallas, TX, and Health Specialist with the Office of Academic Affiliations, Department of Veterans Affairs, Washington, DC; John M. Byrne, DO, is Associate Chief of Staff for Education and Co-Director of the Center for Advanced Statistics in Education at the Jerry L. Pettis Memorial VA Medical Center, Loma Linda, CA, and Assistant Professor of Medicine, Loma Linda University Medical School, Loma Linda, CA; Steven S. Henley, MS, is President, Martingale Research Corporation, Plano, TX; Richard M. Golden, PhD, is Professor of Cognitive Science and Engineering, School of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, TX; David C. Aron, MD, MS, is Associate Chief of Staff for Education, VA Senior Scholar, Louis Stokes Cleveland DVA Medical Center, Cleveland, OH, and Professor of Medicine & Epidemiology & Biostatistics, School of Medicine, and Professor of Organizational Behavior at the Weatherhead School of Management, Case Western Reserve University, Cleveland, OH; Grant W. Cannon, MD, is the Associate Chief of Staff for Academic Affiliations, George E. Wahlen VA Medical Center, Salt Lake City, UT and Professor and Thomas E. and Rebecca D. Jeremy Presidential and Endowed Chair for Arthritis Research, School of Medicine, University of Utah, Salt Lake City, UT; Barbara K. Chang, MD, MA, is Director of Medical and Dental Education, Office of Academic Affiliations, Department of Veterans Affairs, Washington, DC, and Professor of Medicine (emeritus), University of New Mexico School of Medicine, Albuquerque, NM; Stuart C. Gilman, MD, MPH, is Director, Advanced Fellowships and Professional Development, Office of Academic Affiliations, Department of Veterans Affairs, Washington DC and Clinical Professor of Health Sciences, University of California Irvine School of Medicine, Irvine, CA; Gloria J. Holland, PhD, is Special Assistant for Policy and Planning, Office of Academic Affiliations, Veterans Health Administration, Department of Veterans Affairs, Washington, DC; Catherine P. Kaminetzky, MD, MPH, is Associate Chief of Staff for Education, Department of Veterans Affairs Medical Center, Durham, NC, and Assistant Professor, Department of Medicine, Duke University School of Medicine, Durham, NC; Sheri A. Keitz, MD, PhD, is Chief, Medical Service, Miami VA Healthcare System, and Professor of Medicine and Associate Dean, Miller School of Medicine, University of Miami Medical School, Miami, FL; Elaine A. Muchmore, MD, is Associate Chief of Staff for Education at the VA Medical Center in San Diego, CA, and Professor of Clinical Medicine and Vice Chair for Education, Department of Medicine, School of Medicine, University of California at San Diego, CA; Tetyana K. Kashner, MD, is a resident, Department of Obstetrics and Gynecology, Pennsylvania State University Milton S. Hershey Medical Center, Hershey, PA; Annie B. Wicker, BS, is a Health Science Specialist, Office of Academic Affiliations and Data Coordinator for Center for Advanced Statistics in Education, Jerry L. Pettis Memorial VA Medical Center, Loma Linda, CA.
This study was funded in part by grant SHP 08-164 from the Department of Veterans Affairs' Health Services Research and Development Service (Dr T. M. Kashner). Development of the statistical methods was supported in part by grant R44CA139607 from the Small Business Innovation Research program of the National Cancer Institute and by grant R43AA013670 from the National Institute on Alcohol Abuse and Alcoholism (Mr Henley). All statements and descriptions expressed herein do not necessarily reflect the opinions or positions of the Department of Veterans Affairs or the National Institutes of Health of the Department of Health and Human Services.
