Introduction

Obtaining a correct impression of medical residents' and fellows' readiness for clinical practice is important for medical educators. This requires a valid method for the assessment of clinical competence, a topic that has occupied the minds of medical educators for decades. Despite multiple reviews and authoritative proposals,1–6 clinical educators, evaluators, and examiners in practice still have difficulty assessing medical trainees.7 Instruments to assess separate domains of competence in the workplace are scarce8 and instruments that focus on specific tasks usually rely on simulated conditions to ensure standardization. The mini-clinical evaluation exercise9 and other direct-observation instruments10 are valuable feedback instruments but the documentation of their psychometric properties is limited to date.10,11 One problem may be a lack of validity of those assessment approaches because they do not directly consider the important question of “whether a medical trainee is ready for independent practice” but rather focus on the assessment of particular skills. Traditional checklists established for these purposes may not capture the essence of such entrustment decisions as they insufficiently align with this essential construct.12 In clinical settings supervisors decide on a daily basis whether to trust a medical trainee with a specific task and to what extent supervision is needed.13–16 Specific tasks pertain to a given procedure or skill that is to be carried out; general features pertain to task-independent characteristics. These general features can be called facets of competence (FOCs).

The aim of this study was to uncover general features of trainees that facilitate clinicians' trust of trainees with critical clinical tasks. The research question was: What do experienced clinical educators consider as essential facets of competence that determine decisions to entrust a trainee with critical clinical tasks? Our goal was not to answer this question for specific tasks, but for critical clinical tasks in general. The answer to this question is relevant when developing an instrument to evaluate medical graduates' readiness for practice.

Methods

Instrument Development

We developed a draft questionnaire with a preliminary list of FOCs drawn from a review of the literature on selected competency frameworks and other relevant publications. Three competency frameworks were analyzed, that is, the Canadian CanMEDS framework,20 the Dutch “Blueprint of Objectives for Medical Schools,”21 and the competency framework of the General Medical Council in the United Kingdom.22 A qualitative analysis of the FOCs in the different frameworks resulted in an initial list with the relevant FOCs.

The first draft of the relevant FOCs was completed through a literature review regarding important FOCs for clinicians. The literature yielded 3 articles,13,23,24 of which two13,23 used a holistic approach to the evaluation of medical graduates' competences. Ginsburg et al23 described characteristics of trainees that influence assessments by supervisors. Sterkenburg et al13 focused on factors that guide decisions of supervisors to trust residents with critical clinical tasks. The third article by Kearney24 described the results of a Delphi study exploring the features of professionalism.

We compared these 6 sources (the 3 frameworks and the 3 articles) and found many similarities in content (table 1). We also found differences, mainly in the way the descriptions were ordered and labeled, and in their level of detail. From these sources, we constructed a list of 24 FOCs that met the following criteria: content correspondence with the original source, observable and assessable level of detail, and applicability to medical graduates.

table 1 Final List of Facets of Competence With References

View Fullscreen

table 1 Final List of Facets of Competence With References Continued

View Fullscreen

Each of the resulting FOCs was scored on 2 significant aspects for valid competence descriptions: (1) clarity and completeness of formulation; and (2) importance for the entrustment of critical clinical tasks to starting residents. For each FOC a 7-point Likert scale25 was developed that pertained to the following statements: (1) The description of this FOC is complete and clear. (2) This FOC is important for the entrustment of critical clinical tasks to beginning residents. Only the 2 endpoints of the scale were labelled: 1 (strongly disagree) and 7 (strongly agree). Respondents were asked to substantiate their answers with comments and to propose improvements for the FOC descriptions. At the end of the questionnaire, respondents were asked to add any new FOCs that they felt were missing.

Results

Eighteen physician educators from 8 different hospitals and 11 disciplines participated in this study. In the first round, 14 participated in the Delphi study (58%), 13 of whom were men. The average age was 59 years (range, 52–66 years). In the second round, 15 experts responded (63%), which included 13 men. The average age was 59 years (range, 49–66 years). The distribution in sex and age in the sample reflects the distribution in the population of physician educators in the Netherlands. Responders in both rounds represented a wide range of surgical and nonsurgical disciplines (anesthesiology, cardiology, general practice, geriatrics, gynecology, internal medicine, neurology, oncology, pediatrics, rheumatology, and surgery). Eleven experts participated in both rounds, 3 in the first round only and 4 in the second round only.

table 2 shows the expert ratings on “comprehensiveness and clarity” for the descriptions of the FOCs in both rounds of the Delphi study. In the first round means ranged from 4.77 to 6.46 and standard deviations ranged from 0.78 to 2.07. The results of the second round were comparable, with means between 4.82 and 6.27 and standard deviations between 1.03 and 2.23. For the first and the second round, the level of agreement among the experts was medium or high for 83% and 76% of the FOCs, respectively. To test the differences in ratings on comprehensiveness and clarity of the formulations between round 1 and 2, we used dependent t tests. This difference was statistically significant for only 1 FOC (“active listening to patients”), based on a 2-tailed t test (t₁₀  = 1.5, P  =  .03, r  =  .64). The mean rating for this FOC in the second round was higher than in the first.

table 2 Expert Ratings for “Completeness and Clearness” of Facets of Competence in Delphi Study^a

View Fullscreen

We also asked the panel members to indicate on a 7-point Likert scale the importance of the FOC for entrustment decisions. In the first round these means ranged from 6.50 to 7.00, and in the second round, from 5.45 to 6.90. The level of agreement was high for 92% of the FOCs in the first round and for 100% of the FOCs in the second round (table 3). In both rounds and for both the judgments of “comprehensiveness and clarity” and the judgments of “importance,” all ratings tended toward the “strongly agree” side of the scale, such that in all cases the skewness was negative. After round 2, minor changes were recommended regarding the description of 11 FOCs to complete the final list of 25 FOCs (table 1).

table 3 Expert Ratings for “Importance for Entrustment Decisions” of Facets of Competence in Delphi Study^a

View Fullscreen

Discussion

This study exemplifies which facets of the general competence of trainees seem to inform the decisions by clinical supervisors to trust them with critical tasks. It is important to map these facets, in light of current approaches to assessment in competency-based medical education.31–34

We made a preliminary list of FOCs, based on competency frameworks and the literature reflecting an international perspective.13,20–24 After a Delphi study with 2 rounds, we constructed a list with 25 FOCs. The most important finding was that the experts agreed about the formulation of these FOCs. They strongly agreed that these FOCs are important or very important for the entrustment of critical clinical tasks to starting residents.

Earlier studies about entrustment decisions have shown that these decision processes are complex and relate to several factors.13,14 These factors include the characteristics and achieved level of competence of the trainees. In the current study we clarified which FOCs of the trainees appear relevant for entrustment decisions.

Our study is limited by the small number of participants. Further evaluation of the identified FOCs in other countries and different educational climates will be important to enhance generalizability of these findings and inform understanding of entrustability.35,36 The validity of our findings could be further supported by evaluating actual entrustment decisions and their relationships with trainees' FOCs. Finally, our population was selective and not representative of clinical supervisors in general. Participants' views were informed by their academic background, which could have led to the listing of certain FOCs, although we do not have specific hypotheses in mind to support this concern.

Conclusions

We found consensus support by experts for a list of 25 important FOCs for the entrustment of critical tasks to medical trainees. Our results identified general features of medical graduates that enable supervisors to entrust them with critical clinical tasks. The findings are useful for the development of a valid method for assessing medical graduates' readiness for clinical practice. The findings may also be useful for frame-of-reference training for clinicians who must regularly make entrustment decisions.6