|Year : 2017 | Volume
| Issue : 4 | Page : 241-247
A competency-based simulation curriculum for surgical resident trauma resuscitation skills
Matthew L Moorman, Tony R Capizzani, Michelle A Feliciano, Judith C French
Department of General Surgery, Section of Acute Care Surgery, Cleveland Clinic, Cleveland, Ohio, USA
|Date of Web Publication||5-Dec-2017|
Dr. Matthew L Moorman
Division of Trauma and Acute Care Surgery, OhioHealth Grant Medical Center, 111 S Grant Ave, Columbus, OH 43215
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: Evidence-based curricula for nonprocedural simulation training in general surgery are lacking. Residency programs are required to implement simulation training despite this shortcoming. The goal of this project was the development of a simulation curriculum that measurably improves milestone performance and replaces traditional experienced-based training with a competency-based model.
Materials and Methods: SimMan 3G® (Laerdal Medical, Wappingers Falls, NY, USA) was utilized for simulation. Needs assessment targeted trauma and shock resuscitation. Scenario design applied deliberate practice methodology. Learner performance data included items such as identification of shock physiology, resuscitation products used, volume delivered, use of resuscitation end-points, and knowledge of massive transfusion. Characteristics essential for a successful program were tabulated.
Results: Forty-eight residents in postgraduate year (PGY) 2–5 participated representing 100% of the 48 eligible for the training. Senior residents (PGY 4 and 5) demonstrated near universal improvement. Junior residents (PGY 2 and 3) improved in some areas but showed more skill decay between sessions. Overall, milestone performance improved with each training session, and resident feedback was universally positive.
Conclusions: This prototype curriculum improved surgical resident competency in shock resuscitation in a simulated patient care environment. It can be modified to accommodate centers with fewer resources and can be implemented by clinical faculty. The essential characteristics of a successful program are identified.
Keywords: Competency, milestones, resident, resuscitation, simulation, trauma
|How to cite this article:|
Moorman ML, Capizzani TR, Feliciano MA, French JC. A competency-based simulation curriculum for surgical resident trauma resuscitation skills. Int J Crit Illn Inj Sci 2017;7:241-7
|How to cite this URL:|
Moorman ML, Capizzani TR, Feliciano MA, French JC. A competency-based simulation curriculum for surgical resident trauma resuscitation skills. Int J Crit Illn Inj Sci [serial online] 2017 [cited 2019 May 23];7:241-7. Available from: http://www.ijciis.org/text.asp?2017/7/4/241/219943
| Introduction|| |
Simulation in medical education originated in the 17th century. Professional and military aviation have successfully utilized simulation training for decades. Errors in patient care are becoming intolerable to the public, and in some specialties, residents are required to participate in simulation training before the performance of any invasive procedure. A simulation program of some type is now required by the Accreditation Council for Graduate Medical Education, but this requirement remains entirely undefined for general surgery residency.,,
Laparoscopic and endoscopic skills seem to translate from the simulation laboratory to the actual operating room. Data implies that outcomes can be improved. Similar evidence is lacking for aspects of surgery such as clinical decision-making, communication, and situational awareness. Program and facility costs can become prohibitive. How best to utilize simulation-based medical education (SBME) to achieve these residency training milestones remains elusive.,, The aim of this effort was to develop a competency-based SBME curriculum which measurably improves general surgery resident milestone performance during trauma and shock resuscitation. An additional goal was that the curriculum may be implemented by clinical faculty and does not require extraordinary resources for success.
| Materials and Methods|| |
SimMan 3G® (Laerdal Medical, Wappingers Falls, NY, USA) was used for this training. Curriculum design began with a milestone gap analysis. The resident survey identified critical care as a training area needing improvement. Insufficient clinical exposure to trauma evaluations and critical care resuscitations were noted as causes of this weakness. These areas were chosen as our clinical focus. Goals and objectives were developed for both the overall curriculum [Table 1] and for each training day [Table 2]. These objectives are aligned with the curriculum defined by the Surgical Council on Resident Education (SCORE).
An educational model of deliberate practice (DP) combined with SBME was applied. DP entails repetitive training at progressively higher levels of complexity. This is distinct from the historical “see one, do one, teach one” method., Rather than lengthy and complicated scenarios, focused problems were presented to the learner. The intent was for the resident to experience similar pathophysiology presented in multiple forms and at higher levels of complexity as the training progressed.
Scenarios were developed for multifactorial shock following a format similar to Advanced Trauma Life Support (ATLS) standardized patients [Table 2]. Initial scenarios were basic traumatic hemorrhagic. Subsequent training involved shock due to sepsis, burns, and multisystem traumatic injury. The curriculum built on the basic topics of ATLS and continued with more advanced surgical decision-making such as massive transfusion, damage control, and balanced resuscitation as well as interpersonal and communication skills. Also included at increasing level of difficulty were airway emergencies, tension pneumothorax (tPTX), displaced fractures, abdominal compartment syndrome, and acute respiratory distress syndrome.
All postgraduate year (PGY) 2–5 residents attended training once per quarter. Simulation sessions were approximately 15–20 min per scenario followed by a 10–15 min debrief. Two scenarios were done each training day. Four separate training days were designed.
A learner assessment form was created to provide feedback to the residents and capture performance data. Specific metrics for the assessment of effective resuscitation do not exist. Similar to the previous work in trauma simulation, faculty members (Acute Care Surgeons, ATLS Instructors) participated in scenario development and in defining these performance criteria based on ATLS, local, and nationally accepted guidelines.,, Review of our local needs assessment identified airway and pneumothorax management, identification of shock physiology, resuscitation fluid selection, massive transfusion, and end-points of resuscitation as areas of weakness and items easily measured during training.
Several items (e.g., establishing a patent airway) were assessed as either “yes” or “no”, and the time required to accomplish the task was recorded. Other areas were rated on a Likert scale with each level specifically defined and ranging from the ability to recall basic knowledge to the ability to demonstrate mastery. A resuscitation diary recorded the type and quantity of intravenous fluid or blood product given to the patient.
Debriefing occurred immediately after each simulation scenario. An advocacy-inquiry model was used reviewing key decision points in the care of the patient. Scenario objectives were specifically addressed. Residents consented to being videotaped and were able to review these to assess their performance. As part of their overall critique of the training, residents were asked to assess the quality of the debrief by answering questions similar to the objective structured assessment of debriefing.
A performance baseline was created as residents experienced the scenarios for the first time. Performance trends were then measured throughout the year and compared to this baseline. Rates of improvement, retention, and decay of skills over time were compared for each PGY.
Data analysis was done using R software (R Foundation for Statistical Computint, version 3.1.1, Vienna, Austria) with a significance level established at 5%. The outcomes were modeled using generalized linear-mixed modeling techniques, and these models were used to produce expected outcomes by PGY across each simulator session. The significance of the interaction terms was tested using a Chi-squared test based on the Wald statistic. This work was approved as exempt research by the Institutional Review Board.
| Results|| |
Forty-eight trainees completed the curriculum. They were comprised 40 categorical surgical residents (10 each, PGY 2–5) and 8 preliminary surgery residents (PGY 2 and 3). No off-service residents completed the curriculum. This represents a 100% participation rate (48 of 48 available residents).
[Table 3] summarizes performance over the year demonstrating near universal improvement. In several areas, senior residents (PGY 4 and 5) retained their skills while junior residents (PGY 2 and 3) experienced skill decay [Figure 1]. For this curriculum, PGY 2 was thought to represent a baseline level of skill and training. Therefore, performance in PGY 3–5 was compared to PGY 2 for all variables to assess increased skills obtained as higher levels of training. Similarly, performance during training days 2–4 was compared to the 1st day of training which was thought to again represent baseline skills. The odds ratio (OR) of a correct action for each comparison are listed in [Table 4].
PGY had no correlation to the likelihood of promptly confirming a patent airway. While the ORs were better for more senior residents, the differences did not achieve statistical significance. All PGY levels did show improvement over the year. Residents were 14.43 times more likely to appropriately perform this skill at the conclusion of training (P < 0.001).
This result was similar for the identification of a tPTX. While PGY did not correlate with a higher OR of an appropriate diagnosis or response, residents were over 10 times more likely to make the diagnosis at the end of the curriculum (P < 0.001). Subsequent needle decompression was accomplished with an OR improvement from 16 to over 42 (P < 0.001). The improved performance was again maintained by senior residents more so than juniors.
Resident's ability to accurately assess shock physiology improved over the course of the curriculum to an OR of nearly 25 (P < 0.001). Their selection of an appropriate resuscitation fluid, recognition of massive transfusion triggers, and use of accurate end-points of resuscitation improved similarly. Finally, the ratio of packed red blood cells (PRBC) to fresh frozen plasma (FFP) decreased over the year (goal 1:1). Senior residents used a lower, more appropriate, ratio than did their junior counterparts.
Resident critique of the program was universally positive. Evaluation of the program was recorded with each training session. Comments focused on the realism of the simulation, an appreciation of the threat-free learning environment, and their desire for more simulation in the academic year. Trainees universally regarded the training as realistic, educational, and effective. Suggestions for improvement were incorporated. The most profound change was the daily schedule during which residents felt there was too much time spent waiting. Minor adjustment to the “show time” for each resident corrected this problem and was met with positive response. In addition, residents and faculty were nearly unanimous in their feeling that immediate bedside debrief was more effective than formal tape review.
Program evaluation was also sought from simulation faculty and technicians. Review of these data reveals several items thought to be critical to program success. A threat-free training environment was considered paramount. Scenarios were thought best if focused on 2–3 physiologic abnormalities lasting a maximum of 15–20 min. Mannequin moulage did little to improve realism and again can be kept simple. All patient monitors, imaging, and laboratories should be presented to the learner in a manner identical to their clinical environment. Finally, scenarios ran best when simulation technicians had latitude to alter the patient responses and customize the degree of difficulty based on learner performance.
| Discussion|| |
While there is very little which provides validated simulation curricula addressing our needs, several studies provide foundation on which our work is based and will move forward. Our initial efforts in SBME identified that small group and solo learners achieve more effective results than larger sets. This is out of vogue in our current team-based medical culture but is in agreement with recent reports. The concept that SBME correctly simulates live patients and the skills learner is therefore translatable was recently supported by work from the US military. In work similar to ours, trauma evaluations were shown to be accomplished more rapidly and with fewer errors after completion of a mass casualty simulation curriculum.
It appears that our study is one of the first to focus on curriculum development to achieve resident milestone performance improvement in trauma resuscitation skills. Our results demonstrate improved competency over the course of the training year. Nearly, every area achieved statistical significance and as a group, they maintained or continued to improve their skills throughout the year. In several areas, junior residents (PGY 2 and 3) experience some decline in ability, but this was regained with additional training sessions. This was not seen in PGY 4 and 5 residents. These data may be helpful in designing future curricula. They possibly indicate the need for shorter training intervals and additional training in the more junior years. Junior residents begin this curriculum with less overall experience and knowledge which likely explains the prolonged time in mastering the new skills.
Several items are of further interest. Most notable is the fact that the odds of an appropriate action did not correlate well with PGY level. Basic airway management, identification and treatment of a tPTX, and correct identification of shock physiology were handled similarly at all levels of training. Likewise, understanding of massive transfusion triggers and end-points of resuscitation were equal across training years. While this is somewhat counter-intuitive, it is supported by our local needs assessment identifying these as areas of weakness where the residents have an inadequate experience base. The trainees did demonstrate improved understanding of balanced resuscitation strategies at higher PGY level. This may be attributed to their experience in other areas of surgery such as liver transplant where these resuscitation skills are utilized frequently.
The elimination of videotape review was unexpected. This was thought to be a critical part of debriefing at the beginning of the project. Faculty and learners found it to be excessively time-consuming with less yield than immediate bedside discussion of learning objectives. It was eliminated after the first two training sessions. Videotape debrief requires significant skills. As faculty improves in this area, this method of feedback will likely by more satisfying. While we will be working to improve efficiency of our video debrief process, our results parallel others who were also surprised to find it less helpful than expected.
Several limitations of this study are recognized. No control group was utilized in parallel to those completing the curriculum. Performance throughout the curriculum was compared to the first session to evaluate improvement over a baseline. In the course of a calendar year, additional training and information are available to the residents, and this likely affects their performance in our simulation scenarios. This contamination can limit the ability to correlate SBME to performance improvement. In addition, a validated evaluation method is not well defined for this type of simulation training. This limits our ability to definitively prove curriculum effectiveness. This is a common shortcoming in educational research. Finally, the design of the scenarios may not be level-appropriate for general surgery residents. The similar low performance across PGY levels in some areas may be attributable to the topic being simply outside the scope of the curriculum. Certainly, basic airway and pneumothorax cannot be placed in this category. Some of the desired skills may however not be achievable in the general surgery curriculum.
| Conclusions|| |
We have developed a successful curriculum design process. The resultant syllabus is competency-based and founded on the General Surgery milestones, core competencies, and SCORE., Its implementation resulted in statistically significant improvement in performance [Table 4]. Resident's ability to establish an airway, manage a tPTX, and correct shock physiology was greatly improved over the course of the training year. The use of balance resuscitation strategy similarly improved with residents using less crystalloid and a higher ratio of PRBC to FFP as they progressed through the curriculum.
A model for this specific training environment did not previously exist and has subsequently been used to address additional identified training needs in the program. This work provides additional evidence to support the use of SBME in surgical training and gives much-needed direction regarding its implementation.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Buck GH. Development of simulators in medical education. Gesnerus 1991;48(Pt 1):7-28.
Chang CH. Medical simulation is needed in anesthesia training to achieve patient's safety. Korean J Anesthesiol 2013;64:204-11.
Singer BD, Corbridge TC, Schroedl CJ, Wilcox JE, Cohen ER, McGaghie WC, et al.
First-year residents outperform third-year residents after simulation-based education in critical care medicine. Simul Healthc 2013;8:67-71.
Okuda Y, Bond W, Bonfante G, McLaughlin S, Spillane L, Wang E, et al.
National growth in simulation training within emergency medicine residency programs, 2003-2008. Acad Emerg Med 2008;15:1113-6.
Accreditation Council for Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in General Surgery. [Last revised on 2015 Jul 01].
Zevin B, Aggarwal R, Grantcharov TP. Surgical simulation in 2013: Why is it still not the standard in surgical training? J Am Coll Surg 2014;218:294-301.
Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, et al.
Technology-enhanced simulation for health professions education: A systematic review and meta-analysis. JAMA 2011;306:978-88.
Cogbill TH, Ashley SW, Borman KR, Buyske J, Cofer JB, Deladisma AM, et al
. The General Surgery Milestone Project. The Accreditation Council for Graduate Medical Education and The American Board of Surgery; 2014.
Kern DE, Thomas PA, Hughes MT, editors. Curriculum Development for Medical Education: A Six Step Approach. 2nd
ed. Baltimore: Johns Hopkins University Press; 2009.
SCORE Curriculum Outline for General Surgery Residency 2013-2014. The Surgical Council on Resident Education, Inc.; Available from: http://www.surgicalcore.org
. [Last accessed on 2016 Aug 10].
Ericsson KA, Krampe RT, Tesh-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993;100:363-406.
McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med 2011;86:706-11.
American College of Surgeons. Advanced Trauma Life Support Student Course Faculty Manual. 9th
ed. Chicago, IL, USA: American College of Surgeons; 2012.
Holcomb JB, Dumire RD, Crommett JW, Stamateris CE, Fagert MA, Cleveland JA, et al.
Evaluation of trauma team performance using an advanced human patient simulator for resuscitation training. J Trauma 2002;52:1078-85.
Zimmerman JL, editor. Fundamentals of Critical Care Support. 3rd
ed. Mount Prospect, IL, USA: Society of Critical Care Medicine; 2001.
Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65 9 Suppl: S63-7.
Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing with good judgment: Combining rigorous feedback with genuine inquiry. Anesthesiol Clin 2007;25:361-76.
Arora S, Ahmed M, Paige J, Nestel D, Runnacles J, Hull L, et al.
Objective structured assessment of debriefing: Bringing science to the art of debriefing in surgery. Ann Surg 2012;256:982-8.
Lim YS, Steinemann S, Berg BW. Team size impact on assessment of teamwork in simulation-based trauma team training. Hawaii J Med Public Health 2014;73:358-61.
Bonjour TJ, Charny G, Thaxton RE. Trauma resuscitation evaluation times and correlating human patient simulation training differences – What is the standard? Mil Med 2016;181:e1630-6.
Siriratsivawong K, Kang J, Riffenburgh R, Hoang TN. Immersion team training in a realistic environment improves team performance in trauma resuscitation. Surgery 2016;160:586-90.
Brown KM. The Value of Simulation Enhanced Disaster Preparations for Care of Patients with Known of Suspected Ebola Infection. Consortium American College of Surgeons Accredited Education Institutes 9th
Annual Meeting, Chicago, IL, USA; March, 2016. [Abstract].
[Table 1], [Table 2], [Table 3], [Table 4]