UC College of Medicine Offers a Refresher Session for New Residents

Two weeks ago, 118 brand-new physicians arrived on the University of Cincinnati campus. Fresh out of medical school, these residents committed the next three to seven years of their education, and life, to training at UC Medical Center, West Chester Hospital, Cincinnati Children's Hospital Medical Center and other Cincinnati-area hospitals.

But before the newcomers began their high-pressure internship year, educators at the College of Medicine already collected a wealth of data on their skills and knowledge. Thanks to a system developed at UC, residency programs are able start their education process with a complete picture of their residents' abilities, skills and performance and use that data to build a better physician. 

The Baseline Resident Assessment of Clinical Knowledge, or "BRACK," was first piloted at UC in June 2011, and received approval from UC's Institutional Review Board in 2012. BRACK's mission is to gather data on incoming residents' knowledge to inform patient care and safety on day one of clinical service. 

During BRACK evaluations, residents spend a half-day in eight mini clinical sessions with standardized patients and physician evaluators. In mock exam rooms in the College of Medicine's Simulation Center, they are scored on their ability to render a differential diagnosis and nearly every other aspect of the patient encounter. 

During an orientation, the residents learn what types of cases they will review and what skills they are being tested on while scenarios of patient-doctor encounters are displayed and discussed in detail. This will help residents understand how their interactions should flow with mock patients and physician evaluators. 

"How many of you have not done any clinical medicine during the past month?" asked Paul Wojciechowski, MD, assistant professor of anesthesiology, during an orientation session with a group of 16 residents. "What about after two months? What about after three months? Or four months?"

Most hands in the room went up after the first question and remained. 

"My point is you have been out of this for a little while, so what we want to do is to give you a little of a refresher," explained Wojciechowski.  Part of the whole experience is to give you some level of a reminder of, 'Hey I learned this and I can do this.'"

Testing Five Key Elements in Residents

Amy Bunger, PhD, assistant professor in the Department of Medical Education at the College of Medicine and co-principal BRACK investigator with Wojciechowski, says each resident is assessed on five different elements.

"So the first is professionalism. Here we are looking to insure that they introduce themselves and they speak to a patient in terms that they can understand, and they avoid medical jargon while listening to the patient and treating the patient with courtesy and respect; we also want to make sure they do basic safe-care practices such as hand washing," says Bunger. 

"Secondly, they perform a history and a physical; the history would include the family history, the social history and the risk history around drugs, alcohol and elicit activities," she adds. "Third, they do a full physical exam. They have to come up with what we call a five-item differential. Those are the five most common things that can be wrong with this patient. There is one correct answer, but a doctor should always have a range of things in mind when they see particular risk factors. 

Bunger says the resident must be able to order appropriate tests based on what they think the problem or differential is and be able to interpret those tests.  

"It is one thing to order an arterial blood gas test, but it's another to get the results back and know if it looks good or bad," she says. "Sometimes we might have them read an EKG and see that they actually can interpret correctly," she says. 

"The last thing we ask them to do is communicate a treatment plan," Bunger says. "They let the patient know what the doctor is worried about, what he or she thinks is wrong with the patient and what is next for the patient along with what that follow-up will look like. Do I think you need urgent care or do I need to admit you to the hospital?"

After 10 minutes of patient interaction, residents receive three minutes of feedback from their patient, an actor who is trained by Bunger and Wojciechowski, MD, and their physician preceptor on what BRACK is attempting to measure in residents. The immediacy of that feedback is key, since residents arrive at UC with a variety of educational experiences and amount of time since their last clinical rotation.

"Ideally, the sessions knock some of the rust off their clinical practice," says Bunger. "They've realized there's a better, clearer way to explain something to patients, and they've practiced it eight times in a row. That immediate reinforcement helps transfer the knowledge from short- to long-term memory. It's solidifying and concretizing learning—not just measuring, but improving performance."

Within 72 hours of the final BRACK evaluation, Bunger and Wojciechowski send residency directors at UC Health their reports, including quantified data and qualitative and case-specific feedback on each new physician. With those reports, Bunger says program directors have baseline information that can guide the next two years of residents' education. Evaluations can even uncover assets that benefit the entire program.

Program directors may alter a resident's rotation if the test shows a need for additional guidance in some areas.

One of the bigger rotations for interns is the Veterans Affairs Intensive Care Unit (VAICU) because there are no senior residents there, and they have a fellow who offers good supervision, but it is a high intensity rotation, says Lauren Ashbrook, an assistant professor in the Department of Internal Medicine, and UC Health physician who was a former chief resident during her own residency at UC. 

"For example, if someone did really poorly on this, and they were slated to go to the VAICU, that might make us stop and think about rearranging them and getting them on an elective or on a ward team where they can have a lot more supervision. It's very helpful," says Ashbrook.  "It's really good for your bottom residents. Everyone above the bottom quartile usually does fine. Not everyone in the bottom quartile does poorly. Sometimes they are just nervous, but it helps us get an eye on those residents. We want everyone to succeed."

Bunger says residents have also generally given positive feedback about BRACK in surveys.

"I enjoyed being able to take responsibility and provide acute management," wrote one resident. "It gave me confidence that I am well prepared in the basic qualities a good resident needs. Good communication, basic medical knowledge; I just need to work on specifics."

Another wrote, "Really liked the feedback from the attending physicians. Appreciated practicing my knowledge and interpreting labs and studies. Overall, a great learning experience.  Also, liked how we had a variety of cases."

BRACK on the National Stage

UC's BRACK program was modeled after programs at the University of Michigan and Vanderbilt University but remains rare in resident education. Increasingly, the method, and the skills it measures, is being recognized by medical educators. 

In 2013, Bunger and Wojciechowski presented BRACK at the International Forum on Quality and Safety in Healthcare and at the request of other regional residency programs expanded it to Christ Hospital in Cincinnati and Clinton Memorial Hospital in Wilmington, Ohio. In 2014, they presented it a standing-room only audience at the Association for Hospital Medical Education conference in Charleston. 

Also in 2015, the Association of American Medical Colleges (AAMC) released their core Entrustable Professional Activities for physicians. Of those, 11 of 13 are already being measured by the BRACK program. 

"We're thrilled that BRACK is one of the first direct tests of these measures that exists," says Bunger. 

Moving forward, the team will continue to refine the BRACK system, expanding on patient safety measurements and continuing to emphasize patient communication and difficult conversations. 

 "We want to tell residents from the first day, 'The interaction you have with the patient matters.' Yes, you have to have the right answer. Yes, you have to be scientifically accurate. But the patient also has to have a good experience. They'll be monitored on that measure throughout the course of their residency and throughout the course of their careers," Bunger says.