How can we ensure our medical students are prepared properly for the United States Medical Licensing Examination (USMLE) and our graduate students are ready to earn their PhDs and become leaders in biomedical research?
The answer: novel ways of assessment.
UME: Predicting the Future
The School of Medicine has had an integrated curriculum for years. This means that our students no longer study subjects in isolation. Learning is organ system-based, and this is how our students progress through the curriculum. Within each system, we assess performance in multiple ways. The weekend formative and summative assessments are main contributors to a student’s grade.
For the past two years, we have been reviewing class performance and determining how it correlates with board scores. James Martindale, PhD, Assistant Professor and Director of Test Development, oversees all SOM testing and is focusing on how overall system grades and weekend formative and summative assessment results relate to Step 1 performance (the application of important concepts of the sciences basic to the practice of medicine). He has found strong correlations in the data and is building a model that can determine how much of the variation in assessment performance can be predictive. While we are not teaching to the test, we do need to know if we are preparing our students for their future careers in medicine — including licensure. Later, we will review Step 2 (clinical knowledge and clinical skills) and Step 3 (taken after medical school graduation) data, to see if similar models can be built.
The impetus for this work is the need to identify the few students who are predicted to perform poorly on Step 1 while they still have time to remediate and enhance their study skills. This will be one additional tool to support our students.
BIMS: Say it out loud!
Amy Bouton, PhD, Associate Dean for Graduate and Medical Scientist Programs, and her colleagues who teach the first-year core course for PhD candidates in the Biomedical Sciences (BIMS) Graduate Program want our graduate students to speak up. In fact, it’s an expectation.
Every first-year student in the BIMS Graduate Program takes an oral examination as the culminating assessment tool for the core course. This exam marks the beginning of their transition from a focus on grades and formal classes to a focus on self-initiated study, research, and professional development.
The examiners ask questions that cover basic information to ascertain that students understand the outline approaches and the purposes and goals of assigned papers. Students must interpret the results to demonstrate that they can make connections between the paper and topics in the course that apply to research, and show that they can make higher order reasoning and intellectual jumps.
We have found this method to be an excellent way to assess our graduate students. It allows us to determine whether they have acquired the basic skills emphasized in the core course and evaluates their ability to problem solve. It gives students an experience similar to what they will face later in their training when they must pass their advancement to PhD candidacy exam. A student who performs poorly on the oral exam is encouraged to work with the mentor to develop a plan for addressing the identified deficiencies.
A tremendous amount of preparation goes into administering these exams, which take place over a period of between 1-2 days and involve nearly 30 faculty members. These assessments provide an important means for early intervention for those students who have not yet developed the skills and knowledge needed to become a strong researcher and scientist.
We realize that English is not the primary language for some of our students. Our data indicate that this has not been a barrier to success, yet we continuously are monitoring for any outlying performance indicators. Since we have been administering this oral exam for only two years, it is too early to determine outcomes with respect to the impact on students’ advancement to candidacy exams.
Thank You, Faculty!
When I see the dedication of our faculty in ensuring we are continuously improving ways in assessing our students — driven by mission and goals — I am in awe. Thank you to Jim, Amy, and all the faculty whose tireless effort goes into student assessment.
R.J. Canterbury, MD, MS, DLFAPA
Wilford W. Spradlin Professor
Senior Associate Dean for Education