Email this page- About the School
- Departments & Faculty
- Alumni and Friends
- Health Science Libraries and Technology
Dissecting the Morse Code of Medical School Rankings
Dr. David Guzick, M.D., Ph.D.
April 28, 2009
Last week, U.S. News and World Report published its 2009 rankings of graduate education programs, including medical schools. In their “research” listing, we improved our ranking by 5, from a tie for 36th last year to a tie for 31st. In their “primary Care” listing, we improved our ranking by 4, from 19th last year to 15th this year.
In February, the Blue Ridge Institute for Medical Research released rankings of medical schools according to NIH funding (http://www.brimr.org/NIH_Awards/NIH_Awards.htm) for the last three federal fiscal years. From a ranking of 30th in 2006, we rose to a rank of 26th in 2007 and to the 24th position in 2008.
We should pause and take pride in this improvement in our rankings—improvement in NIH ranking is especially remarkable in view of the financial constraints that have limited recruitment of scientists over the past few years. I like to think that growth in our NIH funding at a time of flat funding nationally is due to the well-placed investments in faculty and scientific cores we have made over the years, and to the collegial spirit of our faculty to collaborate in multidisciplinary research. This has led to large Center and Program Project grants, in addition to the mainstay of individual research awards.
Robert Morse is the man responsible for developing the methodology for ranking medical schools and other graduate schools in the U.S. News and World Report. As we shall see, when we analyze the “Morse code” of medical school rankings, NIH funding is extremely important, both by itself and as it influences peer assessment.
What is measured in the U.S. News rating system? The variables are peer assessment (both dean assessment of overall school reputation, and residency program directors’ assessment of residency training), NIH funding (highly correlated with peer assessment, r = 0.7), MCAT score, undergraduate GPA, faculty:student ratio, and a few other items.
The aggregation of some of these metrics across different components of a large University and its affiliated teaching hospitals lead to overestimates of some variables—e.g., counting the NIH funding of affiliated hospitals or other components of a University increases the apparent NIH funding for a medical school far above its position if the medical school were considered on its own. Similarly, counting all faculty members in affiliated teaching hospitals as medical school faculty in some cases creates an artificially high faculty:student ratio. Collectively, these factors explain most of the discrepancy between our NIH ranking of 24 and our research US News ranking of 31.
Notwithstanding these biases in some of the U.S. News criteria, we decided to estimate the contribution of each of these variables to the total score that serves as the basis for the rankings. (Our “research” total score this year was 54, as compared with 51 last year.) Thus, I ran a regression of total score on the variables noted above. Using the four variables that provided the most explanatory power—peer assessment, NIH funding, MCAT and GPA—the model showed an extraordinarily good fit between predicted and actual total score, with an R2 of 0.994. Based on the size of the standardized regression coefficients (which gives the change in total score as measured in standard deviations in response to a one standard deviation change in the explanatory variable), the two most important variables were peer assessment (coefficient = 0.44) and NIH funding (coefficient = 0.39). MCAT and GPA also contributed, but less so with coefficients of 0.13 and 0.08, respectively. All of these coefficient estimates were highly significant statistically.
Between last year and this, we improved a bit in each of these variables—peer assessment, NIH funding, MCAT and GPA. Thus, we are proud to have moved up in the U.S. News rankings. To be sure, these variables are measurable and capture some aspects of “quality.” But let me ask the question, “Do these criteria capture the truly important features of a medical school?”
On April 2, 2009, a dean’s advisory group called the SMD National Council met in NYC. This Council, comprised of many of our most distinguished alumni, were asked to describe the features of SMD that were most important to them when they were students, and which distinguish SMD from other medical schools.
None of the factors that were emphasized as important in their subsequent medical careers are captured in the Morse code. For example, many alumni stated that, as an intern, or as a MASH surgeon in Vietnam, they felt that their Rochester medical education prepared them better to take care of patients than any of their colleagues. Such high-level preparation for direct patient care as a resident is not considered in the U.S. News criteria.
Nor are the life-long lessons of the biopsychosocial model. A faculty presenter, for example, told the story of a long-standing patient she had seen the previous day, who came to the office for an annual exam. The faculty member entered the patient’s responses to a series of questions on her computer, and then asked: “Have there been any significant changes in your life since I last saw you?” The patient said, “Yes, I lost my husband two months ago.” The Rochester-trained faculty member, 30 years after her introduction to the Rochester method of medical history-taking, moved her computer screen aside, turned her chair to face the patient directly, looked into her eyes, and asked, “How do you feel about that? Tell me how this has changed your life.” They spent the next 30 minutes exploring the implications of the patient’s loss, both medically and psychosocially.
Moreover, the variable called “faculty:student ratio” in the U.S. News criteria is simply a quantitative guesstimate of faculty availability, not a qualitative reflection of the quality of teaching or the commitment of faculty to education. Thus, there is nothing in the U.S. News methodology that captures the commitment to teaching that our faculty demonstrate day in, day out, and which is remembered well by our alumni.
We should all be proud of our advances in NIH and U.S. News rankings. This progress reflects the talent and dedicated efforts of our faculty and staff in the School of Medicine and Dentistry. I would submit, however, and I know that many Newsletter readers would agree, that there is even more about our medical school that gives us pride which isn’t captured in the Morse code.
Meliora,
David S. Guzick, MD, PhD
Dean, School of Medicine and Dentistry
University of Rochester
Dean's Newsletter
Posted May 28, 2009:
A Fond Farewell to the University of Rochester
Latest News
Climbing the Rankings - View the story - See Our Ranking!
