Top Colleges, Rated by Those Who Chose Them
by ERIC DASH
The New York Times
October 20, 2004
For years, editors and educators have turned out lists grading the nation's coolest, hottest, hardest-partying, and of course, academically best colleges and universities.
But while those surveys captured what administrators, faculty members, students, and other experts thought, no one ranked colleges based entirely on where America's best and brightest ultimately decided to go. Now, a group of economists has compiled a new college ratings system that does just that.
In a study submitted last week to the National Bureau of Economic Research, Harvard and Yale not unexpectedly take the top two spots. But the authors contend that their method of rating colleges and universities based on the real-life decisions of admitted students is more accurate and more relevant than rankings published each fall in college admissions guidebooks and magazines, like U.S. News and World Report.
''What you are getting in all these other systems is sort of an expert analysis of polling data,'' said Andrew Metrick, one of the study's four authors and a professor at the Wharton School of the University of Pennsylvania. ''This provides a market-based view.''
Instead of measuring statistics like admissions rates or having college administrators rate their peers, the researchers analyzed the college choices of more than 3,200 high-achieving high school seniors from the class of 2000. By asking those students where they enrolled when they were accepted by several different colleges, the economists compiled a won-loss record for each college as if it was a competitor in a chess tournament. The researchers then generated a preference ranking for more than 100 colleges, employing the same scoring system used for chess masters.
College admission deans say they have long used similar preference ratings systems in their own backyards, internally tracking the students who matriculate and those who turn them down, to help them refine their marketing messages and define their competition. But the researchers' ranking may be the first study to statistically gauge the strength of a college's appeal in a way most students, parents and alumni can understand.
Harvard and Yale came out one and two, and renowned research institutions like Stanford, Cal Tech, M.I.T. and Princeton rounded out the Top 6. But if this was predictable, perhaps the rankings of colleges like Emory (61) or New York University (39) were a bit lower than one might expect.
''Once you get outside the Top 10, people tend to have only a vague sense of how schools are ranked in terms of student preference,'' said Caroline Hoxby, a Harvard economics professor and one of the authors. Some high school guidance counselors said that the study smacked of a popularity contest.
''If one is a lemming, this would be a great way of picking a college,'' said Burke Rogers, a veteran college counselor at St. George's, a private boarding school in Newport, R.I. The study, he added, seemed to encourage conformist thinking and the myth of a best college, not a best-fitting one, that high school guidance counselors work hard to dispel.
The researchers conceded that sometimes student preferences might not accurately value an individual college. ''If students are all wrong, then the ranking is reflecting students' wrongheaded ideas,'' Ms. Hoxby said. But by and large, the authors said, their ranking gives students and colleges useful information.
''It tells you something about what quality of students you will find and how the world might view you depending on which college you go to,'' said Christopher Avery, another author, who studies college admissions at Harvard's Kennedy School of Government. Mark Glickman, a statistics professor at Boston University, also contributed to the study.
The degree of Harvard's dominance was staggering. From their analysis, the researchers predict that a student who gets into both Harvard and Princeton is three times as likely to go to Harvard. That same student, on average, is more than eight times as likely to prefer Harvard to Brown, which ranked No. 7 on the list. M.I.T. and Cal Tech, the researchers said, have Harvard-like appeal but mainly among engineers and scientists; they added that the same holds true for Brigham Young University, which has strong Mormon ties, among Utah residents.
There were major differences between this ranking, which rates the colleges that top students prefer when they have more than one letter of acceptance, and the list published by U.S. News, which uses various statistics to measure academic quality.
Indeed, the two may not always go hand in hand. (Besides their distinct methodologies, there was a three-year time lag in the data used by the researchers, and the economists' study combined national universities with small colleges while U.S. News ranks them separately and allows ties.)
Washington University in St. Louis, for example, was tied for the No. 11 spot in this year's U.S. News list of national universities; it was the 62nd rated choice when students' preferences were revealed. Likewise, places like the University of Pennsylvania, the University of Chicago and Duke placed at least eight spots lower on the preference ranking than on the U.S. News list.
Meanwhile, colleges like Brown University, which offers a flexible curriculum, along with Notre Dame and Georgetown, which attract a loyal Catholic following, placed several spots higher in the student preference rankings.
Although he said no single ranking should be the determining factor in a college decision, Robert Morse, the director of data research at U.S. News, also has reservations about how the economists' list might be used.
''It would be interesting to know the preferences of other people who got into these schools,'' he said. But to ignore indicators like class size, graduation rates and faculty resources, he added, ''You would be leaving a lot of things on the table to make the best possible choice.'' Several admissions deans said they found the researchers' analysis interesting, but they expressed skepticism. ''People are becoming weary of all rankings,'' said Lee Stetson, the dean of admissions at Penn. ''Whether it helps students or parents make their choices or if we can measure the strength of our institution based on this study is probably still a question.''
The researchers, whose study is available at ssrn.com/abstract=601105, say they do not intend to commercialize their rating system or produce an annual list; they say they want to offer an unbiased, scientific alternative to existing rankings.
But even if they do have a better method, Bruce Poch, the dean of admissions at Pomona College, asks, ''What does this tell kids?''
''This is where someone's announcements of preferences poisons the well of independent thinking,'' he continued. ''It's like buying a car. Do you buy a Toyota because it is the No. 1 selling car? It doesn't mean it is a match for you.''