Economists apply market principles to develop a new way to rank colleges

Critics of college rankings say universities aren't like their sports teams. You cannot settle who is best with head-to-head competition on the playing field. <br/><br/>But in a new paper, a group

Tuesday, October 19th 2004, 12:57 pm

By: News On 6

Critics of college rankings say universities aren't like their sports teams. You cannot settle who is best with head-to-head competition on the playing field.

But in a new paper, a group of economists and statisticians begs to differ.

They lay out a system that ranks colleges on how they perform in one kind of head-to-head competition they claim says a lot about a school, and can be measured: the battle for students who are admitted to several colleges and have to choose among them.

While the U.S. News & World Report rankings use statistics like admission percentage, SAT scores and student-faculty ratio, critics say that those figures are not necessarily much use to prospective students and that colleges can manipulate them.

In their proposal, the economists sidestep the tricky question of what makes a good college. Instead, they assume top high school students know best, and they simply report their choices. Of the students admitted to, say, both Brown and Penn, how many choose each place? It is the same principle as Zagat's restaurant guides: Don't try to grade the food, just reveal whether a lot of people like it or not.

The authors _ Caroline Hoxby and Christopher Avery of Harvard, Andrew Metrick of the University of Pennsylvania and Mark Glickman of Boston University _ have been working on their model for years. Their most detailed results yet were published recently by the National Bureau of Economic Research.

Here's how it works: Imagine two students, one choosing between Stanford and Harvard, another among Stanford, Berkeley and Pomona. The statistical model views each one of those students as a ``tournament'' between the colleges involved; if Stanford ``wins'' either student, its ranking rises in relation to the schools it beat.

Colleges often compete against the same schools over and over, and may never compete against others. But with enough data, Stanford's place in relation to all schools begins to emerge and the rankings take shape.

The model, which resembles the one used to rank professional chess players, adjusts to balance out influencing factors such as a big financial aid offer from one school.

The authors offered a preliminary demonstration by tracking the college choices of 3,240 high-performing students from 396 high schools nationwide. They say that it works well for top schools but that more data is needed to improve their confidence in the rankings lower down.

So what does their trial run reveal? The top 20 schools look similar to the top liberal arts colleges and universities in the U.S. News rankings (the categories are combined in this study) but their order changes. In U.S. News, for instance, Harvard and Princeton share the top spot among national universities; here, Harvard is No. 1, while Princeton falls to sixth. Duke, fifth in U.S. News, drops to 19th.

But the system rewards other schools. Wellesley's appeal as a women's college evidently helps it win ``tournaments'' with elite universities. Georgetown and Notre Dame score higher than they do in U.S. News, probably because of their popularity with Roman Catholic students.

In an interview, Hoxby said the ``market'' of high school students has a good sense of what makes a good college. And she said colleges cannot game this system.

Many critics of college rankings claim that that some schools try to lower their admissions percentage _ and make themselves look more selective _ by encouraging applications from students who have no chance of getting in. Or they try to boost their matriculation rates _ the percentage of admitted students who enroll _ by turning down top applicants they suspect are just applying as a back-up and are not likely to come.

But in this new system, the only way for colleges to improve their ranking is to get more top students to apply and attend.

``Nobody would feel under pressure to manipulate their admissions and matriculation rates,'' Hoxby said.

Facing criticism, U.S. News & World Report recently dropped matriculation rate as a component of its formula, and Robert Morse, the magazine's director of data research, said admissions percentage plays a tiny role.

As for the economists' approach, Morse said he doubts they could get the expanded data they need to make the system more credible.