Do College Rankings Mean Anything?

On August 20th, the annual “America’s Best Colleges” issue of U.S. News & World Report was released. Among North Carolina schools, Duke was tied for fifth, Wake Forest 27th, UNC-Chapel Hill 29th, and NC State 86th.

UNC’s ranking stayed just where it was last year. Unable to celebrate any upward movement, UNC released a statement cheering its “improvement” in one of the factors in the U.S. Newscalculation, faculty resources. This was like a basketball team coach, after his team has had a so-so season, issuing a triumphant statement extolling the team’s improvement in free throw shooting percentage.

University administrators usually cannot resist commenting on the U.S. News rankings because they’re widely read as signifying something important about the quality of education that a school offers. If you look closely at the way the magazine calculates its rankings, however, you might well conclude that the whole enterprise is a waste of time. In “Do College Rankings Mean Anything?” released August 23 and available at www.popecenter.org, Michael Lowrey and I argue that the U.S. News rankings are a completely unreliable guide to educational quality.

In preparing its rankings, U.S. News relies on six factors. Four of those factors are input measurements: financial resources, alumni giving, faculty resources, and student quality. One is an output measurement (student retention and graduation rates) and one is a subjective guess (academic reputation). Not one of the factors purports to measure the thing that academic quality is centrally about, namely learning.

Let’s look at each, starting with academic reputation.

Here’s how the magazine figures out academic reputation, which is 25% of the entire ranking. It sends a survey form to the three top academic officials at every college and university in the country. Those individuals are asked to rank on a five point scale their assessment of the academic reputation of other schools in their category. That is, the leads of liberal arts colleges rank other liberal arts colleges, the heads of research universities rank other research universities, and so on. The problem here is one of knowledge. The college officials are unlikely to have much if any concrete knowledge about the academic climate at other schools. Whatever numbers emerge from the reputational assessments are simple guesswork.

What about student quality? The way U.S. News performs its calculation (15% of the total score), the more selective a school is, the more points it gets. Schools that enroll brighter students score better. But just because a college enrolls a lot of bright students does not necessarily mean that they will be well taught. Students at some of the most elite schools admit that some of their professors have very low academic standards and that they can get Bs without much effort. There isn’t any automatic connection between the academic capability of the typical student and how much he learns.

Faculty resources is another criterion. Twenty percent of the score is based on a number of faculty-related matters, including compensation, percentage of faculty members with a terminal degree, the percentage of full-time faculty, and student to faculty ratio. Once again, there isn’t any necessary connection between those factors and how well courses are taught and how much students learn. Consider compensation, for example. A school with a lot of highly-paid “superstar” professors looks great to U.S. News, but we should not assume that because they’re paid a lot, they are correspondingly effective in their teaching. In fact, students often complain that the “superstars” tend to neglect their teaching obligations in favor of writing and consulting.

Next, U.S. News awards points for having high student retention and graduation rates. Just because a lot of students stay in school and graduate, however, doesn’t necessarily mean that they are learning anything. College A might have generally tough academic standards and therefore have a high percentage of students decide to leave to enter the job market or switch to an easier institution. College B might keep most of its students enrolled through grade inflation, an undemanding curriculum, and professors who do more entertaining than teaching.

Financial resources and alumni giving are likewise poor proxies for educational quality. Merely because a school has a big endowment and can spend lavishly doesn’t guarantee that its students learn more than at a school which has to pinch its pennies. Nor does alumni giving prove that a school has a good academic climate.

And that’s why I say that college rankings don’t mean anything.

George Leef (jsanders@popecenter.org) is executive director for the John W. Pope Center for Higher Education Policy in Raleigh. For more information visit our Web site at www.popecenter.org.