Table of Contents

INTERNATIONAL HIGHER EDUCATION

The Dilemmas of Ranking

Philip G. Altbach
Philip G. Altbach is Monan professor of higher education and director of the Center for International Higher Education at Boston College.

Rankings of academic institutions, programs, and departments are all the rage worldwide. National rankings are ubiquitous and at least two worldwide rankings exist. These operations are widely criticized for questionable or flawed methods as well as for the concept itself, but everyone uses them. When done well, they can be valuable to consumers, policymakers, and to academic institutions themselves as they compare themselves with peer institutions at home or abroad.

Rankings range from irresponsible musings by self-appointed experts and money-making schemes by commercial organizations to, at their best, serious efforts by academic or research organizations. Publications-including U.S. News and World Report in the United States, the Times Higher Education Supplement (THES) and the Financial Times in Britain, Der Spiegel in Germany, Reforma in Mexico, and Asiaweek (now defunct), and others-have sponsored rankings. A few outlets, such as U.S. News and THES, have achieved a degree of respectability. Rankings have achieved a degree of public legitimacy and an aura of credibility because respected research and policy organizations have sponsored some of them. The research and teaching assessments carried out by the funding councils in the United Kingdom, the rankings of disciplines done by the National Research Council in the United States, and some others are examples. This past year, Shanghai Jiaotong University and the THES have published worldwide university rankings.

The Rationale for Rankings
Rankings and league tables have been around for a long time, but there has been dramatic growth in the past several decades. The stakes are now much higher. Rankings serve a variety of purposes, good and bad. Rankings are also inevitable-in the era of massification, those who finance higher education and the public want to know which academic institutions are the best. Governments and funding authorities want to know how best to invest their resources and need to be able to differentiate among a large number of institutions. Mass higher education requires differentiation since institutions serve diverse purposes and students attend universities for many reasons. Rankings can help to define differentiated academic systems if they can be devised to capture a variety of metrics, and thus make decision-making easier.

Universities also try to legitimate their positions for reasons of prestige, student and staff recruitment, and other goals. There is increasing competition among universities and countries for funds, prestige, and the best and brightest students and staff. Intense competition also exists among students to study at the most prestigious schools. Faculty compete to be appointed at the best possible universities. Institutions compete for research grants and public support. Competition has long been a part of a small number of academic systems, such as the United States and, to some extent, Canada, but it is a new factor in most countries. Until recently, most countries had small and elite academic institutions, and it made little difference where a student matriculated. Ranking and competition did not exist.

Problems
The problem with ranking concerns the practice, not the principle. How is it possible to accurately measure a nation’s academic system, or for that matter the quality of a single institution? Or of academic institutions worldwide? Many rankings resemble “popularity contests”-asking groups in the academic community, especially administrators, their opinions about peer institutions. This method is especially popular among the many magazines and newspapers worldwide that rank institutions. Even the most sophisticated rankings include these peer opinions, although many more measures are also included.

Rankings count factors such as external funding, numbers of articles and books written by faculty members, library resources, proportion of faculty members with advanced degrees, and quality of students (measured by scores on admissions or other tests). These numbers are assumed to be a proxy for quality, which they are to a significant extent. However, the number of articles published does not necessarily relate to the quality or impact of the articles. Institutions strong in the biomedical sciences will usually have more external grant or contract funds than those with strength in the humanities or social sciences. Rankings generally do not include teaching quality. There are, in fact, no widely accepted methods for measuring teaching quality, and assessing the impact of education on students is so far an unexplored area as well.

Universities have different missions and goals-and ranking also tends to ignore these issues. The rankings generally emphasize the norms of the top research universities. The assumption is that “one size fits all” and that the norms of the research universities are the gold standard. Focusing on undergraduate teaching, stressing specific programs in limited professional fields, providing access to underserved populations, and other goals are not rewarded in most ranking schemes.

International Concerns
If rankings are problematical nationally, they present even more challenges globally. Publication counts often stress established refereed journals included in such databases as those of the Institute for Scientific Information (ISI). These are mainly journals published in English and selected with the norms of the major academic systems of the United States and Britain in mind. While English is increasingly the language of science, it is not necessarily the central medium of communication in the humanities, law, and a number of other fields. Using international recognition such as Nobel Prizes as a proxy for excellence downplays the social sciences and humanities, fields in which Nobels are not awarded, and causes further disadvantages for developing countries and smaller universities around the world. Using citation counts as a way of measuring excellence also presents serious problems. Such counts emphasize material in English and journals that are readily available in the larger academic systems. It is well known, for example, that American scientists mainly cite other Americans and tend to ignore scholarship from other countries. This may artificially boost the ranking of US universities. The fact is that essentially all of the measures used to assess quality and construct rankings enhance the stature of the large universities in the major English-speaking centers of science and scholarship and especially the United States and the United Kingdom. It is also the case that universities with medical schools and strength in the hard sciences generally have a significant advantage because these fields generate more external funding, and researchers in them publish more articles.

Conclusion
Rankings and league tables play a useful role. They focus attention on key aspects of academic achievement and may influence policymakers who might otherwise be content to slash budgets and maintain mediocrity. Everyone wants to be “number one,” and countries want to have top-ranking universities. They may stimulate the academic community to strive to improve quality and encourage competition and productivity. Rankings are benchmarks of excellence for the public. And they help to mark differences among academic institutions and in this way help may lead to differentiated goals and missions in academic systems.

Yet, they often measure the wrong things, and they use flawed metrics to do the measurements. They privilege the already privileged and stress certain academic disciplines (mainly in the hard sciences) over others. Rankings ignore key academic roles such as teaching and do not look at all at how students are affected by their academic experience.

The solutions to these significant problems will be a difficult task. There are many conflicting interests at play in the “ranking game.” Creating generally agreed criteria that can be used to do the rankings may be a useful first step. Providing appropriate ways of measuring them is also necessary. Transparency throughout the process is central-many of the current rankers are notably unclear about both criteria and methods. Applying the norms and values of the major academic “powers” will not accurately measure quality worldwide, nor will it result in meaningful international rankings. In the competitive and market-oriented academic world of the 21st century, rankings are inevitable and probably necessary. The challenge is to ensure that they provide accurate and relevant assessments, and measure the right things.

[Online] Available: http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/Number42/p2_Altbach.htm