International Higher Education, Spring 1999

Management of Quality Assurance

A review of recent international practice in the management of quality assurance shows tremendous variety in approaches and methodologies, providing a wide range of possible models for both systems and institutions.

Administrative Responsibility
The most common patterns at the national level are for responsibility to lie with specialized units or agencies set up by the government or for responsibility to be given to the central agency responsible for higher education coordination, whether it be a ministry of higher education or a university grants commission. One of the major issues concerning government agencies responsible for quality assurance relates to the degree of independence they should have from both ministers and major ministries and departments.

In a small number of countries, responsibility at the national level is under the control of an agency set up by higher education institutions. Similarly, within higher education systems, arrangements differ widely. Sometimes presidents or rectors take responsibility, while in other cases responsibility lies with an academic council or board.

Participation in the Program
An important variation between quality-assurance systems is whether participation is voluntary or compulsory. Many countries began with institutional audits, on a voluntary basis. Thus, in Britain, the institutional audits run by the Academic Audit Unit (AAU) were voluntary, and the Research Assessment Exercise run by the Higher Education Funding Council of England for the funding councils continues to be based on the principle of voluntary participation. Generally, however, with national reviews of disciplines, participation is compulsory. Even when participation in such reviews is voluntary, strong moral and professional pressures often operate on institutions.

Methodologies
Most quality-assurance mechanisms depend on one or a combination of a limited number of key methodologies, the most important of which are: self-studies or self-evaluation; peer review by panels of experts, usually involving at least some "external" members; the use of relevant statistical information and performance indicators; and surveys of key groups, such as students, graduates, and employers.

Self-studies have proved both effective and cost-efficient, achieving a high degree of ownership since key staff are heavily involved and such involvement increases the chances of improvement being achieved. On the other hand, experience points to the value of combining self-study with some element of external peer review to ensure that the self-study is taken seriously and to bring in outside perspectives.

Peer review by outside experts is a well-established academic process and generally works well, as long as panel members show respect for the values of those being evaluated and accept that often a panel's main contribution is in assisting with self-learning. At the same time, peer review can easily introduce outside values and constructs.

External reporting often is thought necessary not only to ensure accountability requirements but also to ensure that a review process is taken seriously. A crucial question, however, is to whom reports should go, and how widely and publicly they should be distributed.

Focus
At the national level, the most common forms of assessment are "horizontal" reviews of disciplines and "vertical" evaluations of institutions. Reviews of disciplines are usually carried out by panels of experts using site visits and analysis of documentary information, much of which is usually produced by the academic units being reviewed.

Institutional reviews include academic audits of quality-assurance processes and outcomes, and more extensive comprehensive reviews. The international practice of institutional academic audits has been considerably influenced by the methodology developed a decade ago by the AAU in the United Kingdom. At the institutional level, the main evaluation mechanisms are regular reviews of departments and faculties; reviews of academic courses and programs; reviews of administrative and services units; and reviews of functional areas such as information technology.

Purposes
Quality-assurance programs often can serve a variety of purposes, but generally their primary purpose is a combination of public accountability, improvement, and renewal. In some cases, there is a gap between stated and actual purposes, and as a number of writers have pointed out, there is often tension between accountability and improvement purposes.

Reporting and Follow-up
Reporting and follow-up activities are vital parts of any worthwhile quality-assurance program, either at the national or institutional level. But a major challenge is to devise fair and effective methods that are likely to lead to improvements but at the same time will not be unduly damaging to the unit or units being reviewed.

A variety of approaches are widely used with regard to the distribution of reports. In some cases, reports are provided solely to the institution or the unit concerned, but increasingly the practice is to make the results more widely available. What happens to a report can be a most contentious issue.

In some cases, a system of rankings based on performance in relation to established criteria is used. In the United Kingdom, for example, participating university departments are ranked in separate reviews of performance in both teaching and research, and these results are publicly announced.

In a minority of cases, some element of performance funding is used as part of a quality-assurance program. In the United Kingdom, performance in research, as measured by the Research Assessment Exercise, is used as the basis for allocating substantial research funding, while in Australia from 1993 to 1995 a sum of between $50 million and $80 million was allocated annually on the basis of assessments.

In a limited number of cases, the final result is accreditation or validation of the program or institution, using predefined standards to assess whether or not the institution or the program is given formal recognition.