Managing the Register

In December 2018, the Register included 44 quality assurance agencies based in 23 countries. As a result, 24 EHEA countries fulfilled the EHEA Key Commitment that all higher education institutions are subject to regular external quality assurance in line with the ESG, through relying on EQAR-registered agencies. Another 6 EHEA countries fulfilled the commitment partially, in that some – but not all – higher education institutions are externally quality assured in line with the ESG.

In order to enhance its overview on the scope of registered agencies’ activities, EQAR has been collecting information on their external QA activities annually since 2014. The collected data reveal the volume of external QA activities within the scope of the ESG as well as trends and changes within higher education systems across the EHEA.

In 2018 the 46 EQAR-registered agencies carried out a total of 8902 external QA activities, e.g. evaluation, accreditation, review or audit of a higher education institution or programme.

General trends in EQA activities

Similar to previous years, the activities were spread across 66 countries, covering most of the countries within the EHEA (38 of the 48 higher education systems). Although we observed a 14% decrease in the total volume of external QA activities carried out in 2018, the drop is caused solely by the drop in the programme level reviews. External QA activities at institutional level have, in turn, increased by 22%. This reflects a change of focus in the external QA systems of several countries, from primarily considering individual study programme towards institutional audit/accreditation, with institutions themselves assuming greater responsibility for the quality assurance of their indivudal programmes. Additionally the monitoring also showed a rise in the number of cross border reviews by 26% and joint programmes by 37%.

Highs and lows in EQA activities – specific reasons

The drop in the external QA activities at programme level is a result of several factors:

• Several agencies (or the national external QA system in which they operate) have transitioned from programme evaluation/accreditation/review to institutional evaluation/accreditation/review/audit during 2018, or are in process of such a transition. This was/is the case for EKKA (Estonian national QA agency), SQAA (Slovenian national QA agency) and AI (Danish national QA agency). In the case of HCERES (French QA agency), the transition was from programme-by-programme evaluation to a review of disciplinary fields, leading to a summary report for the field as a whole. All these agencies have registered a decreased by 50% or more in their external QA activities compared to 2017.

• Some agencies simply experienced high fluctuations in their number of external QA activities, without a specific systematic change. This was the case for MusiQuE (QA agency with an international scope), CTI (French QA agency for accreditation in the field of engineering) and SKVC (Lithuanian national QA agency).

• The absence of reported reviews by CAQA/NEAQA, as the agency’s registration ceased in 2017, which was not compensated by the additional activities of the newly registered AIC, EAEVE and NEAA (see under Decisions on Inclusion and Renewal).

In contrast to the drop in external QA at programme-level, the data also reveals increases both at programme and institutional level in some cases:

• Six QA agencies have doubled their programme evaluations or accreditations compared to 2017: IAAR, Unibasq, EAPAA, ACPUA, ASIIN, NOKUT.

• The external QA activities at institutional level have doubled for A3ES (Portuguese QA agency), ACSUG (Spanish regional QA agency), ASHE (Croatian QA agency), AAC-DEVA (Spanish regional QA agency), QQI (Irish, QA agency), AQAS (German QA agency), ANECA (Spanish QA agency), NOKUT (Norwegian QA agency) and FIBAA (German QA agency).

• While some of the QA agencies have just introduced institutional evaluations and audits (ACSUCYL, fmid) the increase is to a small extent also due to the reported activities of NEAA (Bulgarian QA agency) who has been included in the Register as of February 2018.

Overall, institutional evaluations made up the highest share (7.9%) of the total number of external QA activities carried out since 2016, pointing to a possible trend towards less programme-level reviews and more institutional evaluations, audit or accreditations.

Annex: Data on Activities of EQAR Registered- Agencie’ Activities in 2018

External QA activities of EQAR-registered agencies (2014-2018)

In 2018, the Register Committee considered 13 applications for inclusion and renewal of registration. Following the Committee’s conclusion that they substantially comply with the ESG, three agencies newly joined the Register:

• European Association of Establishments for Veterinary Education (EAEVE)
• National Evaluation and Accreditation Agency (NEAA, Bulgaria)¹
• Academic Information Centre (AIC, Latvia)

Three quality assurance agencies had their registration on EQAR renewed:

• Estonian Quality Agency for Higher and Vocational Education (EKKA)
• National Agency for the Quality Assessment and Accreditation of Spain (ANECA)
• Norwegian Agency for Quality Assurance in Education (NOKUT)

The Register Committee has further deferred five applications (three application for initial inclusion and two application for renewal of registration) pending additional representation on the reasons for a possible rejection and one application pending additional clarification; final decisions are to be made in 2019.

The application for renewal by the Commission for Accreditation and Quality Assurance (CAQA) – now changed into the National Entity for Accreditation and Quality Assurance (NEAQA) –  was rejected as the Register Committee was unable to conclude that the agency continues to comply substantially with the ESG. The Committee nevertheless noted that the major reorganisation and restructuring effort was an important step towards addressing the issues that prevented a positive conclusion.

Due to unexpected circumstances that affected the submission of its review report, the Slovenian Quality Assurance Agency’s (SQAA) application for renewal was delayed and the agency’s registration suspended, pending a final decision by the EQAR Register Committee on SQAA’s renewal of registration.

The decision on the application by QANU (Quality Assurance Netherlands Universities) was published following an (unsuccessful) appeal in 2018. The agency was nevertheless encouraged to address the issues that led to its rejection and to re-apply for registration, potentially based on a focused review.

The full decisions can be consulted here.

^{1</ NEAA was also previously registered from 2009-10-07 to 2013-07-31.}

Based on the seven final decisions made in 2018, the Register Committee found that most challenges could be observed in relation to standard 2.6 (Reporting) and standard 2.7 (Complaints and appeals), with four out of the seven agencies only partially compliant.

Considering the publication of reports (ESG 2.6), the Committee noted that some agencies only publish a final processed or summarised report, but not the full experts’ report. Other agencies did not publish the reports for certain specific activities, e.g. consultation visits or initial accreditation procedures. The standard, however, clearly states that full reports by the experts should be published for all external QA activities.

The analysis of complaints and appeals (ESG 2.7) revealed that several QA agencies lacked clearly defined complaints processes, but handled any complaints in a less formal and structured manner. Moreover, in some agencies appeals processes were unavailable for certain external QA procedures that lead to no decisions at all or where the final decision is taken by a body external to the agency; that is, the agency’s work ends with the publication of a report. The Register Committee considered that the wording of the standard requires QA agencies to have an internal appeals system that allows institutions to question the formal outcome of an external QA procedures, e.g. in cases of erroneous judgments or incorrect statements within the text of the review report. The Committee considered that also a report (without a yes/no-type decision) is a formal outcome and should be open to appeal, especially since it might have an impact on the institution by being published.

Partial compliance with standard 3.1 (Activities, policy and processes for quality assurance) usually concerned the limited involvement of stakeholders or an unbalanced composition of the governing bodies; or insufficient mechanisms to separate the agency’s external QA activities within the scope of the ESG from the consultancy projects and other activities it carried out.

The analysis into the level of compliance with standard 2.1 Consideration of internal quality assurance showed that in two of the seven application, ESG part 1 was not satisfactorily covered through the totality of QA procedures carried out by the agency or that the ESG elements were not always consistently addressed in reports.

For standard 2.4 (Peer-review experts) a conclusion of partial compliance was the result of not involving students in the expert panels for all external QA activities. In some cases, agencies did not include student experts in the panels for some particular activities, or were not able to fully implement in practice their policies to include students in all panels.

The Register Committee considered two substantive change reports in 2018. In both cases, the agency reported a change in its organisational arrangements i.e. establishment of a new committee, change in the decision making body of the agency and in its the administrative set-up.The Register Committee took note of the information provided in both cases.

In March 2018, the Register Committee decided to exclude the Kosovo Accreditation Agency (KAA) from the Register following the decision by the Minister to dismiss the entire KAA Board as well as its Acting Director. The Committee could not identify any exceptional circumstances that would have justified the dismissal of the Board and found that KAA’s ability to operate autonomously and without serious risk of third party influence was impeded. The Committee concluded that KAA failed to be in compliance with ESG standard 3.3 and therefore excluded the agency from the Register. At the invitation of the Minister, the EQAR President and Director travelled to Kosovo to discuss the issue and steps to remedy the situation. KAA has subsequently re-applied for registration and the application is expected to be considered in 2019.

In 2018 EQAR did not receive any third-party complaints on the work of registered agencies.

EQAR maintained an active dialogue with the coordinators of external reviews used for applications for inclusion on the Register. Due to a change in the legal framework for quality assurance in Germany, the German Accreditation Council (GAC) no longer coordinates agency reviews and ENQA is therefore now the coordinator of most reviews.

In addition to regular meetings and dialogue between the staff in charge, EQAR and ENQA have been holding approximately one official meeting per year, to discuss current issues in relation to agency reviews. In 2018, that meeting took place in June.

ENQA has launched an external review of its agency reviews in 2018. To inform ENQA’s self-evaluation process, a focus group with the EQAR Register Committee was held in December 2018, discussing the use and work with ENQA agency review reports.

In the few cases where agencies choose to work with another review coordinator, EQAR aims to have a briefing conversation with the review coordinator before the process, in order to ensure that the review corresponds to EQAR requirements and expectations, and to avoid common pitfalls.