Evaluation of EQAR

• Call for Tender for the Coordination of the External Evaluation of EQAR

Briefing notes for the Members’ Dialogue 2021 (they summarise key findings from the surveys carried out for the self-evaluation, other data and input available, and introduce key questions):

• The Register and the future of reviews
• Cross border recognition of QA
• EHEA key commitments
• Analysis & policy briefs and serving the European higher education sector
• Agency vs system level
• DEQAR

• Adoption of the approach and roadmap for the Self-Evaluation and External Evaluation by the General Assembly in June 2020
• Start of Self-evaluation process: October 2020 (first SEG meeting)
• Appointment Coordinator: December 2020
• Endorsement Self-Evaluation Report by the General Assembly: April 2021
• Panel appointed: summer 2021
• Site visit: September 2021
• External Evaluation Report: November 2021 (for factual comments)
• External Evaluation Report published on website: February 2022

A Self-Evaluation Group (SEG), including members of the General Assembly, the EQAR Executive Board, the Register Committee and Secretariat, coordinates the self-evaluation process and drafts the Self-Evaluation Report, in consultation with all EQAR bodies.

The SEG is composed of the following individuals:

• Karl Dittrich (President, Chair of the SEG)
• Michal Karpíšek (Executive Board)
• Tia Loukkola (Executive Board)
• Daniel Miescher (Liechtenstein, former observer on the Register Committee)
• Ivana Radonova  (Bulgaria)
• Beate Treml (Register Committee)
• Colin Tück (Director)
• Steffen Westergard Andersen (Register Committee)

In preparing the Report, the SEG draws upon the feedback regularly elicited from applicants and on internal reflections of the Register Committee and the Executive Board. Furthermore, the SEG received feedback on the workings of EQAR from a survey among EHEA governments, stakeholder organisations and quality assurance agencies.

The Coordinator for the evaluation of EQAR appointed an independent Panel to carry out the evaluation, including experts in quality assurance of higher education from different backgrounds and different stakeholder perspectives. The following individuals serve on the Panel:

• Liviu Matei
  CEU Provost – Chair of the Panel
• Tatiana Yarkova
  Higher Education Consultant (Germany). Former CEU Academic Secretary (2012-2020), Co-Chair of the Self-Study Steering Committee for CEU’s re-accreditation with the Middle States Commission on Higher Education (2016-2019).  – Secretary of the Panel
• Matyas Szabo
  Senior Manager, CEU Yehuda Elkana Center for Higher Education (External evaluation manager)
• Terhi Nokkala
  Senior Researcher at the Finnish Institute for Educational Research (FIER), University of Jyväskylä and an Adjunct Professor of Higher Education Administration at the Faculty of Management, University of Tampere
• Liv Teresa Muth
  Steering Committee member of ESU’s QA Pool of students’ experts for the last 2 years. Extensive experience in QA both on a national level in Germany and internationally
• Sauwakon Ratanawijitrasin
  Professor at Mahidol University, Thailand, former Director of South-East Asian Ministers of Education Organization-Regional Centre for Higher Education and Development (SEAMEO RIHED), and Deputy Executive Director of the ASEAN University Network)
• Oliver Vettori
  Dean, Accreditation & Quality Management, WU Vienna University of Economics and Business
• Adrian Curaj
  Former minister of education, science and innovation of Romania. Head of the UNESCO Chair on Science and Innovation Policies at the National University of Political Studies and Public Administration, and a professor at the “Politehnica” University of Bucharest

EQAR’s Strategic Plan 2013-2017 included the commitment to undertake a comprehensive self-evaluation exercise in 2015/16, involving all bodies of EQAR, in order to assess progress made and to consider a revised Strategic Plan in 2017.

The self-evaluation exercise consisted of a self-reflection of the organisation as to how it has performed with a view to its mission and how it would improve. The specific objectives of the 2015/16 self-evaluation exercise were:

• To take stock of how recommendations of the 2011 external evaluation of EQAR were implemented;
•  To feed into the revision of the Strategic Plan;
•  To demonstrate accountability to its members and stakeholders.

The self-evaluation therefore focussed on the following key questions:

• Are EQAR’s mission and objectives still relevant?
• What are the expectations of governments and stakeholders?
• Is the singularity of EQAR’s mission clear?
• Are EQAR’s activities fit for purpose with regard to its mission and objectives? How does EQAR know, what impact did they have?
• How did EQAR perform against the indicators for success defined in our Strategic Plan 2013-2017?

The self-evaluation process was coordinated by a selfevaluation group (SEG) which included representatives of EQAR’s main bodies and stakeholders. Next to an analysis of existing documents (published by EQAR or other organisations) the SEG gathered data through a number of surveys and events, see section 1.3 of the report for details.

EQAR Strategic Plan 2013-2017 (March 2013)

EQAR Strategic Priorities and Follow-Up to the External Evaluation (February 2012)

Terms of Reference for the External Evaluation

1. Members and Meetings of the SEG
2. Ministerial Communiqués, E4 Report (extracts)
3. EQAR Statutes
4. EQAR Mission Statement
5. EQAR Procedures for Applications
6. Composition of EQAR Committees
7. Overview of Applications for Inclusion on the Register
8. Annual Reports 2008 2009
9. Register Committee Reports
   1. Report to the Third General Assembly, October 2009
   2. Summary Report 2008-2010, October 2010
10. Mock-up Register Entry
11. Feedback on the Guide for Applicants
12. Summary of interviews of governments and stakeholders
13. Website visitor survey – raw summary of results
14. Quality assurance agency survey – raw summary of results

When mandating the E4 Group to establish EQAR, European ministers asked the E4 organisations “to ensure that after two years of operation, the register is evaluated externally, taking account of the views of all stakeholders” (London Communiqué 2007).

The evaluation addresses three main questions, as specified in the Terms of Reference:

• Are the organisational structures and methods of EQAR fit for purpose in the light of the agreed objectives? Have they functioned effectively and efficiently in practice?
• What has been the initial impact of EQAR? Is it in line with the desired goals?
• What improvements are desirable? How might the organisation develop and act further with a view to best achieving its mission and objectives?

The Steering Group included the following individuals:

• Judith Eaton (Chair of the Steering Group) – President of CHEA
• Jan Levy – Former Director General of the Norwegian Ministry of Higher Education
• Norman Sharp – Former Director of the Quality Assurance Agency Scotland
•  Martina Vukasović -Director of the Centre for Education Policy, Belgrade

The Steering Group for the evaluation of EQAR appointed an independent expert Panel to carry out the evaluation, including European and non-European experts in quality assurance of higher education, coming from different backgrounds and including all stakeholder perspectives.

Evaluation Panel
The following individuals served on the Panel:

• Sir John Daniel (Chair of the Evaluation Panel)
  President and Chief Executive Officer, Commonwealth of Learning, Canada
• Barbara Brittingham
  President/Director of the Commission of Institutions of Higher Education, New England Association of Schools and Colleges, USA
• Liam Burns
  President of NUS Scotland (National Union of Students)
• Eduardo Marçal Grilo
  Professor and Member of the Board of Trustees of the Calouste Gulbenkian Foundation, Portugal
• Lis Lange
  Senior Director, Institutional Research and Academic Planning, University of the Free State, South-Africa
• Pall Skúlason
  Professor of Philosophy at the University of Iceland
• David Parry (Secretary to the Evaluation Panel)
  Higher Education consultant, United Kingdom

A Self-Evaluation Group (SEG), including members of the EQAR Executive Board, Register Committee and Secretariat, coordinated the self-evaluation process and drafted the Self-Evaluation Report, in consultation with all EQAR bodies. The final Self-Evaluation Report was submitted to the Evaluation Panel in March 2011.

Self-Evaluation Process
The SEG was composed of the following individuals:

• Lucien Bollaert (Register Committee, Chair of the SEG)
• Andrea Blättler (Executive Board)
• Dorte Kristoffersen (Register Committee)
• Helle Otte (Denmark, Observer on the Register Committee)
• Lesley Wilson (Executive Board)
• Colin Tück (Director)

In preparing the Report, the SEG drew upon the feedback regularly elicited from applicants and on internal reflections of the Register Committee and Executive Board. Furthermore, the SEG elicited feedback on the workings of EQAR from interviews of a sample of European governments and stakeholder organisations as well as surveys.

The Evaluation Panel conducted a site visit to Brussels from 12 to 14 May, in the context of the External Evaluation of EQAR. The Panel met with the Self-Evaluation Group, all EQAR committees (Executive Board, Register Committee, Appeals Committee), staff, governmental members, quality assurance agencies and stakeholders.

Site Visit and Interviews by the Panel
A range of different stakeholders were interviewed by the panel:

• Members of EQAR (E4 organisations, social partners, governments); Governments that are not member of the EQAR association;
• Bologna Secretariat, Council of Europe, the European Commission and UNESCO/CEPES;
• Applicant and non-applicant quality assurance agencies.

The external evaluation report sets out the panel’s findings – based on the self-evaluation report, the site visit and the interviews – and make recommendations for improvement. The final external evaluation report was submitted to EQAR in September 2011.

In November 2011, EQAR organised the first Members Dialogue for all EQAR’s members and committees to discuss the results of the evaluation and give input to a follow-up and implementation plan.