Database of Precedents

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	2.5 Criteria for outcomes
Keywords	Inconsistent application of criteria
Panel conclusion	Compliance
Clarification request(s)	Agency (18/07/2023)
RC decision	Partial compliance

“The current set up brings confusion not only for the higher education institutions, but also for the

agency itself. The Committee found that agency’s criteria are not always applied consistently in the reviews and this leads to different standards being covered in the reports. During the clarification call, the agency explained that the SOP 2023 will be stabile - only reviewed and amended after 3 years again. The Committee welcomed these changes but found that the agency still enables usage of different SOPs – a practice that can lead to different outcomes and inconsistencies in the conclusions of its reports. This is especially problematic considering the regulatory function of the agency’s reviews for veterinary education.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Involvement of stakeholders
Panel conclusion	Compliance
Clarification request(s)	Agency (18/07/2023)
RC decision	Partial compliance

“The Committee was displeased to learn that EAEVE has not involved students and other stakeholders in the decision-making bodies since its registration on EQAR in

2017. While the Committee understood that involvingstudents from a narrow scientific field in the work of the agency could be a challenging task, it found that EAEVE could have found alternatives since its registration (e.g. establish stronger ties with stakeholders associations and or/create incentives for motivating other stakeholders to participate in the work of the agency, expand the involvement of students from closely related scientific fields etc.). The Committee found that the agency’s involvement of stakeholders, including students, in its governance and work is insufficient.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	2.4 Peer-review experts
Keywords	training, training of experts
Panel conclusion	Compliance
Clarification request(s)	–
RC decision	Partial compliance

“8. The Register Committee noted from the panel’s analysis that the trainings of experts have been reduced to only a briefing session taking place before the site visit. The panel’s analysis further show that the last training organised by NEAA took place in 2018 (before the Covid-19 pandemic).

9. The Register Committee underlined that the agency is expected to ensure that experts have the appropriate skills and competences to carry out external reviews and that such skills and competences are acquired through regular (periodic) trainings organised by the agency.

10. In its representation, NEAA provided information that it has started intensively working on preparation and implementation of several trainings focused on specific standards and criteria. NEAA also informed that future training sessions will be included in its Action Plan.

11. The Register Committee welcomed the steps taken by NEAA to address the earlier concerns, but noted that the Committee could not verify whether these training activities will ensure that all members of a panel will be systematically trained prior to undertaking an external quality assurance procedure. The Committee noted that these will remain to be determined in NEAA’s next external review.

12. The Register Committee therefore remained unable to concur with the panel’s conclusion, but considered that NEAA complies only partially with ESG 2.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	2.7 Complaints and appeals
Keywords	appeals and complaints, committee
Panel conclusion	Compliance
Clarification request(s)	–
RC decision	Compliance

“13. In its initial application for inclusion the Register Committee raised concerns regarding the lack of an internal appeals system within the agency.

14. The Register Committee noted that since the last external review of NEAA, nothing has changed in relation to the possibilities for higher education institutions to make an appeals with the agency.

15. The Register Committee further noted, that NEAA does not have its own appeals process nor a separate body in considering appeals and that the only existing appeals procedures are outside of NEAA’s remit, defined by law and under the legal competency of the Bulgarian courts.

16. The Committee considered that external quality assurance processes should always include an internal possibility to appeal within the responsible body that carried out the review itself.

17. In its representation the agency provided full documentation on its new internal provisions for complaints and appeals and for the functioning of its body the Complaints and Appeals Committee. The Committee noted that the new body is a standing committee within the agency, fully operative with permanent members and clear provisions outlined in the Statutes of the Complaints and Appeals Committee.

18. Having been able to verify that the agency provides both internal processes for complaints and appeals, the Register Committee finds that the initial concerns have been addressed. The Register Committee therefore concluded that NEAA now complies with ESG 2.7.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	stakeholders, stakeholders involvement, Accreditation Council
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“19. In its last decision, the Register Committee noted that the composition of NEAA’s Accreditation Council (AC) is dominated by representatives of the academia and that there is a lack of stakeholder involvement with no student or labour market/employment representatives involved in the work and governance of NEAA.

20. The Register Committee noted that while NEAA expanded the direct participation of stakeholders in its work i.e., including representatives of students and employers in the Expert Groups and the Standing Committees, that the composition of the Accreditation Council (AC) is still limited to only representatives of the academic community.

21. Despite the improvements done by NEAA in the involvement of stakeholders’ in some of its processes and bodies, the Register Committee considered the lack of stakeholder involvement (as per the requirement of the standard) in the core governance body of NEAA as of crucial importance.

22. The Committee further emphasised the recommendation of the panel to ensure the inclusion of a representative of students and employers/industry in the AC, including a representative of students (preferably with a legal background) in NEAA’s Ethics Committee.

23. In its representation, the agency provided information on the actions taken toward the government and parliament around the need for legislative changes. The Committee took note and welcomed the positive developments towards the involvement of stakeholders in the Accreditation Council, however at the time of the Register Committee’s consideration, no changes have happened, nor a clear timeline for such changes has been provided.

24. The Register Committee acknowledge that although NEAA has taken all possible actions to improve the involvement of stakeholders within relevant normative documents, the fact remains that the level of compliance is negatively impacted by the regulatory framework within which NEAA operates.

25. The Register Committee therefore concurs with the panel that NEAA complies only partially with ESG 3.1.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.5 Resources
Keywords	resources, financial independance
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“26. The panel noted that the main challenge for NEAA remains the fact that it cannot manage its own finances in a manner that will enable NEAA to ensure the best quality of its activities. Even if the majority of the funding comes from its external quality assurance activities i.e., mostly accreditation fees, due to the strict regulations and rules defined on national level, NEAA cannot access these revenues. The Committee concurs with the panel that this could have a negative impact on the sustainability and the quality of its processes.

27. The panel underlined that despite the improvements and increase of staff members since its last review, from 8 to 19 staff members, NEAA still had a high number of vacant positions.

28. The Committee emphasised the panel’s recommendations on the need for the agency to pursue with the Ministry changes in its financial management and to continue the recruitment of new staff to ensure optimal workload and implementation of external QA processes on time.

29. In its representation, NEAA reported of a further increase in its staff (i.e., from 19 to 23 employees), with recruitment for some of these positions currently underway. The Committee noted that the agency has taken active steps addressing specific proposals to increase pay rates to the Ministry of Education, the Ministry of Finance and the Prime Ministers office. Similar efforts are directed towards the Council of Rectors. Further, the agency stated that there are constant efforts made towards the responsible national authorities with a request to increase the budget of the agency.

30. The Register Committee welcomed the actions taken by NEAA. The Register Committee, however, noted that despite the improvements in NEAA’s permanent staff, the agency’s financial independence, due to external factors, remains constrained, and while the limitations in resources may not pose an immediate concern as to the sustainability of the agency, it may negatively impact the scope and quality of the activities undertaken by NEAA.

31. The Register Committee therefore concurred with the panel’s conclusion that NEAA complies only partially with ESG 3.5.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.6 Internal quality assurance and professional conduct
Keywords	internal, internal QA system,
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“32. The panel noted that NEAA started developing its internal quality assurance system introducing, some new mechanisms and procedures. However, the panel underlined that the elements of the procedures are fragmented and not fully structured, integrated and connected in a systematic way. The panel also noted the lack of transparency of the mechanisms and their results.

33. Despite NEAA’s efforts to develop its own internal quality assurance system, the Register Committee found that the process is still in a developmental phase. The Committee underlines the panel’s recommendation that all the elements of the internal QA system needs to be better connected, regularly implemented with the involvement of all internal and external stakeholders.

34. In its representation, NEAA informed that in 2023, the agency undertook an internal audit to identify the weaknesses and bottlenecks in its own internal processes and review procedures. The agency also provided details of the internal audit process and the series of recommendations that were outlined to correct the issues. NEAA further stated that necessary measures are being implemented to improve the functioning of the agency.

35. While the Register Committee welcomed the steps taken by NEAA, the Committee underlined that the issues outlined before remain to be considered and reviewed by an external review panel to determine the improvement in the functioning of NEAA’s internal QA system.

36. The Register Committee therefore concurred with the panel’s conclusion that NEAA only partially complies with ESG 3.6.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	2.6 Reporting
Keywords	Publication of reports
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“At the time of the site visit the

agency’s website was under development, and the access to all reports was

not possible. The Committee noted that the website was still not upgraded

and fully functional at the time of taking the decision - almost one year after

the panel’s visit. This circumstance prevented the Committee from verifying

whether all the reports are publicly available. The Committee

concurred with panel’s conclusion of partial compliance.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.3 Independence
Keywords	Operational independence; Independence of formal outcomes
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Committee learned that both the president and the vice president

of KAZSEE have voting rights in the two bodies in which they are members

of, i.e. the Accreditation Council and the Supervisory Board. In panel’s view

the “mixture of roles [of the President and the vice President] could make

the [decision making] system vulnerable”. The Committee could follow panel’s reasoning that the current

arrangements could pose threat for the operational independence and the

independence of formal outcomes.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.4 Thematic analysis
Keywords	Performing thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The agency does reports covering different

topics – however they are mostly related to projects, rather than findings

from the external quality assurance activities. The panel further highlighted

that the agency is aware of the importance of collecting and analysing data

from the review processes for its own development purposes, but also for

contributing to the system level policies. The Committee acknowledged that while the thematic analyses are

not taking place in a systemic and formal manner, KAZSEE is a relatively

new agency that demonstrates interest to further formalise the processes

for producing thematic analyses.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	03/03/2023
Standard	2.6 Reporting
Keywords	ex-post accreditation, Technical Committee, experts reports
Panel conclusion	Compliance
Clarification request(s)	Panel (13/02/2023)
RC decision	Partial compliance

“10. The Register Committee noted that the responsibility for the preparation of the external review reports for the monitoring and ex-post accreditation of study programmes is with the Technical Committees.

11. The Register Committee noted that the experts’ site-visit reports were not part of the final external review report and they were only available for the analysis of the Technical Committees. The Register Committee could not understand why the experts reports (in the case of the monitoring and ex-post accreditation of study programmes) is not published as well and therefore asked the panel for further clarification.

12. The panel explained (see minuted clarification 2023-02-13) that the practice of having a committee develop these reports is common among all Spanish QA agencies and it ensures the consistency of reports.

13. The Register Committee further asked the panel to clarify the forms of factual and non-factual allegations made by institutions on the reports of the Technical Committees. The panel explained (see minuted clarification) that the allegations have always been focused on the technical part of the reports (factual comments). However, the panel added that the agency should clearly differentiate within its procedure that the allegations deal only with factual errors and do not address diverging views and opinions on the statements and assessments taken in the report.

14. While the Register Committee welcomed the use of a Technical Committee to ensure consistency, the Register Committee could not follow the reasoning of not including or separately publishing such reports (i.e. as an annex). The Register Committee underlined that it can be of interest for the public to know the basis on which the final reports are being developed. In particular, such information is important to ensure the transparency in the Technical Committee’s decision making.

15. The Register Committee therefore could not follow the panel’s conclusion on the compliance with the standard, but concluded that ACCUEE complies only partially with ESG 2.6.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	03/03/2023
Standard	2.7 Complaints and appeals
Keywords	allegations, appeals, complaints
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“16. The Register Committee noted that the agency defines three different processes for allegations, appeals and complaints.

17. The allegations (considered as a form of feedback opportunity for the institutions) are dealt with by the Technical Committees, who is also responsible for drafting the reports (see also under ESG 2.6).

18. The appeals and complaints procedures are defined by law and are under the legal competency of the Canary government.

19. The Committee noted, that ACCUEE’s appeals and complaints policies are limited as the agency does not have its own appeals and complaints processes or a separate body in considering complaints or appeals.

20. The Committee considered that external quality assurance processes should always include an internal possibility to appeal and complain with the responsible body that carried out the review itself.

21. The Register Committee noted that ACCUEE intends to appoint an independent and permanent body, Guarantee Commission that will consider all appeals and complaints related to all procedures that are in the scope of the ESG.

22. The Register Committee welcomed ACCUE’s intention to set up a separate body to handle appeals and complaints, but the Committee underlined that such changes are yet to be implemented and to be reviewed. The Register Committee therefore concurred with the panel that ACCUEE only partially complies with ESG 2.7.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	03/03/2023
Standard	3.3 Independence
Keywords	regional government representation, Governing Board, Technical Committee
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“23. In regards to organisational independence, the Register Committee noted that the Governing Board, the body responsible for the approval of accreditation methodologies and criteria, has a strong regional government representation, with one third of the members nominated by the regional ministry The Committee further noted that the regional ministry is also responsible in appointing the president and vice-president of the Governing Board.

24. The Committee further noted that the regional government directly appoints the Director of the agency. The Committee found this problematic given the Director’s responsibility in ensuring the independent operations of the agency and in chairing all of the agency’s Technical Committees for ex-post accreditation of study programmes.

25. The Committee further noted that the agency is limited in its recruitment processes as this responsibility lies with the regional government (review report, p. 27).

26. Given the close link to the regional government, the Committee shares the panel’s view that ACCUEE should (in consultation with the regional Government) amend the current processes regarding the appointment of the Director as well as the legal frameworks definition on the composition and the proportion of governmental members in the Governing Board and ensure the recruitment processes are under the competency of the agency.

27. The Register Committee welcomed ACCUEE’s plans for addressing the panel’s concerns, but the Committee underlined that these changes are yet to be fully enacted. The Register Committee therefore concurred with the panel that ACCUEE only partially complies with ESG 3.3.”

Full decision: see agency register entry

Application	Renewal
Review	Targeted, coordinated by ENQA
Decision of	30/06/2023
Standard	2.3 Implementing processes
Keywords	study programme groups
Panel conclusion	Compliance
Clarification request(s)	–
RC decision	Compliance

“The Register Committee found the difference between the initial and re-assessments of study programme groups on the one hand and the phasing out of assessments of study programme groups on the other hand not to be completely clear.

The agency explained (see clarification of 2023-06-06) that the initial and re-assessment procedures remain compulsory for the opening of any new study programme groups. For existing study programme groups that have gone successfully through multiple assessment processes, the assessments of individual study programme groups are being phased out for a sample of these programmes, within the new model for institutional accreditations.

Having considered the agency’s explanation and the implementation of the new procedures, the Register Committee concurs with the panel’s conclusion of compliance.”

Full decision: see agency register entry

Application	Renewal
Review	Targeted, coordinated by ENQA
Decision of	30/06/2023
Standard	3.5 Resources
Keywords	state budget
Panel conclusion	Full compliance
Clarification request(s)	Agency (06/06/2023)
RC decision	Compliance

“The Register Committee noted that the proportion of HAKA’s activities financed through state budget has been decreasing. This is caused by the fact that HAKA has taken on activities for other education levels/types not falling under the ESG, which are financed through projects.

The Register Committee understood from the agency’s clarification (see minuted response of 2023-06-06) that the financial concerns do not affect the agency’s ability to carry out its regular external QA activities.

The Register Committee therefore concluded that the agency continues to comply with ESG 3.5.”

Full decision: see agency register entry

Application	Renewal
Review	Targeted, coordinated by ENQA
Decision of	03/03/2023
Standard	2.1 Consideration of internal quality assurance
Keywords	coverage of ESG Part1
Panel conclusion	Compliance
Clarification request(s)	Panel (10/02/2023)
RC decision	Compliance

“11. The Register Committee noted that a protocol for the assessment of transnational education in the Netherlands (NL) came into effect in

2018. The Committee however could not find any information on whether NVAO-NL has addressed the standards of ESG Part 1 (ESG 1.1-ESG 1.10) in its new protocol and has therefore sought further clarification from the panel.

12. The panel explained (see clarification letter) that a transnational education programme may be provided on the condition that the programme abroad is equal to the one accredited in the Netherlands. This may only concern programmes that have already been accredited in the Netherlands. Given this condition, the review panel explained that the study programmes abroad follow the same accreditation protocol as the programmes accredited in the Netherlands.

13. The Register Committee understands that ESG Part 1 has been verified by the panel for the renewed 2018 NVAO-NL assessment framework and noted that a clear link between the institution’s internal and the NVAO’s external quality assurance procedures was ensured.

14. Having considered the clarification provided, the Register Committee can now follow the panel’s conclusion of compliance with the standards 2.1.”

Full decision: see agency register entry

Application	Renewal
Review	Targeted, coordinated by ENQA
Decision of	03/03/2023
Standard	2.5 Criteria for outcomes
Keywords	deviation from the outcome of a panel’s report
Panel conclusion	Compliance
Clarification request(s)	Panel (10/02/2023)
RC decision	Compliance

“15. The Register Committee noted that NVAO-NL may modify a recommendation for a positive outcome in a panel report, although it has never so far questioned this. The Committee was unclear on the situations that may lead to a deviation from the outcome of a panel’s report and whether such deviations are documented.

16. In its clarification response (of 10/02/2023) the review panel explained that NVAO-NL may occasionally seek additional information from panels and in a limited number of cases, and after due deliberation may expand conditions or deviate in a minor sense from the panel’s advice. Such changes may be done by NVAO-NL to reduce the subjectivity of reports and ensure the consistency of recommendations as well as of the final outcome. Deviations from the final recommendation of the panel have not happened yet, but according to the agency’s procedure these changes are documented in the final published decision by NVAO-NL.

17. Having considered the clarification provided, the Register Committee can now follow the panel’s conclusion of compliance with the standard 2.5.”

Full decision: see agency register entry

Application	Renewal
Review	Targeted, coordinated by ENQA
Decision of	03/03/2023
Standard	2.6 Reporting
Keywords	readability of reports, delay in publication
Panel conclusion	Compliance
Clarification request(s)	Panel (10/02/2023)
RC decision	Compliance

“18. In its previous renewal decision, the Register Committee stressed the delay in NVAO’s publication of reports and noted issues related to the readability of reports.

19. The panel’s findings show that NVAO has since its last review introduced instructions and templates for reporting and that the readability of submitted initial assessment reports is also checked by NVAO. The panel further confirmed that the publication of reports was done without any more significant delays, but suggested setting up an automatic uploading system of NVAO-NL reports (as it is done for NVAO-FL).

20. The Register Committee welcomed the improvements in the agency’s reporting and concurred with the panel’s conclusion that NVAO now complies with the standard 2.6.”

Full decision: see agency register entry

Application	Renewal
Review	Targeted, coordinated by ENQA
Decision of	03/03/2023
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	separation between activities that are within and outside the scope of the ESG
Panel conclusion	Compliance
Clarification request(s)	Panel (10/02/2023)
RC decision	Compliance

“21. In its Terms of Reference for the review the Register Committee asked the panel to consider how the agency clearly separates between activities that are within and outside the scope of the ESG, in particular considering the ‘Assessment of Quality Agreements in the Netherlands’ and the ‘Assessment of the quality of Training Schools’. While the panel concludes that there is a clear separation between NVAO’s activities within and outside the scope of the ESG, the Committee could not find the argumentation to support the panel’s conclusion and has therefore sought further information.

22. In its response (see minuted conversation), the panel explained that the separation between the agency’s activities that are within and outside the scope of the ESG did not pose any concern.

23. Considering the Assessment of quality agreements in the Netherlands the panel stated that the activity does not address the teaching and learning aspects within higher education and that the focus of the assessment is on how institutions (plan to) spend the so-called study advance grants. The panel added that the agency’s protocol or description does not misrepresent the activity in any way (i.e. referring to ESG or EQAR registration).

24. Considering the evaluation procedure for teacher training schools, the panel clarified that the activity does not address or evaluate the teaching and learning in higher education, but it assesses the collaboration between schools for primary and secondary education and institutions for teacher training.

25. Having considered the clarification provided, the Register Committee can now follow the panel’s conclusion of compliance with the standard 3.1.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	03/03/2023
Standard	2.4 Peer-review experts
Keywords	Student involvement in panels
Panel conclusion	Non-compliance
Clarification request(s)	Agency (14/02/2023)
RC decision	Partial compliance

“Further efforts made by the

agency to engage students in the review panels - in particular

students have participated in the first reviews by the time the Register Committee analysed the application. While the panel’s

conclusion of non-compliance did reflect accurately the situation at the

February 2022’s site visit, the Committee concluded that the agency is now

partially compliant with the standard. The Committee, however, underlined

that further evidence of the actual involvement of the students in the panels

will be needed and has to be thoroughly analysed in the next review of HAHE”

Full decision: see agency register entry