Database of Precedents

Application	Initial
Review	Full, coordinated by ECA
Decision of	19/06/2019
Standard	2.5 Criteria for outcomes
Keywords	limited track record, issues in one specific case
Panel conclusion	Full compliance
Clarification request(s)	Panel (19/11/2018)
RC decision	Partial compliance

“24. The Register Committee considered that there was a limited body of practical experience, even considering the track record built up under ELIA, as EQ-Arts has so far – as pointed out in the panel’s clarification – in its evaluations only “applied criteria in collaborative agreement with those of other bodies in all except the single enhancement carried out in 2017”. Moreover, EQ-Arts (and neither ELIA) has never made any formal assessment decisions itself.

25. The Committee noted the panel’s discussion of the one review carried out in Kazakhstan (see under ESG 3.3), which raised questions whether the agency had applied criteria in a consistent manner.

27. The Register Committee, however, concluded that EQ-Arts did not allay the concerns that stem from the review in Kazakhstan. Despite EQ-Arts' certainly larger track record of critical friend reviews, the review in Kazakhstan represents half of EQ-Arts’ total track record in terms of formal assessments.

28. Given the limited evidence for formal assessments and the fact that the review panel appeared to have had concerns in one out of the two formal assessments carried out so far, the Register Committee remained unable to concur with the panel’s conclusion, but considered that EQ-Arts only partially complies with the standard.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ECA
Decision of	19/06/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Separation of external QA and consultancy activities
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (19/11/2018)
RC decision	Partial compliance

“33. The panel considered that it was “clear from EQ-Arts procedures and documentation, and confirmed during the site-visit – and, importantly, ‘ethos’ - [...] that those involved in any consultation activities cannot be involved in any assessment/ reviews nor in the making of any accreditation decisions nor involved in any related appeals procedures” (letter by the panel).

34. In its additional representation, EQ-Arts elaborated on the nature of its enhancement-oriented reviews and formal assessments, respectively. The Register Committee, however, underlined that the question related to the clear separation between consultancy activities, i.e. activities outside the scope of the ESG, and either of the two types of reviews (enhancement- oriented and formal assessment) offered by EQ-Arts. While it might be regarded more crucial for formal assessments, the Register Committee underlined that safeguards should be in place to avoid conflicts of interest in both types of reviews.

35. Having considered the panel's clarification and EQ-Arts' additional representation, the Committee remained unable to identify a specific provision in EQ-Arts’ suite of documents that rules out the possibility of reviewing an institution or programme that was previously consulted by EQ- Arts.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ECA
Decision of	19/06/2019
Standard	3.3 Independence
Keywords	nomination of the Board members; limited track record
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (19/11/2018)
RC decision	Partial compliance

“38. The review panel found appropriate that "Potential Board members are identified / appointed by the Board not by public competition but [...] through informed ‘networking’".

39. Especially considering that there are no rules as to the Board’s composition otherwise (i.e. profiles or backgrounds of members to be appointed), the Register Committee considered that such an approach did not ensure sufficient transparency and did not provide a safeguard against one-sided third-party influence.

40. In its representation, EQ-Arts referred to the “skills audit” of its Board members and various documents that codify its independent status and that of its Board members. It further described how potential candidates are identified and appointed.

41. The Register Committee noted that EQ-Arts relies on the Board effectively recruiting its own successors, but without a public call. The Committee saw a risk in the fact that, due to the absence of any nominations by other bodies or a public competition, potentially suitable candidates cannot propose themselves unless contacted.

42. Bearing in mind the guideline to standard 3.1, i.e. “to ensure the meaningfulness of external quality assurance, it is important that institutions and the public trust agencies”, the Register Committee considers that the requirement of independence also implies a need for transparency and accountability.

43. The Register Committee underlined that most agencies ensure independence, transparency and accountability by receiving nominations to their Boards from different stakeholders. Such nomination rights exercised by different bodies constitute certain checks and balances, and prevent one single interest group from gaining full control.

44. The Register Committee considered that the absence of a nominations system paired with the informal approach to recruiting candidates did not ensure sufficient transparency and accountability to the agency’s sector.

46. The Register Committee had considered that, due to the small number of reviews carried out so far, the independence of EQ-Arts’ outcomes could only be demonstrated to some extent.

47. In its representation EQ-Arts reiterated its view that the number of reports should be considered sufficient to demonstrate compliance with the standards. The Register Committee accepted the argument considering the previous experience under the auspices of ELIA and the number of critical friend reviews that EQ-Arts carried out in addition to the formal assessments reviews; it considered that these indeed were sufficient to judge the independence of outcomes.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	2.1 Consideration of internal quality assurance
Keywords	Addressing IQA
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“Register, the Committee flagged for consideration whether SQAA paid greater attention to higher education institutions' internal quality assurance systems. The Register Committee considered that the revisions of SQAA's standards and the shift to institutional accreditation addressed the flag.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	2.3 Implementing processes
Keywords	Formal follow-up processes
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“The review panel considered there was a lack of a formal follow-up by SQAA to "touch base with HEIs” before the next cyclical re-evaluation/re-accreditation in case of unconditionally positive decisions. The Register Committee further took note of SQAA's response to the review report, setting out its approach to monitoring higher education institutions' internal quality assurance systems during the re-accreditation cycles.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	2.5 Criteria for outcomes
Keywords	clarity of the criteria for assessment
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (15/03/2019)
RC decision	Compliance

“Following the review panel's analysis that SQAA's criteria for assessment were not always clear and left room for interpretation, the Register Committee sought and received clarification from the panel on its conclusion as to the present standard. The Register Committee understood that SQAA's criteria were by and large perceived as clear, and that these remarks related to some – but not all or the majority of – criteria. It became clear that the panel's findings were more nuanced than the language might have suggested.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	2.6 Reporting
Keywords	publication of negative reports
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“Review panel further noted that reports from initial accreditation procedures with a negative outcome are not published and rightly stated that the publication of all reports is required by the ESG, in order to ensure full transparency. While SQAA's statement on the report held that all “reports (with the positive and negative outcomes)” were now accessible, it was not clear whether SQAA officially changed its policy to that effect. The Committee was unable to verify whether reports on initial accreditation procedures with a negative outcome were now published.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	3.3 Independence
Keywords	Operational independence
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (15/03/2019)
RC decision	Compliance

“The Register Committee obtained clarification from the panel concerning the remarks in the review report that the independence of the agency was not always fully understood by the Slovenian authorities, that “the Ministry might have interfered with the agency's work" and that “the private HEIs mentioned some cases of inconsistency and partiality” in SQAA's decisions. The Register Committee understood that the type of interference that had occurred (i.e. refusal to enter two accreditation decisions into the register maintained by the government) was now ruled out following changed responsibilities, with the register being under SQAA's control, thus strengthening the agency's independence. The Committee further noted that the panel found that the accusations by some private HEIs were not substantiated.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	3.4 Thematic analysis
Keywords	Implementation of thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee considered the panel's analysis as well as SQAA's statement on the panel report, referring to its recently published “methodology and procedure for drafting and disseminating system-wide and thematic analyses”. While the Committee welcomed the steps taken by SQAA to swiftly address the panel's recommendation, their actual implementation remains to be analysed within the next external review of SQAA.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	3.5 Resources
Keywords	financial sustainability
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“When SQAA was admitted to the Register in 2013, the financial sustainability of the agency after the cease of the funding from the European Social Fund (ESF) was flagged for attention. The Register Committee welcomed that the flag was addressed, as already determined in its decision of 18 March 2016”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	3.6 Internal quality assurance and professional conduct
Keywords	Effectiveness of agency’s IQA
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“When admitting SQAA to the Register, the Committee noted the 2013 panel's recommendation that SQAA systematise its internal quality assurance processes. According to the panel's report, SQAA has further systematised its internal quality assurance system.The Register Committee, however, also noted the review panel's critical appraisal of SQAA's interaction with the different stakeholders from different types of higher education institutions, and the question raised whether its quality policy was shared by all stakeholders. ”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.1 Consideration of internal quality assurance
Keywords	Insufficiently addressing ESG Part 1 in cyclical audits
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Partial compliance

“ The Committee, however, noted that the cyclical audit does not include consideration of ESG 1.3, 1.4 and 1.5, as indicated by the panel's comments[...]. While NOKUT stated that these standards were anyway checked in the programme or institutional accreditation, the Committee considered that universities – once they obtained self-accrediting power – only undergo cyclical audits. As a result, these standards were not regularly considered for universities.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.3 Implementing processes
Keywords	light systematic follow-up approach (compliant)
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“The panel noted that “actions taken by the institutions after the audit are not checked comprehensively”. While the panel noted that NOKUT provided an avenue for follow-up through seminars and conferences organised to discuss the audit findings and recommendation, it did consider those to be a systematic follow-up and noted that “the institutions do not feel obliged to implement the recommendations received”. The Register Committee considered that a very “light” approach to follow-up can be appropriate for (purely improvement-oriented) recommendations from an audit with an unconditionally positive outcome. Moreover, it considered that it was in the nature of a recommendation that there is no obligation to implement it.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.4 Peer-review experts
Keywords	student involvement in pilot procedures
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that there was no systematic inclusion of students in all expert groups for initial, paper-based accreditation as yet, but that the inclusion of students in these expert groups was currently piloted.[…] the Register Committee concluded that the flag has only been partially addressed so far, pending the pilot advancing to become a full regular part of the process.. The Committee further noted that there were no students included in the panel for the pilot on Combined Education and Research Evaluations, but appreciated that the panel was confident that students would be involved in case this activity became permanent.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.7 Complaints and appeals
Keywords	Absence of a formal complaints procedure
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted the panel's analysis that the complaints procedure “is less clearly defined”. While the Committee acknowledged that the prevailing culture of dialogue meant that institutions were aware of their opportunities to voice complaints, it considered that the possibility to complain and the related procedures should be explicitly spelled out, e.g. in the relevant guides for institutions, as required by the standard.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	06/12/2018
Standard	2.7 Complaints and appeals
Keywords	Functioning of the appeals processes; no possibility to appeal with the body that carried out the review
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that higher education institutions can file appeals for the decision taken by the AIC committees on the accreditation of study directions (CAS) and the licensing of study programmes (CLSP). In such cases the chairperson of the agency’s board reviews the conclusions of the Appeals Committee and takes the final decision on the appeal (review report p. 57). The Committee found that as long as the final decision may be changed by the chairperson of AIC, the integrity of the appeals process might be affected. In case of the institutional accreditations carried out by AIC, the Register Committee noted that appeals can be filed only against the Ministry of Education and Science’s (MoES) decision, in a court of law, following the Latvian Administrative Procedure and not against AIC‘s report and review processes. The Register Committee considered that external quality assurance processes should always include an internal possibility to appeal with the responsible body that carried out the review itself.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/06/2018
Standard	2.2 Designing methodologies fit for purpose
Keywords	flexibility of the accreditaiton system
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“While concurring with the panel's conclusion that NEAA complies with the standard, the Register Committee underlined the suggestion by the panel that NEAA should explore ways to make the accreditation system more flexible.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/06/2018
Standard	2.4 Peer-review experts
Keywords	Nomination of students in expert panels
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (30/05/2018)
RC decision	Compliance

“The Register Committee, however, noted that there might remain a residual risk of influence as long as higher education institutions could de facto exercise a veto right in the nomination of students to the NEAA pool of experts. While not impeding the agency's compliance with the standard (nor with standard 3.3) the Register Committee consider that NEAA should be mindful of that issue (also in regard to operational independence under standard 3.3).”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/06/2018
Standard	2.6 Reporting
Keywords	Publication of full reports
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (30/05/2018)
RC decision	Partial compliance

“The review panel noted that the expert groups' (EG) reports were “reflected” in the standing committee (SC) reports, but were not published as such. The panel clarified that the practice reflected NEAA's “division of labour”, that there was no substantial difference between the EG and SC reports, and that the SC was not able to include in its report “other findings than those of the EG”. […]In its statement on the review report, NEAA noted that it was now publishing the full EG reports. This could, however, not be verified by the Register Committee as the reports are only available on the Bulgarian version of NEAA's website.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/06/2018
Standard	2.7 Complaints and appeals
Keywords	Availability of appeals & clarify in the role of the 'Appeals Committee'
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that NEAA established an Appeals Committee that can make recommendations to the Accreditation Council (AC), but no binding decisions. The review panel, however, considered that the type of issues handled by that committee were complaints rather than appeals.The Register Committee concurs with the panel that the ESG should not be interpreted as requiring the establishment of more than one appeal systems. The Register Committee considers that the wording of the standard suggests an internal system, set up and operated by the agency. The Register Committee considers as the key requirement of the standard that the outcomes of external quality assurance are open to appeal. Furthermore, the possibility to appeal needs to be clearly defined and communicated to the institutions concerned, cover the possible grounds for appeal indicated in the guidelines and be effective and efficient in practice.”

Full decision: see agency register entry