Database of Precedents

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.2 Designing methodologies fit for purpose
Keywords	Involvement of students in developing the international accreditation and evaluation processes.
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that the processes and criteria for evalag’s work outside of the German accreditation system are designed by the Foundation Board. Based on the analysis by the review panel the Committee understood that students have little involvement in designing evalag’s own processes and are also not involved in the Foundation Board (see p. 30).The involvement of students in developing the international accreditation and evaluation processes was a recommendation from the previous ESG review of evalag in 2014, but has not yet been addressed.While the Register Committee welcomed evalag’s commitment expressed in its statement on the review report, no changes have been made as yet. The Committee was therefore unable to concur with the panel’s conclusion of substantial compliance, but considered that evalag only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.3 Implementing processes
Keywords	Implementation of follow-up procedures; accreditations carried out without a site visit
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“For accreditation in Germany, the Register Committee underlined that evalag retains responsibility for follow-up to take place, even if GAC makes the accreditation decisions under the new legal framework. This does not exclude that GAC actually implements the follow-up processes, as long as evalag has assured itself that this indeed happens.Given the small number of accreditations under the new legal framework thus far, it was not possible to analyse the actual practice at this point. The Register Committee therefore noted that this is a matter for further attention in future reviews of evalag.While the German legal framework potentially allows for an accreditation procedure to be carried out without a site visit, the panel understood from evalag that the agency did not plan to make use of that option. The Register Committee underlined that it might be helpful if evalag would point that out in its official documentation.Notwithstanding the above remarks, the Register Committee concurred with the panel's conclusion that evalag complies with the standard”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication of negative reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed it was flagged for attention whether evalag has moved to publish reports where the accreditation decision was negative. The review panel observed that evalag has had no negative decisions so far. The panel nevertheless confirmed that evalag’s clear policy is to publish all reports regardless of the outcome; the panel had no doubts that a negative report will be published. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.7 Complaints and appeals
Keywords	Independence of the appeals committee
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“In the previous decision to renew evalag’s registration the Register Committee flagged for attention how evalag has followed up the recommendation to separate the bodies in charge of appeals from the bodies deciding on accreditation, and to rule out parallel memberships.The review panel confirmed the independence of the appeals committee from the evalag decision-making body for accreditation. The panel further reported that parallel memberships are now ruled out. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Involvement of student in evalag’s governing structure, and overarching strategy
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The review panel found that evalag had no clear overarching strategy, bringing together planning, budgeting and risk assessment. The review panel noted stakeholders are formally involved in evalag’s governance through the Foundation Board, while the Board does not include a student member. Stakeholders are further involved in evalag’s work through participation in workshops and similar activities. The panel recommended that a student be appointed to the Foundation Board.Considering the concerns re. the lack of an overarching strategy and that one important stakeholder group is not represented in evalag’s governing structure, the Register Committee was unable to concur with the panel’s conclusion of substantial compliance, but considered that evalag only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.4 Thematic analysis
Keywords	Producing effective thematic analysis reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“When evalag’s registration was last renewed, the Register Committee noted that the current review should address whether evalag produced analyses of the general findings of all reviews carried out by the agency. The review panel described clearly how evalag conducts thematic analyses based on the findings from its reviews. The Register Committee therefore concluded that the flag has been addressed.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.4 Peer-review experts
Keywords	Involvement of students in panels.
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“The Register Committee noted that the agency has a wide range of experts involved in institutional, departmental and programme evaluations and that higher education institutions can make reasoned objections concerning the composition of the Expert Evaluation Committees (EEC - panel of experts conducting external evaluation).According to the analysis of the review panel, there is room for a more substantial role of students in evaluations, which is currently limited to a few issues (review report p. 41).While the panel found the arrangement for the appointment of EEC transparent, the panel also underlined that CYQAA should publish the procedures and criteria for the selection of all categories of experts and to explicitly define and communicate the criteria for the selection of student experts.In its additional representation the agency stated that it has published on its website the procedures and criteria for the selection of all categories of experts, including students. The agency further explained that the involvement of student experts is not restricted by the law or limited by CYQAA, and that the agency endeavours to support students’ participation in evaluation processes. The agency further included details about a recent training workshop organised for student experts in preparation for an external review. CYQAA added that the review panel extract referring to the participation of students on limited matters referred to a few students and that the review panel concluded that ‘all groups of experts feel that they participate on an equal footing’ (review report p. 41).14.The agency further argued that students are equal to the rest of the panel members, that they follow the same terms of reference, have the same responsibilities (defined in the “Guidelines for the Membersof External Evaluation Committees”), receive the same information, directions and clarifications and have the same rights and obligationsas the other panel members.15.Having considering the clarification provided by the agency, the Register Committee was able to follow the panel’s conclusion of (substantial) compliance with ESG 2.4.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication and quality of reports and decisions
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“At the time of the review not all evaluation reports carried out by CYQAA were available on the agency’s website. In its statement to thereview report the agency explained that it has begun to upload the missing reports. The Register Committee confirmed at the time of its first consideration of CYQAA’s application (June 2019) that most reports have been published (including reports with a negative outcome), however the Register Committee observed that a number of reports were nevertheless still missing.The Register Committee further noted that the decision taken on cross-border higher education provisions are not published by CYQAA, as the agency only provides an explanatory note related to the outcome of these reviews. While the panel found this practice satisfactory, the Committee could not follow the reasoning of the panel and underlined that the standard requirements specify that any formal decision based on the reports, should be published together with the report.The analysis of the panel further showed that the quality of reports varies in terms of evidence provided, depth of analysis and consistency and that expert panels have a very short schedule to write evaluation reports.In its additional representation CYQAA stated that, as it has previously committed to publishing all reports, that the agency has proceeded in doing so. The agency has further provided direct links to the different sections of its website where the reports of accredited institutions, programmes, joint programmes, franchised programmes are published, including the accreditations resulting ina negative decision. The CYQAA added that it has also published the official decisions and reports for all the inbound ‘franchised programmes’ reviewed by theagency. The Register Committee took note of the explanation concerning the time allocated to drafting of the report and welcomed the agency’s development of a more detailed guideline to support the consistencyand quality of reports (according to CYQAA additional representation and statement to the review report). As the Committee could verify the link for all published reports and decisions by CYQAA, including the evaluation outcomes of cross-border higher education provision of ‘franchised programmes’, the Register Committee concluded that CYQAA now complies with the ESG 2.6.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.7 Complaints and appeals
Keywords	Limited scope of complaints procedure and implementation of appeals and complaints procedures still to be assessed
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“While the panel confirmed that CYQAA has in place formal and clearly defined processes for higher education institutions to appeal against its accreditation decision, the panel also pointed out that the process of considering appeals is not independent as it is being handled by the Council of CYQAA. The Register Committee understood that following the amendments to the law, CYQAA would appoint a three-member ad-hoc committee to handle appeals on a case by case basis. In addressing complaints, the analysis of the panel shows that while CYQAA has an established practice for handling complaints there is a ‘tendency to rely mainly on informal communication, and that the arrangements in place do not add up yet to a clearly defined and formal’ procedure. In its additional representation CYQAA stated that its revised appeals’procedure sets out the appointment of a three member Advisory Committee of Experts (ACE) to examine the appeals that have been approved by the Council of CYQAA. Additionally CYQAA has written a formal regulation concerning the withdrawal of the accreditation of an institution, department or program of study offered by the institution. The appeal in these cases is being handled by an Independent ad hoc Advisory Committees. In addressing complaints CYQAA stated that the agency has published its complaints policy on its website and explained that individuals and organisations may issue complains about an accredited institution, department or programme. The Register Committee however noted that the concept of complaints is limited to general issues concerning higher education institutions and that it does not allow the possibility to address complaints related to the conduct of a review or complaints concerning the agency’s own processes. While the Register Committee welcomed the steps taken by the agency to address its shortcomings related to the functioning of the Appeals Committee and the implementation of the Complaints Procedure, the Committee underlined the limited scope of the complaints procedure and that the implementation and functioning of these procedures are yet to be considered by a review panel. Considering these limitations, the Register Committee could not follow the panels judgment and therefore concluded that CYQAA complies only partially with ESG 2.7.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.3 Independence
Keywords	CYQAA’s link to the ministry of education
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“In its analysis the panel noted that the agency’s Council members are appointed by the Council of Ministers upon recommendation from the Minister of Education and Culture (MOEC) who consults the Rectors’ Conference and relevant professional bodies (except the student member, who is appointed by POFEN).In terms of organisational arrangements, the Ministry is further involved in the recruitment process of the agency’s staff and in CYQAA’s financial services. The panel noted that “MOEC provides what the agency needs, as evidenced by a big budget increase and new staff, and the agency itself is proposing amendments on fees to the law which would further increase its income. While the panel noted that the provision of the quality assurance law and the strict rules for Council members reduces the risk of the Ministry’s interference with the agency's independence, the panel also stated that the current arrangements could be further revised, especially in reconsidering CYQAA’s organisational ties with the Ministry of Education and Culture. The Register Committee underlined the recommendations of the panel to ensure a clear separation of CYQAA from MOEC in its staff recruitment process, infrastructure and management of its own finances. In its additional representation the agency presented its proposed legislation changes and argued that this would safeguard the agency’s independence and the Council’s autonomy. Considering the proposed changes the Register Committee was not convinced that the changes in the appointment of CYQAA’s Council members would result in an increase of independence from the Ministry as, according to the proposed changes most of the Council members (eight out of eleven), are proposed (following stakeholder consultation in some cases) by the Minister. The Register Committee took note of the provision regarding the operational independence of the agency and welcomed the steps taken to increase its financial independence i.e. including a budget provisions that will allow the agency to hire additional staff, if the agency intends to do so. While the Committee noted some progress towards safeguarding the independence and autonomy of the agency, the Committee underlined the existing close interlinkage between CYQAA and MOEC and therefore concluded that the agency complies only partially with ESG 3.3.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.4 Thematic analysis
Keywords	Producing effective thematic analysis reports
Panel conclusion	Non-compliance
Clarification request(s)	–
RC decision	Compliance

“The review panel’s analysis show that CYQAA does not have a thorough and systematic analysis of findings from the evaluations conducted which could feed into a national quality assurance policy or guide quality improvement at higher education institutions. The panel nevertheless found that CYQAA’s Annual Report includes a few paragraphs referring to general issues and good practices in higher education in Cyprus. In its statement to the review report, the agency stated that it has collected the General Evaluation Reports from higher education institutions in Cyprus in November 2018 and has appointed a team of qualified members responsible for producing systematic analysis using the agency’s collected reports. The agency added that the findings of its thematic analysis are expected to be published within

2019. In its addition representation, the agency provided evidence of its implementation of thematic analysis and provided a link to its new publications. The agency has carried out an analysis of the “trends and dominant issues“ within i.e. e-learning programme evaluation, master programmes’ evaluation and the evaluation of short cycle programmes within colleges. The agency stated that it has made use of its thematic results, identifying weaknesses in aspects related to distance learning and recommending further attention to these issues. Considering the changes enacted by the agency, the Register Committee considered that CYQAA now complies with ESG 3.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.1 Consideration of internal quality assurance
Keywords	New standards published, but not reported to EQAR
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“The Register Committee noted that following the site-visit of its external review, CTI prepared and published (in February 2019) a new version of its standards and guidelines (R&O) for the accreditation of study programmes. While the revised R&O is expected to take into account the (minor) shortcomings identified by the panel in CTI’s coverage of ESG Part 1, these changes have not been included in the application to EQAR, nor reported via change report. The Register Committee therefore underlined that CTI is expected to report such substantial changes in its methodology immediately after they are adopted. CTI is thus expected to provide without delay a change report providing further information i.e. mapping of its new R&O against ESG Part 1.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.4 Peer-review experts
Keywords	Involvement of students in panels.
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“In its previous decision the Register Committee flagged CTI’s involvement of students in some of its review panels. The Register Committee noted that CTI works together with the French engineering students’ association, to ensures that CTI systematically nominates student experts in all its regular programme evaluations. The review panel also confirmed that in its CeQuInt evaluations, CTI’s panel include among its four experts also a student. ”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.5 Criteria for outcomes
Keywords	Lack of consistency in its decision making
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The panel’s analysis show that while the standards and criteria for accreditations are explicit and published, there are no clear deliberation rules detailing the basis upon which a specific decision is made and therefore concluded that consistency may not always be assured. In its response to the review report, CTI stated that it has taken a number of steps towards more consistency i.e. developing a new reporting template and updating its compliance table. The agency further declared that it will revise its rules for decision making for its different types of evaluation procedures following the analysis of its decision-making (after January 2020). The Register Committee welcomed the steps taken by the agency to address the shortcomings in ensuring consistency in its decision making but noted that the changes have not yet come into effect. The Register Committee therefore concurred with the panel’s conclusion that CTI complies only partially with ESG 2.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	Publication of full reports
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“In its previous decision the Register Committee flagged CTI’s practice ofnot publishing in full its review reports and not publishing its accreditation decisions. The panel’s analysis show that the agency’s practice it is to still publish summary evaluation reports. While the summary includes the decision, the main findings and recommendations of the review, the agency does not publish the full detailed evaluation results, called “minutes”. CTI argued thatit be difficult to ensure the consistency in the style and length of the differentpanel reports if it were to publish it in full, and that it would prove to be less valuable to its intended readership, including the reviewed higher education institutions which appreciate concise reports. In view of the panel’s recommendations of publishing the full accreditation reports, the agency stated that it has taken the decision that within the 2019-2020 external review campaign to make available the full reports for those higher education institutions that agree to having their report published (on a voluntary basis). CTI added that starting with its 2020-2021 external review campaign, that the agency is committed to publishing in full from that point all its review reports. The Register Committee acknowledged the actions taken by the agency towards the full publication of its reports, but stressed that the flag has not been addressed. As the agency does not currently meet the requirements of the standard (to publish full reports) at the Register Committee agrees with the panel’s conclusions that CTI complies only partially with ESG 2.6.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Involvement of students in the governance
Panel conclusion	Full compliance
Clarification request(s)	Panel (20/04/2025)
RC decision	Compliance

“The Register Committee noted that the legal regulation over CTI’s governing structure limits the involvement of students in the agency’s formal bodies. The Committee has asked the panel to clarify whether the agency has taken any steps towards changing its legal framework and involving students in its governance. In its clarification response the panel emphasised the positive collaboration and involvement of students in the work of CTI but also admitted that students are not involved in the final decision making processes of CTI. The panel noted that students participated in the work of CTI as part of review panels, meetings and training activities along with CTI members and that the student union is consulted in the revision of CTI’s frameworks and guidelines. The panel argued that the involvement in CTI’s ‘committee’ would be very time consuming, as members fulfil the equivalent of one fourth to one half of a full time position and that in practice this would not lead to a significant increase in students participation. The panel concluded therefore that, given the reactions of all stakeholders, including the students, CTI was in (full) compliance with ESG 3.1. Given that CTI in practice ensures the regular consultation of students, considering that students did not request to be represented in the CTI’s Commission and due to their limited capacity to meet the expected workload, the Register Committee was therefore able to concur with the panel’s conclusion that CTI complies with the standard. The Committee further concurs with the panel’s remarks that students (and international experts) can add value to the governance of CTI, even though their expected overall workload for CTI could be more limited. The Register Committee also underlined the recommendation of the panel to recognise the official status of students in CTI’s governance. The Committee added, that in order to allow for the possibility of students’ involvement in the governance of CTI, a change in the legislative restrictions should be considered.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.3 Independence
Keywords	Operational independece
Panel conclusion	Full compliance
Clarification request(s)	Panel (20/04/2025)
RC decision	Compliance

“Considering the organisational independence, the panel’s findings show that CTI outsources its accounting and management of human resources to the Conference of Deans of French Schools of Engineering (CDEFI), an organisation that represents the engineering schools that CTI accredits. CTI also shares its premises and one staff member with CDEFI. Considering the close interlinkage between CDEFI and CTI, the Register Committee asked the panel to elaborate how the agency ensures its operational and organisational independence. Regarding the operational independence the panel responded that procedures and methods are defined by CTI’s committees on the basis of preparatory work of working groups and that third parties are not involved in this processes. The panel was convinced on the basis of the self-assessment report and the meetings with representatives of CTI and its stakeholders, that independence was guaranteed. The panel commented that the financial administration of CTI is separate from the administration of CDEFI and while CDEFI administrates the contracts and selection of personnel, that the daily management of the staff are the responsibility of CTI alone. Considering the organisational independence, the panel argued that the independence of CTI from CDEFI remains guaranteed based on a signed agreement (as of June 2015) between the two organisation. The agreement defines the tasks expected from CDEFI and CTI, the annual fee in detail and stipulates how the independence of CTI and of CDEFI remains guaranteed. The panel added that the sharing of renting facilities in the same building is considered by all parties the panel spoke with as very positive. Having considered the clarifications from the panel, the Register Committee was therefore able to follow the panel’s conclusion that CTI complies with ESG 3.3.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.4 Peer-review experts
Keywords	absence of student reviewers in panels for NCE procedures and overseas accreditations
Panel conclusion	Substantial compliance
Clarification request(s)	Panel (21/10/2019)
RC decision	Partial compliance

“The involvement of students was flagged when A3ES was admitted to the Register. While the panel’s conclusion was that A3ES was substantially compliant withthe standard, the Register Committee noted that students were not involved in prior accreditation of study programmes (NCE) and international assessment teams. The Register Committee sought clarification from the panel in that regard. Inits response the panel explained that NCE was a compliance-check procedure based on juridic judgement and it involved only experts knowledgable in law. Furthermore, the panel considered A3ES arguments not to involve students in international assessment teams as reasonable andas exceptional cases that do not indicate that A3ES deviates significantly from the quality standards expected under 2.4.The clarification from the panel thus confirmed that students are not part of the panels for prior accreditation of study programmes (NCE) nor part of the panels for overseas compliance check. The Registered Committee considered that this arrangement did not meet the requirements of the standard and that students should be involved as expert panel members in all activities that involve an assessment by a panel of experts. Also for new study programme concepts and for existing programmes implemented in a new setting abroad, students may add a valuable additional and specific perspective to the process. Given the absence of student reviewers in panels for NCE procedures and overseas accreditations, the Register Committee concluded that the flag was not fully addressed and was unable to concur with the panel’s conclusion. The Register Committee concluded that A3ES only partially complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	2.6 Reporting
Keywords	readability and accessibility of reports
Panel conclusion	Full compliance
Clarification request(s)	–
RC decision	Compliance

“In the previous decision for inclusion, the Register Committee flagged for attention the readability and accessibility of reports. Based on the review report the Register Committee noted that the agency has made good efforts to address the issue.The Register Committee therefore considered that the flag has been addressed and concurred with the panel’s conclusion that A3ES complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.1 Activities, policy and processes for quality assurance
Keywords	Clear distinction between activities within and outside of the scope of the ESG
Panel conclusion	Full compliance
Clarification request(s)	Panel (21/10/2019)
RC decision	Compliance

“When A3ES applied for renewal, the Register Committee stated in its Eligibility Confirmation that the external review report should address how the agency clearly separates activities outside the scope of the ESG from activities within the scope of the ESG. As the report did not address the issueof clear separation of these activities, the Register Committee sought clarification from the panel. The panel clarified that A3ES' consultancy activities are not provided to universities or colleges, but solely focused on advising the national authorities or organisations on the design of national quality assurance policies, criteria and processes. Having considered the panel’s clarification, the Register Committee concurred with the panel that these activities are clearly separated from activities within the scope of the ESG by their very nature, and do not bear the potential for unclarity or conflict of interest that services rendered to higher education institutions do. The Committee therefore concurred with the panel's conclusion that A3ES complies with the standard”

Full decision: see agency register entry