Pilot studies

For the first pilot study, published in April 2019, Jeroen Huisman and Maria Manatos analysed institutional QA review reports exploring the following research questions:

• “Whether the ESG related to programme’s design, approval and monitoring are present in institutional external quality assurance reports;
• Whether quality assurance agencies are translating the ESG differently when performing institutional audits of HEIs. Particularly, we aim to understand how quality assurance agencies are translating the ESG, their main concepts and dimensions, and how audit panels are applying the national guidelines for institutional external quality assurance reports and how they are approaching the ESG.”

• “Globally, our research responds positively to both our main research questions. On the one hand, the research concludes that the ESG related to programme design, approval and monitoring are present in institutional external quality assurance reports. A quantitative analysis to the frequency of ESG’s concepts and dimensions confirms that the ESG ‘appear’ in the reports, though to different extents across countries.”
• “On the other hand, quality assurance agencies interpret and translate the main concepts and dimensions from the ESG differently, when performing institutional audits of HEIs. The differences are reflected in the ‘form’ of the reports, in particular, in the length, the focus and the guidelines for the reports. Indeed, quality assurance agencies have different foci and formulate different guidelines for institutional external quality assurance.”
• “Under an umbrella of ‘standardisation’ and ‘homogenisation’ it is then interesting to notice how the quality assurance agencies translate and interpret the ESG, developing their own national guidelines, and how the audit panels apply such guidelines in the reports.” 

For the second pilot study, published in October 2019, Jeroen Huisman and Maria Manatos analysed programme QA accreditation exploring the following questions:

• How do accreditation panels offer evidence for their findings with respect to:
  • encouraging students to take an active role in creating the learning process (ESG1.3)
  • and consistently applying pre-defined and published regulations covering all phases of the student “life cycle” (ESG 1.4)?
• Are there differences – with respect to the findings between countries and types of institutions?

• We conclude that globally the ESG 1.3 and 1.4 are ‘present’. […] However, they are subject to interpretation, first by the agency, which edits the ESG, though not radically changing them; second, by the panels, who also edit the ESG and present different evidences for the different dimensions inside the ESG
• re ESG 1.3, “the panels offer more explicit evidences for findings related to student’s assessment (mostly assessment procedures) and also (though to a lesser extent, and sometimes less explicitly) to the adaptability of teaching and learning (according to students’ paths, labour market needs, etc); and more implicit or even no evidence for findings related to students’ motivation, complaints, engagement, student-teacher relationship.”
• re ESG 1.4 “the panels tend to offer more evidences for findings related to the admission (mostly admission procedures) and progression (namely procedural aspects and ways to support students to progress in their studies), and certification, than related to informal learning and recognition (particularly important European references with regard to recognition). “