Rapid Review of National Qualifications Experience 2020

Abstract

Summary of findings - This report draws upon a range of evidence, including stakeholder testimony (generated in panel and individual interviews) and analysis of relevant documentation (including government and SQA emails). - SQA, the government, local authorities and schools faced an extremely difficult set of circumstances, within which there were no easy solutions. In this context, a workable system for qualifications, the Alternative Certification Model (ACM), was developed. This was based on three core principles and four stages. - All parties involved in the process were found to have acted with integrity, with the best interests of students in mind. - Respondents (teachers, lecturers, head teachers and local authority officials) generally found that SQA guidance was clear and useful. - The generation of estimated grades, while clearly undertaken with integrity in the majority of centres, has been subject to variation (in the types of evidence available, the processes followed for internal moderation and the support given by local authorities), which has impacted on reliability and consistency of assessment at this stage. - The statistical approach to moderation could have been more transparent earlier in the process, and moreover it has led to anomalies in grade adjustment, especially at the level of subject cohorts within centres and individuals. - There is widespread criticism by respondents of SQA for a perceived lack of transparency and a failure to engage in participative development of solutions with stakeholders. - While the application of the appeals process offered an in-principle technical solution to address these anomalies, it paid insufficient attention to the severe impact on those students obliged to undergo it (in terms of mental health and wellbeing, missed opportunities to transition into Higher Education, etc.). - Principles relating to what data is appropriate to be held by certain organisations at certain points in time.(i.e. SQA, the Scottish Government), which make perfect sense in normal times (e.g. arrangements around data sharing), appear to have impeded the development of actions that might have led to an earlier anticipation and mitigation of subsequent problems. - The equalities implications of an over-reliance on a statistical approach, premised on comparison with historical cohort data, had been raised repeatedly from April onwards, but seem to have been under-emphasised by both the government and SQA until late in the process. - Many stakeholders believe that, subsequently, opportunities were missed (or dismissed) to engage in qualitative moderation of the statistical process (e.g. sense-checking of anomalous cohort patterns by local authorities). - There has been an erosion of trust/confidence in SQA amongst teachers and young people, and damaged relations in some cases between young people and their teachers. - Communications (with professionals and with young people and their families) has been a constant source of criticism. - Our overall assessment is that, despite the extremely difficult environment for decision making, there are points in the process where different decisions may have led to better outcomes and at least partially avoided the controversy that ensued in August 2020. Of course, we are making this observation with the benefit of hindsight, thus our primary intention is to illustrate how the system can benefit from lessons learned in 2020 to avoid a similar predicament in 2021.