Education and Skills Committee

Meeting date: Wednesday, January 22, 2020

Official Report 611KB pdf

---

Scottish Qualifications Authority: Performance and Role

Agenda item 2 is on the performance and role of the SQA. We welcome from the Scottish Qualifications Authority: Fiona Robertson, who is the chief executive; John McMorris, who is the director of business development; and Robert Quinn, who is head of English, languages and business.

I invite Ms Robertson to make an opening statement.

I thank the committee for the opportunity to attend today to discuss again the SQA’s role and performance following our appearance at committee on 11 September 2019 and related correspondence, most recently on 19 December, which covered a range of issues.

When I appeared before the committee on 11 September 2019, it was very early in my tenure as chief executive—just six weeks had passed since my appointment. During that time, my focus was naturally on the successful completion of the 2018-19 examinations diet in early August and the immediate post-results processes. However, since then, I have been able to focus on other issues, including on a number that are of specific interest to the committee. I hope that my response of 19 December shows the progress to date and the direction of travel, and I will highlight a number of those issues now.

On international expenses, I have taken action to review our travel policy and to address concerns about international travel costs. A revised travel policy has been in place since 22 November 2019, in line with the timetable that I set out in September. Key changes to the policy include that business class is permitted for international air travel only when the one-way flight time is 10 hours or more, with authorisation for international travel and accommodation to be agreed at director level or above before booking a trip; and Foreign and Commonwealth Office guidelines apply for overseas accommodation and subsistence, which provides cost limits that SQA staff should follow.

The committee had expressed concerns about the SQA’s available management information about individual travel costs following a number of freedom of information requests. A number of data sources have been merged and the SQA is now able to report on all sources of financial information regarding individual staff travel and expenses. Additional systems integration will take place by the end of this financial year. Thereafter, the international travel costs of senior staff across the SQA will be published quarterly.

On our international work, the SQA and its predecessor bodies have been working internationally for more than 25 years, reflecting our globally recognised expertise in qualifications and assessment development and delivery. We have used our international engagement as an opportunity to increase respect for, and understanding of, human rights. We support and align with the Scottish Government’s international trade and investment aims, as reflected in the national export growth plan, “A Trading Nation—a plan for growing Scotland’s exports”.

The committee has shown particular interest in our work in Saudi Arabia. The Department for International Trade highlights Saudi Arabia as one of the United Kingdom’s important trading partners, with education as a key sector, and Scottish Development International continues to maintain an office in Saudi Arabia to promote international trade with Scotland. However, I recognise that it is important that the SQA, in line with all other public bodies, follows guidance from the UK and Scottish Governments in considering the markets in which we operate. We believe that we do, but I have asked the director of business development to consider whether any additional policy and associated guidance for assessing our international engagement is needed in order to align it with the Scottish Government’s approach to promoting human rights internationally and be based on the United Nations guiding principles on business and human rights.

I want to ensure that the SQA can make fully informed decisions about its operations in all its international markets and be an exemplar in considering human rights issues. We will review and refine our existing due diligence processes to take account of that policy and guidance. Due diligence assessments will be undertaken for all prospective and existing international partners. As part of that process, we will undertake a human rights impact assessment that will consider each country and partner with which we propose to have a business relationship. It is intended that that process will provide additional assurance that human rights are considered and respected.

That work is well advanced, and my colleague John McMorris will be happy to provide more detail on it this morning, should the committee request it. However, it is important that we engage with the Scottish Government and other agencies in that work, so I hope that the new arrangements can be finalised and put in place before the beginning of the next financial year. I would also be happy to engage further with the committee on the matter.

On marker withdrawals, I have provided further information on the 794 withdrawals from the 12,450 marker appointments at the start of the 2019 marking period. I understand that the committee will want to be assured that we have sufficient markers in place for 2020. Marker recruitment and re-recruitment, which are a key feature of our annual processes, are progressing well. I am confident that we will meet the total marker requirements for this year. We will continue to monitor withdrawals.

I have also outlined for the committee some review work that the SQA will be undertaking at national 3 and national 4, as part of our on-going review processes. Those courses have not been reviewed for more than five years. The review will look at a range of factors, including the assessment standards, as benchmarked against the relevant level in the Scottish credit and qualifications framework; the currency and appropriateness of the key skills and knowledge that are required for each course; feedback on the delivery of the current courses, such as the management of the assessment approach, including the added value unit for national 4; and progression to and from national 3 and national 4.

As part of that review, we will consider the skills and content alignment between the various levels. We will be informed by the practicalities of delivering the courses, including bi-level or multilevel classes. If the review suggests that an adjustment of content is required to more smoothly align each level of a national course while enabling appropriate national standards to be maintained, that will be considered when revised arrangements are developed.

We are determining the scope of that work, which will be finalised when our resourcing position for 2020-21 is clear, but I hope that the work will be completed by June 2021 and that any further changes implemented by academic year 2022-23. That work will be further informed by any wider education review work that is done. As agreed by the curriculum and assessment board, there are no plans at present to change the overall design principles of national 3 and national 4.

On post-results services, I have provided further information to some committee members in correspondence and have also provided more information to the committee about the development of our approach, which included significant engagement with and support from the wider system.

The SQA has received no feedback to date from any centre saying that it has been unable to submit an evidenced-based marking review request due to cost considerations. I note the committee’s reference to the overall marking review uptake statistics by sector, and I am happy to undertake some additional engagement with the Association of Directors of Education in Scotland, the Scottish Council for Independent Schools and centres on the criteria that are used to make such requests. That will allow the SQA to determine whether there are any issues to do with access to the service for centres and to ensure that the cost of an unsuccessful marking review is not deterring centres from making a valid request in the first place.

I am conscious that, since the SQA’s appearance last September, the committee has published its inquiry on subject choices and there has been further parliamentary scrutiny of educational performance, including of the SQA’s results.

Gayle Gorman—the chief inspector of education—and I wrote to the committee jointly in response to the inquiry on subject choices. We highlighted the importance of our joint work and our commitment to work with the Scottish Government as key national partners. That will include forthcoming review activity.

My submission to the committee last September contained my commentary, as chief examiner, on the SQA’s results. I highlighted that some variation in attainment is to be expected between subjects over time and that, in 2019, the attainment rate at national 5 increased and the attainment rate at higher and advanced higher levels decreased.

High-quality learning and teaching remain a fundamental component of the successful delivery of qualifications, to ensure that candidates are well prepared across all aspects of every course that they undertake. The SQA will continue to work with teachers, lecturers, schools and colleges to support them in the delivery of our qualifications.

Thank you for the opportunity to clarify those points. I hope that my response fully answers the questions that the committee has raised. Since my appointment, six months ago, I have been working to ensure that the SQA builds on its strengths but is also open and responsive to any issues that are raised. I am committed to working through a range of issues, to the benefit of Scotland’s learners.

Thank you very much. We will move to questions from the committee.

Good morning. I highlight to members that I was a standard grade and higher marker for the SQA in a previous life.

Before I ask a few questions on post-results services, I note that Fiona Robertson mentioned a review that the SQA is carrying out into the progression to and from national 3 and national 4. Is that part of the Government’s senior phase review, or is it part of a separate review that the SQA is undertaking?

As well as delivering our qualifications, we have responsibility for their on-going development and review. In my letter to the committee of 19 December 2019 and in my opening remarks this morning, I have outlined that, as part of our on-going review processes, we will do work on national 3 and national 4. That work has not been part of the work that we have done over a number of recent years on the further revisions to national qualifications.

I should highlight—as I think I did in my letter of 19 December 2019—that that review work does not impact on the decisions that have already been made in relation to national 4. We will consider carefully and make sure that it does not pre-empt any related wider education review work that is being undertaken by the Scottish Government.

We will keep in close contact with the Scottish Government, other agencies and the wider sector on this work, as you would expect. I very much see it as part of our responsibility for on-going review, and the issue that you highlighted concerning progression is only one aspect of that.

It is helpful to get that on the record.

The post-results services replaced the appeals service, through which a teacher could use a preliminary result to appeal on behalf of a pupil, which could lead to a pupil’s grade being boosted.

We now have a system that carries out clerical checks. Please correct me if I am wrong, but that checks that all materials have been marked and that the marks have been allocated correctly. We also have marking review, which checks whether the marking is in line with the national standard and that the correct results have been entered.

Why was the previous system replaced with a new system?

One crucial element that you did not mention, which is an important part of our relatively new system, concerns exceptional circumstances. For those candidates who, for whatever reason, have either been unable to sit an exam or have been in distressing circumstances on exam day, we offer to look at the evidence that is available to us, with the aim of certificating on results day. The exceptional circumstances service kicks in for those young people who most need it, due to circumstances that are usually beyond their control on the day of an exam.

The post-results services are available for young people who, on results day, get a result that their school has not anticipated. As you have described, its different elements are a clerical review and a marking review. For those candidates who might have university or college entrance issues, there is also a priority marking review, so that the procedure can take place quickly.

The introduction of the post-results services was some time ago—it predates my taking up my role at the SQA. If we look back, we can see that the previous appeals service was used very significantly by the system and, as I have highlighted, did not differentiate between candidates who may have been in distressing circumstances on the day of their exam and those who do not get the result that they wanted. As you rightly highlighted, the prelim—and alternative evidence—was taken as very significant factors in the determination of any change to a grade.

The new process that is in place was carefully considered and discussed across the system, with teachers and the directors of education. There was a process of engagement around the new approach, which has been in place since 2013.

As I have highlighted in my opening statement and in my letter to the committee, I am very happy to have conversations about the process and the services that are in place. For example, there has been a lot of focus on the fact that there is a charge if there is no change in grade as a result of a marking review, whereas there is no charge for the exceptional circumstances service.

I highlight the point that any marking review must be carried out by the school—by the centre. We absolutely rely on the professional judgment of teachers to make appropriate decisions in determining whether a marking review is appropriate, or indeed whether an exceptional circumstances service is appropriate. That is a really important aspect of the system more widely and of that service in particular.

10:15  

Can you give us a wee bit more evidence about what the SQA might consider when it comes to the exceptional circumstances service? Presumably, you would not look at the prelim result. I ask this because many schools have done away with the prelim, so I am wondering whether there is the potential for inequality.

We will look at a variety of alternative evidence. We would not rely solely on the prelim. Robert Quinn might wish to say a little bit more about that and to give examples for some of the subjects for which he is responsible.

That is a good point, Ms Gilruth. One of the key drivers of the new process is our looking at a much wider range of evidence. One of the disadvantages of the old system was that it penalised the candidates for which it was properly designed: those candidates who had suffered some exceptional circumstance, such as being absent from an examination. If we looked at the prelim evidence and we had 70,000 other appeals, we would have to benchmark the evidence across schools in order to be fair, and if there was any deficiency in the prelim evidence that was presented, candidates would often receive a no award. We discussed that at our board of management and in various committee structures, and one of the key drivers was to get a fairer deal for candidates who were properly disadvantaged by the system, such as the absentee candidates that I have mentioned.

We now look at a much wider range of evidence. For a period, we did not use the term “prelim” in order to get people away from the idea that pupils always have to take a prelim, and then a second and third prelim, because that approach brought with it some negative aspects associated with teaching to test. We would consider class tests and examples of work that pupils had done in examination and non-examination conditions, as well as any mock examinations. We are less focused on the prelim and its structure, and on whether the prelim replicates the exam, than we are on the candidate and on what that tells us about where the candidate sits on the grade spectrum. That is a subjective part of the process, which relies a lot on the professional judgment of examiners, but it is an area that has been improved vis-à-vis the old approach.

Fiona Robertson highlighted the costs. Could you tell the committee how much it costs to have a clerical check? How much does a marker’s check cost?

I do not have a figure to hand, although I can provide that information separately. I think that it is about £30. Do you know, Robert?

It is about £30. It is about a third less than for the other UK awarding bodies.

Why was a cost introduced? Previously, the appeals service was free.

We had to consider the volume of requests that we received. Effectively, the charge would operate as a disincentive for those schools that might be tempted to put in a request where there was no evidence that the request would be successful. There was a practical set of issues to do with charging that we had to consider.

There is a cost attached to our services. My understanding is that there was quite a lot of discussion about charging before 2013 and that approach got support from the system. We were mindful, in all parts of the system, of the need to ensure that the charge or cost did not act as a disincentive. On that basis, quite a lot of work was done with the Association of Directors of Education in Scotland to put guidance in place for local authority schools. That guidance is also used by the Scottish Council of Independent Schools to advise its schools on what a good and strong process for post-results services might be.

Do you see any inequality in schools in the private sector having more money at their disposal, which means that they can apply for the service, whereas schools in the state sector might not have the funds to apply? In your opening statement, you said that you have no evidence that schools are not able to afford the service. Such evidence has not been fed back to you.

That has not been fed back in any of the discussions that I have had about the issue since my appointment. As I said to Iain Gray at the committee, and as I indicated in my letter, I am very happy to have further discussions on the matter.

I reiterate a couple of points. The ADES guidance is really important in making it clear that costs are not a consideration when deciding whether to take forward a review. This is about the professional judgment of teachers. There are a number of things that we need to keep in mind.

I have a couple of supplementary questions. You said that it is about the professional judgment of teachers. That concerns me because, previously, surely teachers used their professional judgment to decide who was presented for appeal in the same way. We are talking about the cost being a disincentive to people coming forward. I think that the committee is concerned—I am certainly concerned—that teachers in schools in which there are budget concerns would feel pressure not to present cases for appeal.

You said that the charge was introduced in 2013. How long are exam papers retained for? Is there a time limit on when an exam can be appealed?

There is an annual cycle, with dates, in relation to post-results. My understanding is that we retain scripts for a certain period.

I think that we retain the scripts until the January of the following year. If the examinations are in the summer, the scripts are destroyed the following January. If someone has a live case as part of the post-results service, we keep their scripts for a year.

That is right. We keep them for a further 12 months.

That is the process.

Do you want to comment on the charge being a disincentive? Some schools might feel pressure not to present cases for appeal.

I understand why the committee is concerned that the cost might act as a disincentive. However, I want the committee to be reassured by the ADES guidance that has been provided to schools and by the feedback that I have received on the issue. I do not think that cost is acting as a disincentive to putting forward an appeal.

That said, I have undertaken to have further conversations with ADES centres and the Scottish Council of Independent Schools to consider whether there are any issues that we need to pursue. At this stage in my tenure, I am quite open to having those discussions with the system. The last thing that I want to do is to support an inequity. However, as the chief executive of a qualifications authority, I must be cognisant of the choices that we need to make on resources and so on. Those discussions are part of the discussions that we have with Government. The charge for the post-results service is less than the cost of providing the service. We have to be open to thinking about some of those issues more generally.

Are you aware of whether the costs are being picked up by individual school budgets or by local authority education budgets?

Usually, the costs are picked up by local authority education budgets. In their feedback to us, schools have said that there has not been any pressure on people to adjust their criteria because of cost considerations. The ADES guidance is really strong in telling people to reflect on the results. If something sticks out as being unusual or not to expectation and is a cause for concern, it would be an area to home in on. My experience and the feedback that I have received is that, in the main, schools are following the ADES guidance. The cost element is not for every marking review; it is just for when the review is unsuccessful.

The issue was part of the engagement, and it was discussed extensively by our external engagement group, which took evidence from the other UK award bodies. People were comfortable that there was to be a charge. However, the charge was pitched carefully, so that it would not be punitive and act as a disincentive. The further engagement work will check that out.

I appreciate that you have not had any feedback that cost is an issue, but there is other evidence. In the last year of the previous scheme, 5.7 per cent of pupils from independent schools and 6.5 per cent of pupils from education authority schools used the post-results appeal procedure. In 2018, which is the last year for which we have data, the figure for education authority schools was 2.4 per cent and the figure for independent schools was 7 per cent. Therefore, there has been a complete reversal. If that is not an indication that the disincentive, as you have described the cost, is acting as a disincentive more for pupils in state schools, can you provide an alternative explanation?

I have said before that I acknowledge that there is a difference in the requests as a percentage of entries from independent schools and from education authority schools. We also provide information on other centre types and colleges, and they also have differences.

It is quite difficult to get underneath the difference in a granular sense. I can only emphasise that I am happy to have further conversations about that with a number of parties, and I can only repeat that we are not getting feedback from local authority schools that cost is acting as a disincentive to putting through a marking review. If a marking review is successful, there is no charge.

Of course. However, I am not sure that it is hard to get beneath the difference. In 2013, a higher proportion of pupils from state schools used the service than pupils from independent schools did. In 2014, immediately at the point at which the change was made, three times as many pupils from independent schools used the service compared with pupils from state schools. It seems blindingly obvious to me what happened. The disincentive that you have described—you called it a “disincentive”—has acted disproportionately and unfairly on pupils in state schools. If you have an alternative explanation, I would be happy to hear it.

I have said that I am happy to look at that set of issues with centres. Indeed, the decision to introduce the process that we introduced in 2013 was not one that we took alone; it was taken in consultation with the centres, including local authority schools.

I am open to considering the issues more broadly. The SQA does not want to support an inequity, but I am conscious of the need to ensure that we are able to take our work forward and that we make it clear that considering whether a marking review is required involves a professional judgment at the centre.

There was a wee bit of nuance in the old system in that there was not a straightforward free-for-all with regard to whether people could choose to appeal. They had to be eligible for appeal—there had to be a difference between the estimate and the result. We did some research under the old system and found that people from the independent sector were more likely to convert an eligible appeal into an appeal because the schools were presented with a list of candidates who were eligible for appeal, and they were more likely to convert that eligible appeal into an appeal. It is not as straightforward as being about the percentage of entries; there was a subset of data under that, which related to eligibility for appeals.

One of our concerns was that people use any system in different ways, and we noticed that that was the case. Although the headline figures on the engagement rates with the old appeals process are similar, the independent sector was more likely than the state sector to convert an eligible appeal into a real appeal.

10:30  

That would explain why the independent schools figure was lower under the old system, but it does not explain the plummeting use of the post-results service in the state sector, does it?

I would not say that it is a “plummeting use”—

The figure has gone from 6.5 per cent to 2.4 per cent. It is a third of what it was. I think that that counts as plummeting.

I go back to Ms Robertson’s offer to do more work on what lies behind the difference, which is entirely reasonable.

I am happy to do that.

You are offering to ask ADES whether its authorities follow its guidance and to ask the independent school sector whether it thinks that it has an unfair advantage. Call me cynical, but I think that ADES will say, “Yes, cost isn’t a factor, because that is what our guidance says,” and the independent schools will not say that they have an unfair advantage. Can you do an investigation that is rather more in depth than simply asking the people who administer the system?

It is really important that I repeat what I said at the start and in my letter to the committee. I said in that letter that I am

“happy to undertake some additional engagement with ADES, SCIS and centres”.

I did not say that I would be engaging just with ADES and SCIS. I have very frank discussions with ADES and SCIS about a range of matters, including budgetary matters. We will absolutely ensure that we discuss those issues with centres, as I said in my letter of 19 December and my opening statement.

By “centres”, do you mean the headteachers who make those decisions?

I mean the schools. That is not confined to the headteachers.

Can we have an assurance that the approach will be rather more towards the front line and will include those who have been through the system and perhaps some parents and pupils, as well?

I am open to the idea of that engagement, but I am also keen to ensure that we have an evidence-based approach to our work. However, I am happy to make the commitment to engage with centres.

That is good, because I think that the evidence is as I have described it.

It is not the SQA’s policy to provide access to exam scripts as part of the review process. Why is that the case? I certainly take the view—and I think that most people do—that private individuals have a right to see the data that public bodies and other organisations hold about them, and exam information and scripts are important bits of such information. Will you review that situation?

The SQA marks about a million exam scripts a year. Until relatively recently, that was based on one single paper copy; that remains the case for some subjects. To date, it has been very difficult to consider that as part of our service. Currently, exam scripts are exempt from being subject to access through data protection legislation.

We have done some work on that. A couple of years ago, one of our heads of service undertook some engagement activity with centres, stakeholders and others to gather views on returning candidate scripts. That engagement included speaking to young people and others. It is interesting that there is a variety of views on that. We are on a path to the digitisation of assessment, and I think that the direction of travel is to potentially return exam scripts to candidates over time. We need to consider that quite carefully in the context of our resourcing and our systems, but we should keep it live in our thinking, particularly as more assessment material is becoming available digitally, which makes that easier and potentially less costly. We need to think about that in the context of our costs and have a wider discussion about it. Doing it is not cost neutral, and we need to keep that live in our thinking. However, it is not possible to do it at the present time.

That the issue is under active consideration is welcome. Roughly what proportion of scripts are digitised and what proportion are still paper based?

The majority of our examinations are digitised, and the majority of the course work is still on paper. That is the split at the moment. We tend to use our e-marking system for sat examinations under controlled conditions. Course work, which is a bit more complex in that it involves appendices, different types of artefacts and so on, is not digitised to the same degree. However, we are looking at ways in which we can use digital evidence in course work, centres can send their evidence digitally, and we can review it digitally. It is a moving feast at the moment. Our initial target was to move as many of the examinations as possible into the e-marking process first and then to look at course work.

The key thing about e-marking is that, as well as bringing into the realms of possibility the things that we have been discussing, it allows big improvements in the quality assurance of marking. We can have on-going checking of marking in real time.

Good morning, panel. Between 2013-14 and 2017-18, there were 30 performance reports, of which more than half were for independent schools. Will you comment on that, please?

Performance reports have been part of the service that we offer for a number of years. They represent a very small part of what we do. We provide support to all centres through our understanding standards events, SQA co-ordinator events and materials that we produce for both teachers and learners in the wider system, and the performance reports have formed part of that broader service.

As you rightly say, there were only 30 performance reports over that four-year period. Performance reports have tended to be for particular components of particular qualifications, and they are requested by centres from our qualifications development directorate. They tend to be undertaken by senior examiners or principal assessors. There is a charge attached to the service, which covers, in effect, the cost of the report.

We are considering the support that we will provide to centres during 2020-21, and we will look at performance reports in that context. We will look at what part, if any, they play in the wider context of the services that we provide.

Given that they are under review, what is your view of the number of performance reports? Is 30 a low number?

In the context of the work that we do across the system each year, it is a very small number. The number of schools that have taken up the opportunity to commission a performance report is very small, and some of them have asked for further reports on other areas. They are a very small part of what we do, and it is right that we consider them in the context of all the work that we will do to support the system in the coming year.

That is good to hear. Do you have any concerns that more than half of the reports were done for independent schools?

As I said, only a small number of schools in total have used the service. It is legitimate for us to consider any supporting criteria that may be attached to a performance report and, given the small volume of performance reports, whether they should form part of the service in future. There are some issues that we will wish to consider in thinking about whether to continue with performance reports.

I have a question for Ms Robertson on multilevel teaching. Your predecessor, Dr Janet Brown, said in 2017:

“If significant numbers of schools are delivering multilevel teaching, we have to start looking at content. The challenge would be that we would change again after looking at the content: we would have to change the content of either national 4 or national 5 so that blended learning could take place.”—[Official Report, Education and Skills Committee, 13 September 2017; c 13.]

In your letter of 19 December 2019, you say:

“If such a review suggests that an adjustment of content is required to more smoothly align each level of a National Course and still enable appropriate national standards to be maintained, then this will be given consideration when developing revised arrangements.”

Where are you with that? What is your thinking on it? What are the merits—or not—of multilevel teaching? Why is it increasingly being used?

I outlined in my opening statement and in my letter, as you highlighted, some of the work that we are looking at for national 3 and national 4. Part of that will be around content. Crucially, part of my responsibilities is to ensure that appropriate national standards are being maintained. That is really important. It is key that the integrity of the qualifications is being maintained. As I set out in broad terms in my opening statement, the work is being scoped out at present, and it will be considered carefully, including in our various committees within our existing governance and in discussions with the wider system. I anticipate that we will complete that work by June 2021.

There has been a huge amount of change in qualifications over a number of years. There was a big set of reforms around the new national qualifications between 2013 and 2015 and then a further round of revisions that we are just concluding now with the advanced higher changes this year. Where possible, we are giving the system a commitment that, if we do development work on qualifications, we will complete it and then give the system a year to implement any changes. I will give a specific example. We are developing a new higher on applications of maths, and all the work around that will be completed by the end of the current academic year, with the new higher being introduced from 2021-22.

We hope that the work will be completed by June 2021, with any changes being implemented by the academic year 2022-23. That will give the system time to deal with any changes to content, changes to learning and teaching practice and any issues with curriculum design.

On multilevel teaching, as I explained to the committee in September 2019, there is a hierarchical structure to our qualifications. We would expect a young person to reach curriculum for excellence level 4, which is aligned with national 4, before they move on to national 5. As an awarding body, we do not collect information about multilevel teaching, which is not something that we look at explicitly.

We are, obviously, interested in the practical implications of the delivery of our qualifications in the classroom. Robert Quinn might want to elaborate on this, but it can be more challenging for some subjects than for others. I think that, when we gave evidence to the committee in September, Robert made clear the distinction between high-content courses and courses with a high skills element.

10:45  

That is probably particularly true of some science subjects, for which we have had some feedback to the effect that multilevel classes can be more challenging. However, I think that it would be wrong for me to generalise on that, because it does not come up very regularly or forcibly at the school visits that I do. We do get some centre feedback on those issues, and schools take curriculum decisions in line with issues that they wish to consider in relation to the cohort of young people and the wider school community that they have.

There is a combination of issues there that are probably beyond my responsibilities, and I know that the committee has discussed them with other witnesses. Robert Quinn might want to say a little more about—

Before Mr Quinn comes in to elaborate on that, will you say whose responsibility it is to monitor multilevel teaching? You said that it is not the SQA’s responsibility, but surely some education body must be considering it.

Curriculum design and delivery issues form part of the work on empowering schools. The Government’s policy position on curriculum decisions is that they are for schools to take. Schools are inspected as part of the inspection regime that is in place, which is carried out by our colleagues at Education Scotland. As they will have told the committee, if they have concerns about curriculum issues, they will consider them, and that will include issues to do with multilevel teaching.

Over a number of years, Education Scotland has provided messages to the system on curriculum issues, both in the context of work that was undertaken by the curriculum for excellence management board and other fora and, more recently, at the assessment and national qualifications group.

Before this meeting, I reminded myself of the statement that the then chief inspector provided back in 2016, which was specifically about progression from the broad general education to the senior phase. In recent years, there have been occasions on which messages have been provided to the system. That has been done in the early part of the curriculum for excellence management board’s development, when it looked at curriculum models, and more recently when, under the auspices and through the responsibilities of the chief inspector, Education Scotland has drawn on evidence from inspections and other existing fora in considering such matters.

We have probably discussed most of the issues. We have done quite a lot of high-level analysis of how qualifications are being delivered in centres. Except in relation to the science subjects, the issue of what we might call bi-level delivery has not been raised with us as having caused problems for qualification arrangements.

For most subjects, the broad content areas and the skills and knowledge areas have been aligned hierarchically. If circumstances permit, it is therefore possible to deliver those subjects in a bi-level way. In the sciences, though, there is still a debate to be had on the way in which the content at national 5 and the unit requirements at national 4 are specified. If the practicalities turn out to be such that we can align the content more closely to facilitate that, we will need to engage with that debate and consider such an approach.

Especially in the science areas, we are in the early stages of listening to feedback from teachers and schools and considering what would be appropriate. Most of the work that we are doing, especially on national 4, is about looking at the currency and the appropriateness of the key skills and ensuring that we have benchmarked the standards appropriately. In some areas, we had not looked at those issues for five years, so we really needed to consider the alignment and make some adjustments. However, on the specific question of content alignment, the sciences are the one area on which we still need to have a debate.

I understand that multilevel teaching has been around for a long time, but it appears that it is now used more frequently. Do you have a view on why that is the case?

I cannot answer that question easily. The curriculum models that are in place in schools are determined by a number of factors.

Okay. When do you intend to review the national 5, higher and advanced higher qualifications, if you are intending to do that?

As I have highlighted, we have just been through a significant change in qualifications, with the first round of changes to introduce new qualifications in 2013 to 2015 on a three-year rolling programme and, following the decision to remove units, a further three-year programme. There has been a huge amount of review and reform of qualifications over recent years.

We have significant delivery responsibilities each year, but we also have a responsibility to continue to develop and review. However, I am hearing from the system that there has been a lot of change and a period of stability would be helpful. We will continue to consider further development work in discussion with the wider system, as appropriate. We are keen to ensure that there is a rolling programme of development on that basis, and that is why I have highlighted the work that we are doing on national 3 and national 4, which has not been part of the work of the past three years.

It is important for me to highlight to the committee that we have finite resources and we have choices to make about the balance between the serious job of making sure that more than 130,000 candidates each year get the right results on the right day in August and the exams timetable runs smoothly, and our development work. We are cognisant of the fact that changes to our qualifications system impact on teachers and learners so we need to take such work forward carefully.

Okay. Thank you.

I would like clarification of what Fiona Robertson said about the timetables for the reviews. With the review that is happening for national 3 and national 4, will implementation not happen until 2022-23?

The review is being scoped out, but that is my expectation.

That is to give it time to bed in.

Yes.

How will the review go forward? You said that there are continuous reviews, but it was mentioned that national 3 and national 4 had not been looked at for five years. Is five years a maximum? Will the next five-year period start in 2023, when the changes are implemented, or will it start now, because you have started to look at the matter?

As a rule of thumb, when a qualification is implemented and teachers start to deliver it, we will leave it for three years unless there are any serious issues, corrections or changes in legislation. During that time, we will gather a lot of evidence and do a lot of work with teachers on understanding standards, using live examples and so on. Between three and five years after implementation, we will then do a review of each qualification and a decision will be taken on how much to change it. The review might discern that everything is working fine and that no change is required. There might be minor change, or it might be that more serious work requires to be done. Once that is put in the stocks, there will be a year of working with teachers, and then, once they start implementing the qualification, we will move to the next five-year cycle.

The idea is to move away from a big-bang approach. Changes are made subject by subject according to particular needs.

That is right. We have been through a period of significant review and change. We are now moving into, perhaps, a more stable process of on-going development and review, and the work on national 3 and national 4 is very much part of that.

As I highlighted, we are conscious of the need to ensure that changes are made, where appropriate, to respond to the needs of and feedback from the system, but also that a balance is struck between change and stability. As I said, I am hearing from the system that it needs stability, having gone through a period of significant change over a number of years.

If a completely new course and qualification is introduced to the curriculum, how long will you leave it before carrying out a review?

My understanding is that we would apply the three-year rule unless—I give this caveat—there was a concern or we felt that we needed to put something right. Minor changes can be made to qualifications on an on-going basis, particularly if we think that candidates might be disadvantaged for whatever reason if we did not make those changes. We should be open to making a change if it is needed. However, I have made it clear to the system that, if we make a change, we need to be clear about the reason and the timeframe, because I accept that changes to qualifications can have a significant impact on learning and teaching.

I come back to the points that were made earlier about multilevel teaching. My question is specifically about national 5 and higher courses being taught in the same class. I accept that this is anecdotal evidence but, some months back, we held an engagement event at which teachers talked about some of the pressures that that situation creates. They said that if, for example, a history teacher has a group of national 5 students and a group of higher students in the same classroom, there is the pressure, or the temptation, to teach both classes, let us say, world war two—one to national 5 standard and one to higher standard. The following year, the national 5 students who go on to higher hear all the same stuff about world war two all over again, but they answer the question in a different exam. I think that English teachers made the same point about “Dulce et Decorum Est”. Are those criticisms fair? Does the exam system create the temptation, or the pressure, to teach kids the same stuff all over again?

There is a distinction between different qualifications, and we produce materials to ensure that there is clarity on course content and the standards that need to be met. For some subjects, there is some flexibility in the learning that is undertaken. Robert Quinn might want to say a bit more about English in particular, because that is one of the subjects for which he is responsible. A big difference between national 5 and higher is the complexity of learning.

That is not my point.

Your point is about the themes that are followed.

Do kids have the right not to be taught the same stuff all over again?

That relates to the debate about balancing prescription against personalisation and choice. As you know, in English, there is a free choice in relation to texts—

I am sorry to interrupt, but, to be clear, teachers are making the point that there is not a free choice. They say that there is the pressure to do what I have just described, because they have two groups of kids in the same class and the exams allow them to do that. Is there that pressure?

Theoretically, the examinations allow teachers to use the same text for national 5 and higher, but we would never recommend doing that. Separate texts should be used so that that scenario is avoided. Theoretically, it is possible to use the same text, but, in our arrangements and support notes as well as in our work with teachers on understanding the standards and in what we exemplify, we would take the opposite view.

11:00  

Thank you. I thank Dr Allan as well, because I was also going to raise that point. The committee’s concern about the evidence that we heard from teachers was that they were saying that that did not happen under the previous system, when pupils had a breadth of different experiences, topics and texts as they went from standard grade to higher. Our concern was that pupils are getting the teaching for their qualification but not the opportunity of a depth of coverage of their chosen subject.

Thank you, Ms Robertson, for clarifying some of the answers to my questions at our previous committee session with the SQA and for your two subsequent letters. As I understand it from the letters, there were 14,153 marking appointments in 2017—you were careful to distinguish those from markers—12,450 in 2018 and 11,775 in 2019. Can you clarify why the numbers are declining, when the number of qualifications that SQA is responsible for has increased?

Marker requirements change year on year.

Is that the same as appointments? I am confused—

Yes, marker appointments.

Is that the same thing as requirements?

I will try to not get myself in knots on this. Marker requirements can change over the period—that is informed by the entries that we get through and their composition—and marking requirements will be different for each subject because the assessment approach will be different. I am trying to be as helpful as possible. The marking requirements change moderately from year to year. They also change because some markers will do more for us and some will do less. We need to consider a whole range of issues.

The marker requirements for this year—the live data, as of 15 January—are 12,702; 9,236 markers have been contracted, with in excess of 2,000 at invited status. We are also processing a further 1,000 applications. The full list of appointee roles runs from principal assessors, all the way through to deputy principal assessors, senior team leaders and so on. A hierarchy exists around our appointments process, but the marker requirements are as I have outlined them.

In our previous committee session, you were particularly interested in the number of withdrawals—

Can I come to that in a minute?

Okay.

Was my response helpful in trying to explain where we are?

Yes. I am trying to get my head around the substantial reduction in the number of marking appointments over the period of the new qualifications, despite the fact that the number of different qualifications has risen.

I understand the point.

I am anxious that we get some clarity on that; your previous answer was helpful, in that—

There is complexity underneath all of that, which is what I am trying to describe—

I get that.

—subject by subject, qualification by qualification.

Can you assure us that every single one of the markers whom you take on—whether they are a principal assessor or a marker—is qualified in the subjects that they—

I think that I can absolutely give you that assurance.

Can you give me that assurance?

Yes, I can.

Good.

We are very clear about our expectations of markers. In addition, a lot of training and support is provided to them before they go live—before they go through the live diet.

It is just that our briefing paper tells us that the criteria include the need to be

“currently ... delivering the Course (or a closely-related Course or work closely with current deliverers)”.

What does “a closely-related Course” mean?

The person might be delivering another level—they might previously have delivered the same level but are not currently delivering it in that year.

But within the same subject.

Yes—within the same subject. I do not mean in other subjects.

So there is nobody marking exams who is not in—

Who is not qualified in that subject.

No, there is not.

Good.

I turn to the question of the actual number of markers. You have given us the up-to-date figure of 794 withdrawals. That is accurate: you have said that in two letters now.

That was in 2019.

The table that you gave us says that 130 markers were “Withdrawn by SQA” and that another 102

“Did not attend markers meeting”.

In both cases, there are concerns about those markers not being up to scratch.

If someone does not attend a markers meeting, they cannot be a marker. My point is that there is an expectation that markers go through training and development, as you would expect, in order to do marking for us. There will be some withdrawals for absence.

Did you take on those 102 as markers?

Yes—absolutely: they were markers, and then there was a withdrawal—

Why do you think that they have not attended meetings?

There will be a variety of reasons why they have not attended meetings. I do not have information on that, but I guess that, in general, if someone is not meeting marker requirements—for which there may be a range of reasons—they will not proceed to mark.

Presumably, those 102 markers would be responsible for marking at several levels.

They may be.

Do you have statistics? Our concern—and this is why we asked for that table—is to get to the bottom of the reasons why people withdraw, and whether that is for good, personal reasons, such as illness or whatever, or for other reasons. I am slightly concerned by the table, in that quite a lot of the reasons that are mentioned in it suggest that the markers are not content with the SQA process.

I do not think that that is a fair description of what the table says. I have spoken to the head of appointee management about withdrawals, and the majority of markers withdraw for personal reasons—family commitments or health of self or family. Where individuals withdraw for personal reasons, we will keep in contact with them, as they are often happy to return in a future year.

I have had the pleasure of visiting a number of schools over the past few months in my new role, and the markers I have met have been pretty positive about their experience of marking for us because of the professional development opportunity that it brings, some of which can be brought into their learning and teaching practice.

In the context of more than 12,000 appointments, marker withdrawals is something that we manage year on year. The committee is quite right to seek assurance from me and the SQA that we are in a good position when it comes to markers and that we are confident that we can meet the requirements of the diet in the coming year. I think that I have said that—I think that I can be confident about it.

We will continue to monitor marker withdrawals on an on-going basis. If we can provide more granular information on the reasons, we will do that, but the majority of markers withdraw for personal reasons.

If we had a similar table for 2018, 2017 or 2016, would the percentages for the different reasons be broadly the same?

Yes—broadly. I looked at the figures for 2018, as I anticipated that you might—not unreasonably—ask that question. Marker withdrawals were at a similar level. In fact, they were very slightly higher in 2018 than in 2019 as a percentage of the total number of markers.

Given the constraints that you have intimated relating to the pressures on your resources, if people have not turned up to meetings—or, in some cases, have “failed”, as the table says—would you consider putting them back on the marking register in another year?

As I have highlighted, markers withdraw for a variety of reasons. They might do so because, for whatever reason, we are not satisfied that they are able to mark for us at the time, but there might be a point in the future when they could do so.

If they withdrew because of issues with the quality of their marking, we would not use them again immediately—they would not usually be eligible for the following year’s diet.

On the resource constraints, I add that we are looking at marker training and at taking advantage of the electronic tools that we have in relation to scripts and commentaries to consider how we might give people who have previously not marked to standard the opportunity to get back into the system.

Of those who marked in 2019, how many were not up to scratch in one way or another, so you will not be using them in 2020?

It was a very small number.

One hundred and thirty were withdrawn by us—

That is quite a lot of people—

—and 29 failed at performance and quality marking. That is a small number in the context of our overall appointee requirement.

It is important to say that, although the focus has been on those who might not have met the standard, many thousands of teachers do. I should put it on the record that we are very grateful for that wider contribution from the system, because it is critical to the delivery of qualifications each year.

I have two very quick points on that theme. Do you detect that there are specific pressures? Are there subject areas for which it is harder to get good-quality markers?

There can be pressures in some subject areas.

Could you tell us what they are?

There is also a volume issue. For example, we require many hundreds of markers for English, which is one of our biggest subjects, and we require fewer for subjects with lower uptake, such as Gaelic. Sometimes, the volume issue can be more difficult.

Which subjects have the pressure points?

The subjects where that tends to be relatively more difficult year on year are the sciences, maths and psychology—there is on-going pressure. Sometimes, it can be more difficult at a particular level—for example, at advanced higher.

As I said, I accept the committee’s interest in such issues and that it will want to seek assurance on them. We have very well-established processes and procedures to ensure that we have sufficient markers year on year. Many markers who have worked with us for a long time go above and beyond to deliver for Scotland’s learners.

I have a final question. Have any of your higher markers given you feedback on what they feel might be the reason for the four-year consistent decline in higher standards?

Our annual work on how exams perform is contained in course reports, which flow from feedback that we receive through the marking process, including through the principal assessor. Subject by subject and qualification by qualification, course reports provide feedback to the wider system. That is how we receive feedback on how learners have performed in each of our qualifications. We publish the course reports and try to promote their use, because we think that they are helpful to the system in understanding issues that arise through the marking process. Therefore, we get feedback through the marking process; it forms part of a published course report on each subject.

I spoke about that bit when I appeared before the committee in September. It is an important part of what we do in each subject, in that it provides granular evidence of what we see through the marking process each year.

I just want to check whether there is a set number of years for which teachers must have taught a subject in order for them to become SQA markers.

I think that it is two years.

11:15  

I will come to the drop in the numbers attending the markers’ meetings, which is hugely important, because that is where you set the national standard. Please correct me if I am wrong, but I think that, in the past, the SQA used to cover costs to allow teachers to be released from schools, which was a bit of a carrot and stick for headteachers, who would take that funding and allow their staff to be released. Have you considered whether headteachers are not allowing their staff to take part in that valuable continuous professional development because that funding is no longer available?

I acknowledge that we absolutely rely on the wider system in schools being a critical part of the work that we do. We have also been exploring other means of delivering marking, and we are doing much more out of school hours, as well as offering webinars among other things, so that there is more flexibility in the approach that we are taking with our wider programme, both in terms of markers and our understanding standards events, for example. We are looking at a much more mixed model around how we deliver and consider our work, taking into account the impact on teaching time, and we have been doing quite a lot in that area. This year, there have been quite a lot of webinars in the group of subjects for which Robert Quinn is responsible.

Having said all that, I am conscious that teachers like to get together in person for networking and development opportunities, as you have highlighted. A group of subject teachers getting together to discuss those issues is also important, so there is a balance to be struck there.

On a number of occasions, you have talked about markers and their status as appointees. In my 15 years of professional experience as an employer and a line manager, I had never come across that term prior to quizzing the SQA. Indeed, I have your terms and conditions of appointment with me, which states:

“This document, together with the letter of appointment, sets out the terms on which you will provide services to the SQA as an appointee.”

The second paragraph starts

“As an appointee, you are not an employee, director, or officer of the SQA, and nothing in these terms and conditions is intended to create any such relationship.”

Could you just outline the basis in law of the status of an appointee?

You have done a little of that in your introduction to the terms of the appointment. I am not a legal expert, so I cannot comment on that in any detail, but we are not employing appointees. Appointees tend to be employed by either a local authority or a school, and we are using and paying for their services for marking on an annual basis. If you want me to come back to you in more detail about some of those issues, I am happy to do that, but I am not really able to go into detail on the issue here.

A number of teachers will have jobs outside their work as teachers, and all credit to them, as it takes a lot of energy to do that. I do not think that we would be claiming that, because they are employed as teachers, their other part-time jobs are not employment.

Indeed, if we look at recent court judgments, specifically the Pimlico Plumbers case and the controversies around the gig economy, what employment is is a live issue. It is not up to the employer to define that; it is a matter of law. Have those court cases and controversies caused the SQA to review its terminology and the status of the markers? The term “appointee” is in your terms and conditions, but I have not heard of it having any grounding in statute.

The appointee structure and our relationship with teachers through our marking process has been established for a number of years. As I say, the process has worked for us and it also works for teachers. The feedback that we get is positive. I cannot say much more—maybe I need to rely on my newness here, but that is not something that I have looked at in any detail in the short time that I have been at the SQA.

I know that the committee has previously been interested in issues related to the payment of invigilators and others, but that has not crossed my desk to date. I would be happy to look at it.

We seek legal advice on a range of issues, including any contracts that we issue in relation to appointees or any other matter relating to the employment status of people who work with us.

We have previously had exchanges about the SQA’s international business area. I should state for the record that I sent a couple of questions to the panel members in advance of this meeting, on a number of factual points. I am happy to provide them if anyone is interested.

The contract that I have focused on in the past is one in Saudi Arabia. I would like to get clarification on a couple of the questions that I submitted yesterday. The Saudi Arabian Government has a project called Yesser as part of its modernisation programme. It is about equipping Government officials and civil servants with greater information technology and digital skills. Will you clarify whether the international certificate in IT skills that is being delivered under the Yesser project is the SQA IT certificate?

Mr Greer’s researcher provided some questions to us over the past couple of days on those issues. We went back to Mr Greer’s researcher, and John McMorris will be happy to answer those questions. We can be quite definitive about the position.

The certificate that Mr Greer mentions is not an SQA certificate. The SQA is not involved in the Yesser project through that contract or any other contract.

In that case, is the SQA certificate being delivered to Saudi Government staff or civil servants in another way, or is the project being offered through a private college to private individuals in a way that is entirely separate from the Saudi Government?

Yes, it is being delivered to private individuals on the basis of individual need and requests. We do not have any contractual arrangements with any Government department.

Under the heading “Our customers”, the SQA website lists:

“Government agencies in ... Saudi Arabia”.

Can you clarify who that is and what it is for?

Do you have the name of the page?

The framework development page of your website has:

“Our customers

Government agencies in Bahrain, Oman and Saudi Arabia”.

Those are historical contracts that we delivered. We always publish our track record on consultancy projects that we have undertaken. We helped to build the national qualification framework for one of the Saudi Government agencies—I cannot remember the agency’s name. The publication for that is on the website. We have also had contracts for Saudi skills standards in the past, through which we helped them to establish some national occupational standards. We had two large contracts about three or four years ago.

You just clarified that the current contract with TeTec is for private individuals. According to press releases by TeTec, it concluded negotiations with you in June 2017. TeTec announced that your certification was being offered from June 2017; in fact, it announced that trainees who were already enrolled on another IT course that was provided by another British exam agency would be able to transfer over to the SQA course. Yet in September 2019, when you were last before the committee, you let me know that no one had yet been certified and that the SQA were “still reviewing” assessment criteria.

Could you clarify what the situation is? I am confused about how, more than two years after the provider announced that the contract had been concluded and the course was available, no one has yet been certified in it and the SQA is still reviewing the criteria.

TeTec has enrolled 1,378 students on the course, and those students are working their way through it. To date we have not certificated anyone. The quality assurance arrangements are such that we do not have the evidence to be able to certificate, so it is in a quality assurance process.

How long does the course take for a student who is enrolled in it?

It is not a long course. As I mentioned before, it is fairly entry level, but we cannot certificate unless we have suitable evidence from the provider. The provider has not given us that, so certification is on hold until we see the appropriate evidence.

Do you have any indication of when that will be resolved? It seems very odd. As you previously explained, the short courses are about basic skills such as how to use a keyboard. They were announced as starting in the summer of 2017 but, more than two years later, no one has been certified and information has not been provided to you so that you can go through the certification process. What is causing that problem?

As I say, our operations colleagues have been working with the centre to get the suitable evidence to meet the outcomes for the course. As yet, we have not been furnished with that, so we are not in a position to proceed. There have been conversations with the centre to follow up on getting that evidence. As yet, however, we cannot progress.

As for how a student would enrol on the course—you have confirmed that it would be private individuals—is it a case of students going through a Saudi high school and enrolling in further education, is it folk coming in off the street, or is the course being offered to businesses that want to raise the skills of their staff?

I do not have the detail, as this is not something that we track, but the head of centre has confirmed in conversations that it could involve a whole range of current employees or people trying to enter the workforce. As you say, it could be high school students. The students can come from a range of sources. They could be unemployed people.

The reason I ask is that Saudi Arabia is one of those countries where the separation of private and public, and the separation of government from private businesses, is not absolute. Saudi Arabia is an absolute monarchy, but its royal family runs to well over 7,000 people, many of whom have substantial private business interests that are essentially extensions of the Government. That is why I am asking about who the businesses that have enrolled their staff in the programme may or may not be.

Moving on from the specifics of that contract—we can follow up on some of those specifics in writing, but what you have said has been very useful—in your letter of 19 December 2019, Fiona Robertson made some welcome announcements about further work that you are doing to review your processes for international contract work. It would be useful to have a little bit more detail on that. Am I correct in understanding that that is about reviewing existing due diligence processes and strengthening them as required, or have you have identified a need to create a new process? Is there a need for a new human rights due diligence process, or is it a matter of adapting existing processes?

We are currently finalising a new policy for human rights and a framework to be put in place. You are absolutely right: based on the guiding principles and the framework, which are aligned to both Scottish Government and United Nations guidance, our process is not a stand-alone process, and we want to embed it into our key decision making. We have a country appraisal process, and we are embedding human rights into that process according to the principal framework, so that every country that we operate in will be assessed for the human rights impact. As part of our due diligence, we are also doing that for centres, potential centres and partners. Human rights impact assessments will form part of due diligence for the centre and for key individuals in the centre. We are baking the human rights impact assessment and risk mitigation into a variety of processes that the SQA has established. We will also build that into the actual approval and the verification process so that we can keep a constant review of all the international activity.

Excellent. Are you consulting human rights organisations to ensure that the processes are robust? For example, Amnesty International has been consulted by Government agencies in the past.

We did a benchmarking exercise with the Scottish Government and some agencies, and we have examined the United Nations guidance. We will consult a range of sources on a regular basis to keep an alert system in place. Those include various non-governmental organisations such as Human Rights Watch; we will consider the FCO human rights and democracy reports and reports by Freedom House, the Business & Human Rights Resource Centre and so on. Those are common sources of good intelligence for what is going on in various countries, and they tend to be the sources that are primarily used by the Foreign and Commonwealth Office, the Scottish Government, Scottish Development International and other bodies.

Your intention to have that policy and framework in place for the next financial year is really welcome. That speed indicates a sense of urgency that is often lacking among public bodies.

Will you be publishing the new and updated processes ahead of time?

Yes. Once there is internal approval next month, we will publish the principles, the guidance and the framework. Then, all the current activity that we do abroad will go through that process by April.

We would be happy to send that to the committee proactively.

That would be very useful, thank you.

11:30  

As Liz Smith alluded, there is a lot of discussion about attainment rates in Scottish education. In your opening remarks, you referred to the fact that there has been a decline at SCQF level 6, and for four of the past five years the long-term trend—although there has been a small rise—has been downward. How has the SQA reflected on those downward trends? Is work on-going to look at the underlying reasons for that?

Every year, the chief examining officer provides a commentary on exam results for that year. I included my commentary on the most recent year as an annex to the paper that we submitted before our committee appearance in September 2019. The chief examining officer’s report is a commentary on how the assessments have performed and the overall results. It is provided proactively and published on results day, alongside a pretty straight statistical report on SQA results, which contains information on the number of entries, attainment by grade, A-to-C attainment and so on; it is a full summary.

As I highlighted earlier, we also provide course reports by subject and qualification, which give more detail on how candidates have performed and any feedback that might be helpful to the system. We promote those reports as a helpful resource for the system. I would hope that the course report for higher English, for example, would be helpful in providing guidance to someone who teaches that subject. My job is to ensure that the course content and standards are clear; that those standards are maintained; and that there is an understanding of standards across the system.

We also discuss results with the Scottish Government and colleagues in Education Scotland; that is a common feature of the work in which the SQA takes part. I see our job as seeking to explain—in a fairly granular fashion, by subject and qualification—our feedback to the system.

I hope that that provides an explanation of how we see our role. As I highlighted, we would expect to see some variability in A-to-C attainment rates over time. That is a standard feature of the system: when we look back across a range of qualifications over time, we will see some variability. I have a table in front of me that shows A-to-C attainment rates from 2003 to 2019. Of course, there have been changes to qualifications during that time, but there has been some variability.

You have explained your role, but you have not explained any of the insight. At the very least, we have to start at the point at which the new qualifications were introduced—we cannot compare the data prior to the introduction of national 4 and 5 and the substantial reduction in higher assessments. Year on year, we are seeing a significant—perhaps not dramatic, but significant—decline in attainment. As you said, you provide reports and insight by subject. It would be interesting to hear a summation of trends that exist—if they exist—across those subject reports.

We are looking at four years of data. If we see the same trend continuing for another year or two, would that not be significant cause for concern?

We can look at trends over a period of time or at A-to-C pass rates specifically—we can cut the data in a number of ways. The Scottish Parliament information centre briefing for the meeting looks at issues around both the volume of attainment and attainment rates. There are a number of ways in which we can look at data on attainment.

On results day, we publish a straight report of all the SQA statistics, including entries, A-to-C pass rates and all the grades across a wide range of qualifications. In addition, the chief examining officer’s report provides a broad commentary. As part of that commentary, I have a responsibility to talk about how the assessments have performed and demonstrate that standards have been maintained. It is important that we provide that level of detail at subject level, because different issues are likely to play out in different subject areas. It is important, therefore, that that detailed information is published and provided to the system—that is our role.

You highlighted the focus on the A-to-C pass rate at higher level. I acknowledge that, between 2018 and 2019, there was a fall in attainment, for which I sought to provide an explanation by subject, alongside course reports that provide supporting information. Our job is to explain all that.

Indeed—that is what I am asking you to do. You say that you have presented the information elsewhere. For the benefit of the committee, could you provide it now? Although I recognise that there may well be different explanations for different subjects, it is clear that there is an overall trend. I request that you rehearse that for the benefit of the committee and for the record, because we are discussing an issue of national importance.

I highlight the information that is contained in my chief examining officer’s report, which I provided to the committee in September. It highlights my observations and—given that I am fairly new to my role—draws on the observations of my predecessor, who oversaw the conclusion of the diet in 2019 and provided an overall commentary on the diet for the year, including the variation in attainment over time. That report is my statement on issues relating to attainment.

But you are not prepared to summarise it for the committee this morning.

I am conscious of the time, Mr Johnson. If the information has been provided—

It has been provided.

We can take Ms Robertson’s comments to the effect that it is in the public domain.

It is difficult to discuss a topic if people are not prepared to—

With respect, I provided the material in advance of the September meeting.

Okay—fine. I will ask one last question. On top of the data on attainment, we can see that since 2013 there has been a decline in the total number of entries per senior phase pupil, taking in a broad range from S4 to S6. We can understand that an initial drop would have resulted from the trend towards six subjects in S4, but there has been a relatively substantial drop since 2017. That takes us to a 12 per cent drop in the total number of entries over that period, with the final 5 per cent drop occurring in the past two years.

Has any work been done on why, in the past two years, there has been a further drop in the number of entries by senior phase pupils? If the breadth of qualifications is important, that is a concern.

Entries can drop for a number of reasons. We receive entry information from centres that represents the culmination of the decisions that take place in schools. It also reflects the curriculum model that exists in a school; any differences in school rolls over that period; and the times at which young people take their qualifications. For example, if a school offers a two-year higher course that bypasses national 5, there will be no national 5 entries. Entry data will be a culmination of factors.

Schools’ presentation decisions will also be relevant. In that context, the denominator matters. For any qualification, A-to-C pass rates will be directly impacted by the number of young people who are entered for it. I cannot comment on those who are not entered for a qualification, because such information does not form part of the statistics that we have. We report on entry data and the attainment data that follows from that.

I understand that you want a clear, direct answer to your question. I am trying to say that a number of factors can influence entry data, such as school roll, curriculum design, and entry and presentation patterns—there can be a whole range of issues.

That is a fair insight, which it is useful to hear.

I have a final question. If we consider those two trends together, they suggest that pupils in the senior phase are both taking fewer exams and not doing as well as they did four years ago. Is that a correct summary?

For the reasons that I have highlighted—all else being equal—I would not say that that is the case. You are looking at different cohorts of young people, and things do not remain static from one year to another. I absolutely acknowledge that there was a drop in A-to-C attainment at higher level in 2019—I am not suggesting otherwise—but a range of factors will have contributed to that fall.

Within any volume of entries, there will be distinguishable differences in entry and attainment data for lower and higher SCQF level qualifications. In reaching any conclusions about the broader performance of the system, therefore, we must look across the whole suite of qualifications that we and other providers offer.

There has been a growth in interest in wider qualifications in the SCQF, such as foundation apprenticeships and national progression awards. Schools’ interest in those is increasing all the time, and our qualification managers are fielding inquiries on such provision. People now have a bit more headspace, so they are looking at more creative ways to deliver the curriculum and at the range of provision that is offered.

From an SCQF perspective, rather than looking simply at examinations, we can see positive trends. In particular, there is the growth in what we might call the mixed economy of qualifications, which will be critical to ensuring that people are well prepared for the transitions that they make.

I think that we have exhausted our questions, after what has been a very long session. I thank Ms Robertson and her officials for attending.

11:43 Meeting suspended.  

11:49 On resuming—