Education, Children and Young People Committee

Meeting date: Wednesday, November 10, 2021

Official Report 550KB pdf

---

Upper Secondary Education and Student Assessment

Agenda item 2 is evidence from Professor Gordon Stobart, emeritus professor of education at University College London, on his report providing a comparative perspective of upper secondary education and student assessment in Scotland. Professor Stobart has worked as a secondary school teacher, as an educational psychologist, as a senior researcher in policy-related environments and as an academic. His expertise is in assessment, with much of his recent work focusing on assessment for learning.

Professor Stobart’s report was published as part of the Organisation for Economic Co-operation and Development’s series of working papers, in which the OECD publishes papers that describe the preliminary results of work in progress in order to stimulate discussion of issues on which the OECD works. I stress that the report represents the views of Professor Stobart rather than those of the OECD.

We are delighted to have you with us, Professor Stobart. You are very welcome. Before we open up our session to questions, I invite you to make an opening statement.

Thank you for the invitation. The paper resulted from concern about the alignment between curriculum for excellence and the senior secondary school assessment system, which was something that came through as a theme when the OECD was researching curriculum for excellence. The OECD said that curriculum for excellence “loses power” when pupils get to senior secondary school and exams begin. My report complements the OECD’s curriculum for excellence report.

We chose a comparative approach as a reminder that there is more than one way to assess senior secondary school pupils. Assessment systems reflect the culture in which they develop. Often, as is the case in Scotland, those developments go back to the 19th century, so there is a long tradition. When we are embedded in a tradition, we often cannot believe that there are other ways to do something—we have always done it this way. The value of a comparative approach is in seeing that others do things differently. We can look more broadly at the Scottish system by seeing it relative to others.

The Scottish system is the anchor point of the report. I locate it within the British legacy system, which has its own distinctive features, particularly the scale of national exams between the ages of 16 and 18. That is fairly distinctive; many other systems do not operate in that way.

I should say that I walked a bit of a tightrope in preparing the report. It is a publication for an international audience but primarily for a Scottish one. I was trying to give an accessible and simplified summary for international readers, whereas Scottish readers know how complex and nuanced their system and its delivery are—they know about multilevel teaching and the like. Those of you who have come through the system and are embedded in it will be saying at times, “That’s a bit simplistic,” but that was my juggling act.

The final point, which you covered in your introduction, convener, is that I was asked to do this partly because of my own experience. I am not a lifelong academic, looking down from on high, but a late developer. First, I taught and then I was head of research at an awarding body—it is now Edexcel. I developed the general national vocational qualification—the GNVQ—in England, at the National Council for Vocational Qualifications, and I then moved to the Qualifications and Curriculum Authority, which covered exams and vocational qualifications. It was only then that I moved across to the institute of education at UCL, where I did more work on formative assessment, including assessment is for learning in Scotland.

I therefore do not come at this as a purist academic; I have been on the inside and I know some of the compromises that have to be made when doing any curriculum reform. I hope that that sets the scene for the discussion.

That was an excellent introduction, and it was good to get clarity on your wide-ranging experience. Can you talk about your experience of Scottish education and the Scottish educational landscape?

Coming from England, I have been an enthusiastic supporter of the light-touch approach in Scotland. England is on the heavy accountability, heavy testing end of things, so I have always looked to Scotland without fully understanding what goes on in Scotland. I have always thought that you just do it better in Scotland, without necessarily knowing what it is that you do.

It has come as a bit of a surprise, in trying to get inside the system, to find out just how complex it is, with the Scottish diet of examinations having three parts to it, some delivered through multilevel teaching, and a range of vocational qualifications. It has taken me some time to get my head round the detail of that side of things.

There is also a kind of openness in relation to education and there is the importance that is given to education in Scotland. I think that that is part of your tradition: you feel that education is central to Scottish identity, and that has come over powerfully as well.

It is good to hear that, reputationally, you saw our education system as being better than the system somewhere else, simply because it was Scottish. We can all sign up to that, on the basis of our national pride.

Were you surprised by what you discovered about the complexity and the nuances that you have just described?

Yes. Probably the biggest surprise was the sheer volume of examinations that secondary school students go through and the complexity of that, with the three levels—national 5, highers and advanced highers. There can also be multilevel teaching in small schools, with students going for different qualifications, with slightly different curricula. It is a complicated system to steer through.

Yes—and quite difficult to capture in a single report such as this one, as you said.

Every time I thought I understood it, somebody would come along and say, “You know that we do this, that or the other,” and I had to say, “No,” so I have learned a lot.

Thank you, I am enjoying your transparent honesty about what you have discovered. Kaukab Stewart, the deputy convener, will ask the next questions.

Thank you very much for your introduction. As a teacher of 30 years, previously, I read the report with great interest. It was helpful to have the international and United Kingdom nations comparators as well as the historical context. I note that other countries have reformed their assessment systems and practices. What can Scotland learn from their journeys? You mentioned a number of countries, and you may wish to highlight one or two.

09:45  

I suspect that most countries—perhaps all of them—are going through what Scotland is going through. Students are changing and becoming more diverse. They are staying on in education for longer but they have a varied range of needs. Every country that I have looked at has been trying to reshape its curriculum to fit modern times and then trying to align its assessment system with the curriculum. We have learned that that is a difficult process.

Some other traditions find it easier. I am thinking of Norway, where teacher assessment has always been central to the system—they trust their teachers. There are national exams, but they play a relatively small role. Pupils take national exams in only two or three subjects and teachers do the other assessment. There are other systems. The French are trying to widen the range of the baccalaureate so that a greater number of students, who are not necessarily academic, can come through a professional baccalaureate.

The powerful idea is to look at the role of the teacher. Also, what is the role of the student? How much do we take account of student perceptions? As a former teacher, you will know that students have views on that. They are the ones who are most affected by the system. How do we take account of that?

Some of my colleagues might drill down into the issue of student engagement later.

There was a renewed respect for teachers during Covid. You referred in your report to “high trust” in teacher-led assessment. What factors can enable such trust to develop? How might we move forward? There can be resistance to change.

We cannot announce that people should trust teachers. It is a gradual process that will require training and professional development.

We trust teachers to carry out assessments right through the broad general education. We also trust teachers in further education and in universities to make assessments without much supervision. Suddenly, in senior secondary school, a great deal of pressure is put on the system. I know that that is because we are selecting students for university and so need fair and comparable assessments, but other places such as Canada, New Zealand and Queensland in Australia see their teachers as being capable of doing that. That is how their systems operate.

At the other end of the scale, in Ireland, teachers do not want to assess their students and the country has stayed with exams. When reforms were introduced to try to get teachers to do some assessment, there was protest and non-co-operation. There is a delicate balancing act. As I see it, it is to do with moving towards giving teachers more responsibility.

Thank you for your refreshingly frank introduction, professor. My questions cover three areas: the staging posts of qualifications; the effect of a leaving qualification on the leaving age, whether from school or education; and motivation.

On staging posts, some pupils like to get qualifications in the bag as they go rather than leaving it all to the very end—the last year—and putting all the stakes on one option. We currently have three years of nat 5s, highers and advanced highers. I know that you are not making a specific recommendation, but the hint is that you want to strip things down. How far would you go? How far would it be reasonable to go? What are the trade-offs around that?

My personal view is that we do not necessarily need a national examination to record progress—there may be other ways of doing that. I talk about a “mixed economy” in my report. We need examinations—I have not attempted to say that we do not need highers and advanced highers for university entrance. My point is more about whether, at the national 5 level, there are other ways of recording progress and what students have done with their school experience; that is where the notion of a mixed economy comes in.

In addition, although I could not substantiate this particularly, I found that, even at the beginning of the revised nationals and so on, there were still discussions about whether it should be a step-by-step approach—a ladder of qualifications—or whether a student should select the level at which they would leave or continue in education. This is anecdotal, but it seems that many schools go for the step-by-step approach on the grounds that they need to check on or assess students regularly as they progress through the system. However, Mark Priestley and others have commented on the issue of the two-term dash. When students move straight from one diet of exams to the next, it affects the quality of teaching and learning.

Currently, unlike some other countries, we allow pupils to leave at 16. I think that, in the past, some of my family left school at 14. The leaving age is now 16; other countries have gone for a leaving age of 18.

If we had a leaving qualification—not necessarily from school, but from education—would we need to raise the leaving age? Would we need further reform around the leaving age?

I think that it is being done almost naturally, as 88 per cent of students in Scotland continue in education after the age of 16. That is up dramatically from the situation 20 years ago. At that point, most students left at 16, which was the justification for having examinations at 16, and the situation was the same in England.

In England, the leaving age is now 18—you have to stay in education or in education and training or in part-time work and education. In France, the United States and Canada, the assumption is that students will stay on until the age of 18. I am not sure whether it would need to be made a legal requirement; I think that the encouragement is there in the system for students to stay anyway. We probably need to concentrate on making the programme attractive for students who, rather than following a narrow academic route, might want to go in other directions. If students stay on, how do we make it a rich and useful experience for them?

That is an interesting point. The vocational offer in schools is often criticised as not being sufficient. The academic route continues to be the predominant priority. Do you think that the vocational offer in schools is good enough, in the event that we went for a leaving qualification and raised the age for leaving education or training?

Looking from the outside, without understanding the actual programmes and everything else, Scotland seems to be in good position because the vocational and the academic aspects are dealt with in the same agency, although I know that that is changing. The two aspects are linked, so they can be grouped together.

There is a problem with the status of vocational education here, as there is more widely in the British tradition. However, with the foundation apprenticeships and other things, Scotland has a real opportunity to introduce high-status vocational qualifications. As part of the British tradition, it suffers from the fact that vocational exams are seen as being not as good as academic qualifications. The academic route is the easier one in terms of smooth progress.

As I said, I helped to develop a school-based vocational qualification that worked reasonably well. My personal take is that it worked so well that it got absorbed into the academic qualifications, which I would not encourage. It disappeared and became an applied A level, which did not really take off. If we have a diverse student population coming through, many of whom stay in education, we should offer them a diverse set of routes. The issue is the status of the routes.

My final question is about motivation. Some teachers and pupils tell us that having the focus of an exam gets them out of a rut and makes them work—it motivates them to achieve. However, I know that every pupil learns in a different way. Will you talk about what we might lose if we were to move towards having more assessments and away from exams at the end of the academic year? What are the issues?

I understand that kind of motivation, whereby we say to pupils, “You’ve got to do this in order to pass your exams.” However, there are cultures where there is less emphasis on that. We might say that it is an expectation that pupils will do something, but that puts the onus on us to make the programme interesting in order to motivate the teaching and learning and what is being done in the classroom. It is a bit of a carrot-and-stick approach. We can use the stick of saying that pupils have exams coming up and have got to work, but perhaps we should think more of carrots in terms of the usefulness and breadth of the programme.

I use the international baccalaureate quite a lot, which involves a really diverse set of expectations. It could be called an assessment, and it includes exams, but it also has other aspects. The essay, the theory of knowledge, personal projects and those kinds of things can be built into it, so it does not need to be a narrow preparation for exams. I suspect that exam preparation can become quite didactic at times—“You’re here to learn this, this and this”—rather than teachers engaging with students to develop ideas and their profile. Doing community service is part of the system in Canada and is part of the international baccalaureate. Other things can be introduced as part of the assessment.

Thank you, Professor Stobart. That has been very helpful.

10:00  

Quite a few colleagues want to come in on the back of Willie Rennie’s very interesting line of questioning. I want to wind back a bit to the issue of the school leaving age, and I call Oliver Mundell for a couple of quick-fire questions.

It was interesting to hear you dismiss the percentage of pupils who leave school early. Given that they are often the individuals who have been most let down by the school system, do you not think that they have the right to sit for an externally assessed qualification before the education system gives up on them?

I would turn your question around and ask what qualifications the 12 per cent or so of pupils who leave that way get through the examinations system. Early leavers often have very little to show for it with regard to exam results and the like—at least, that would be my experience and reading of the Scottish statistics. If, as I have suggested, there were some kind of portfolio or graduation thing, it could take far more into account and give a richer picture of what people, even those who leave early, did in school, what their strengths were and so on. I do not see having leaving exams that students are often unsuccessful in as being a good way of ending a person’s formal education.

If I understood you correctly, Professor Stobart, you seemed to indicate that, with regard to educational opportunities, there was a cultural bias against that particular pathway. I think that Michael Marra has some questions on cultural connotations.

The conversation has been really interesting so far, and I want to explore a couple of issues on the theme of culture. One of the great successes of Scottish education over the past century has been the integration of women, Catholics and ethnic minorities, and a huge part of that is having a piece of paper that says, “I am equal to other people—I have the talent, the intellect and the capabilities,” and which therefore acts as a passport to prevail against racism or prejudice.

There are merits in the many areas that you have highlighted with regard to comparisons, but we need to ensure that we have that kind of robust culture that sees such an award as having the same value, no matter who has it. That feels like part of the trade-off that you are describing. Would you agree with that?

Yes, that is a powerful argument. What qualifications would be important in the move to equity? It was only 50 years ago that most students were leaving school before they could even get qualifications, and we know that there is an unfairness of access in that respect. I am with the idea of qualifications producing equity, but the issue is what goes into those qualifications. Do they need to be traditional? I am not against traditional exams when they come at the right point and are fit for purpose, but we need to look at what goes into qualifications.

Moreover, if we are trying to level up in this way, are pen-and-paper exams the fairest way of assessing students? That brings us back to the mixed economy. It might be that, for some groups, there are other forms of access, and it then becomes important to think about the use that is made of oral or practical presentations in some traditions and whether that should be part of the qualifications system. There needs to be recognition of what students have done under common terms.

I was going to alight on the word “recognition”, which probably takes me to my next point.

I am sorry, Michael, but you will have to be very quick.

I will be—sorry. I am talking about the culture of recognition as it pertains to universities. For some learners, the next stage in their journey will be going to university, which is a kind of recognition. I am worried about the assessment methodology and whether it is replicated in universities. Essentially, learning the trick of doing an exam at secondary school prepares somebody to do it at the next stage. Do we have to have a certain amount of that to prepare our young people for the next stage, or is there sufficient culture change in higher education to allow us to accommodate that?

We do need some preparation. You will notice that I hardly comment in the report on highers and advanced highers. That is because, on fitness for purpose, if the main purpose is selecting for higher education, and those are the kinds of demands that higher education is going to make, I can see how we would justify advanced highers. We may need to look at their content, how they are taught and so on, but the idea of using them as a selective tool is not much of an issue for me. My concern was that there are three sets of exams in three years, and I am more concerned about the fitness for purpose of the national 5.

I will take some more quick-fire supplementary questions. I emphasise the importance of the quick-fire aspect.

Thank you for your comparative analysis, Professor Stobart. We should be willing to learn from other countries, and we owe you a debt of gratitude for the work that you have done.

My favourite quote about education, which I suspect you will know very well, is from the famous Irish poet William Butler Yeats, who said:

“Education is not the filling of a pail, but the lighting of a fire.”

I think that you alluded to that strand by saying that, although we want a mix of examinations and assessment, the key thing is to inspire young people, engage their interest, get them enthused and let them think that they can do this, and to avoid the scenario in which they turn off because they think that it is beyond them, too difficult or too boring.

If we agree that that is a desirable aim in general, how do you think that we can light that fire more, in practice, for more young people, particularly those who, for whatever reason, become disengaged, perhaps at an early stage in secondary school?

Part of that will be about the kind of teaching and learning offer that we make to young people. That pulls me round again to the vocational side of things for some students and the need for an imaginative offer that involves a more practical, hands-on approach. I am well aware, having taught in inner London, of kids really switching off from education at an early stage. How do we draw them into it and give them an interest? That is why I went on and worked with the National Council for Vocational Qualifications on a school-based vocational qualification. Many students seemed to benefit from that qualification in a way that they were not benefiting from the academic exams. The issue is to do with our imagination about the kind of offer that we can make to students and how we can integrate that within the system.

In the report, I look at systems in Australia, New Zealand and France, where the vocational is part of the qualification or certificate that students receive at the end of the process. For example, in France and Ireland, if someone takes the vocational route, they are still entitled to go to university if they pass the baccalaureate. There are alternative tracks that are well defined. I am not sure how well defined they are in the British tradition.

I picked up from reading your paper that a greater emphasis on and inclusion of vocational education and training at secondary school, at least in the first few years, would be a desirable option to consider. Is that a fair representation of one of your recommendations?

Yes. If the schools have the resources and skills to teach that, I think that it could become a powerful option for students. In some other traditions, it is not just students who struggle with the academic side who do that. Just as some academic students want to do art and music, some might take the more vocationally oriented approaches. It is about school resources and skills, and the willingness to do it.

Thank you.

I will allow one more supplementary question. I appeal to colleagues to make supplementaries really short and pithy.

Thank you, convener. I will be brief.

Good morning, Professor Stobart. There is a suggestion that we should remove national 5 exams in S4. Do you still anticipate there being nat 5 exams in S5, or would they go altogether? That could narrow choice for those who decide to leave formal education after S4. Could there be issues for discrete subject provision in S3 and S4 if we removed nat 5s? Does it have to be a binary choice in relation to removing nat 5s? For example, could we not see nat 5 exams more as an end-of-course external assessment that would give people the course award if they passed, while those who had been continually assessed appropriately through the year could still get the same course award? Could we not expand choice rather than restrict it?

That sounds sensible to me. Again, we might have different routes. It might even be that some national 5 subjects are felt to be central and others could be teacher assessed. That happens in countries such as Norway, where people take one or two exams and the rest of the courses are teacher assessed. Therefore, yes—I am for a variety of routes to get to the same end result.

Stephanie Callaghan has the next line of questions.

Thanks for being here, Professor Stobart. You mentioned that you are a late developer. I did not expect to be in politics, so we are on an equal basis there.

You touched on engagement with students. Obviously, students should be able to influence and help to shape the assessment system, and we should provide opportunities to recognise the full range of young people’s skills and experience and their achievements at school. What can we learn from other countries about engaging with students?

I think that every country would say that it has the same problem, in that not all students are fully engaged in the education system, and there is an issue about what can be done to engage them more or better. The situation varies. I have talked a lot about the vocational side. In Norway, half the students take the vocational track. They can rejoin the school and university track if they wish to go that way, or they can take up an apprenticeship.

10:15  

There are ways of doing student engagement. It is partly about the choices that students have. Are they funnelled into what the school offers? How imaginative are the schools being? Again, I have come across schools that have linked with further education colleges and other schools to provide a broader programme for students. We now have a much more diverse student body, and students are staying in schools for much longer than they used to, so how are we catering for them? Where are the resources for that? Where is the encouragement to link with FE colleges and other schools? I know that there are schools that are models of how to do that.

We talk about engaging and including students in the process of shaping and influencing the new assessment system, but I am concerned that it is easy for student voices to get lost when there are experts and teachers speaking in public and political debates about that. Is there anything from other countries or just more generally that we could do to ensure that those voices have parity and are listened to?

Norway has an annual survey of students in which students comment and give their opinions on what they are getting. That also becomes important feedback for teachers.

I might have missed this, but I was a bit surprised by how little systematic research on pupil perceptions and attitudes is being done in Scotland. Some of the committees have drawn people in, and the Scottish Qualifications Authority did some panel work on that, but I have not seen many longitudinal surveys or much following of students and what they think should happen.

Anecdotally, I have heard students say that they would like more continuous assessment. I am not sure that they know what that means, and sometimes I am not sure that we know what it means when we talk about it. Perhaps a problem for Scotland is that, when we think about teacher assessment, we think about regulated coursework that contributes to the exam grade whereas, in other countries, continuous assessment is just that: it is the teacher assessing the student over a period of time. Work can be selected from that to assess, or it can be done as the Americans do it, with a grade-point average and looking at what the student has done almost every week.

When Scottish students say that they would prefer continuous assessment, are they saying that anything is better than exams, or can they see the value of their work being recognised? That goes back to the earlier point about whether exams motivate students. Continuous assessment might be an important way of motivating students and letting them know that their work has some significance for where they are trying to get to.

You have made a really good point about the engagement process. That is why I am asking these questions. In the evidence that we received from students, they talked about having that breadth and being given opportunities to demonstrate their skills and achievements at school. There seems to be a huge amount of trust in the teachers, which is really positive.

Do you have any comments with regard to young people with additional support needs? An issue that came through quite strongly in that respect was the need for flexibility in the system to allow them to demonstrate their skills and experience. I know that Professor Louise Hayward will be leading the expert group, which will also include people from the curriculum and assessment board. However, does that representation reflect the current situation, given that around 30 per cent of our students have additional support needs? How do you think they can be included or helped to demonstrate their skills in a new system?

Any system that tries to take the views of students into account must look carefully at such a group with special needs and, in a sense, ensure that there is overrepresentation, because the people in question can be easily overlooked. Even though doing so might take a lot of effort and resources out of the system, it is important that that happens, because they, too, need to feel good about education and what they have learned. I realise that the group is not discrete and that it covers a wide range of issues, but we need to ensure that the people involved are fully represented in, say, any sampling of student opinion.

To follow on from the subject area that Stephanie Callaghan asked about, I note that, on page 46 of your report, you quite rightly mentioned

“students’ perceptions and views of assessment arrangements”,

but what about employers and universities? What are their perceptions of assessment?

I cannot claim to speak for universities and employers, but the historical legacy is that they have often been dissatisfied with what they consider students to know when they leave school. You will hear universities say that now, and they were saying it 100 years ago, too. I have some juicy quotes from people despairing of the level of maths and literacy in the students coming through.

That is almost a permanent feature, but it suggests that we need to liaise more carefully on the issue. Indeed, that would fit in with the idea of improving the status of vocational qualifications. I know that employers are involved in the construction of such qualifications, but they could be more supportive and encouraging with regard to such routes. It is up to the education system to listen and ask, “What are employers not finding in the graduates from school? What do universities think is lacking in the students who come to them?” Some of it will be formal educational skills, but some of it might be the very capabilities that we talk about in the curriculum for excellence, such as the ability to work collaboratively and the ability to think for yourself. However, I must plead ignorance of what particular employers are saying and wanting.

Stephanie Callaghan talked about entering politics later on in life. I, too, entered politics later, having previously been an employer, and I agree with what you have said. I suspect that you are also right that the complaints that employers are making these days about the suitability of candidates who turn up for interviews and the internal assessments that they conduct were being made 100 years ago. However, do you agree that employers and universities have a very important voice in this debate?

Yes. Again, we are talking about alignment with the curriculum, but we need alignment between the qualifications and where people are going. That is part of the validity of the system.

I will address directly with you, if I may, part of the struggle that employers have with the education system, to see whether it accords with your experience. Employers sometimes feel that there is a snobbery in the educational establishment about vocational training and about the types of disciplines and professional skills that they are looking for in candidates whom they wish to recruit. Do you agree? Have you come across that?

From an England perspective, I would certainly say that that is the case. There is a terrible saying that vocational qualifications are good for other people’s children. The attitude that, if you cannot do an academic qualification, you should do a vocational one is not held as much elsewhere, but it is around in most cultures. That is unfortunate. As I have said, part of my work was on vocational qualifications that were intended to be attractive to employers and students. There is work to be done on that.

I had the distinct impression from our earlier conversation that I would be pushing on an open door if I shared those ideas with you. How can we change that culture in the educational establishment and change how it portrays vocational qualifications? You have spent a career on that.

I am afraid that the answer to that is slowly. Part of the answer is to produce good vocational routes and qualifications, the benefits of which can be seen. Students need to be seen to do well. That is where business comes in. We need champions from business and higher education, which still has a lot of work to do on recognising the issue and making vocational qualifications an entry route to higher education.

The approach needs to come from all sides, and there needs to be a mindset change within schools and parents. I can see why we make such slow progress on the matter. There is a lot to be done.

We discard and discount a lot of progressive thinking on vocational training for the reasons that you have given.

I apologise for being late. If this point has already been covered, feel free just to tell me to check the Official Report.

I will move the discussion about the skills that employers require away from the framing of academic or vocational qualifications. With either of those, we still use an individual form of assessment. As an employer, when I interview people, I am interested in their ability to work as part of a team and their skills in communicating with other individuals. Those are inherently not skills that we can assess individually, because they are about interaction with other people.

Whether in academic or vocational qualifications and assessment systems, is there best practice elsewhere for how to assess the kind of skills that we cannot assess in individual tasks? How do we assess someone’s ability to interact with other people in whatever form?

That has concerned many people, and attempts have been made to address it. How do we get down to collaborative skills and working with others? That has to be done through classroom assessment rather than through any kind of exam system. The OECD has attempted to do that with group skills using computer-based tasks to work with others, but that always feels—no, I will not say any more on that.

10:30  

Part of our skills development and of what we ask in assessment might be that students have to work together, but I am aware that that is difficult to assess. If you have a group of five students, how do you assess the contribution of each? However, that should be part of the system, and students should learn how to do that, because an employer does not want somebody who has no idea how to work with others or how to collaborate and be creative in a group. That pushes me back to a broader curriculum.

Ross Greer is joining us from the 26th United Nations climate change conference of the parties—COP26. We are glad to have him on board. Michael Marra will take us in a slightly different direction.

Professor Stobart has raised the issue of multilevel teaching a couple of times. An awful lot of teachers have made representations to me about that over a number of years. There are huge problems with it. They do not see it as multilevel teaching; in essence, it is multiqualification teaching—teaching different syllabuses, examination processes or assessment processes in the same classroom. Do other jurisdictions or countries do that?

I do not think that they do it to the same extent. It is partly a feature of having three levels of examinations in the senior secondary school. Most systems have only one or, at most, two levels of qualification. I am sure that, in small schools all over the place—in Norway and the like—staff have to do some multilevel teaching, but in Scotland there are different syllabuses for the different qualifications, and it is difficult to put those together coherently in a classroom.

Scotland has got itself a problem with multilevel qualifications. It takes a great deal of teaching skill to take three groups through their qualifications in the same classroom, particularly when the syllabuses are a bit different for each of them.

It strikes me that, if there is an opportunity to solve that problem or at least to do something about it through the process of reform, that opportunity must be grasped. The Education and Skills Committee in the previous parliamentary session concluded that something had to be done on the issue, and nothing has been done. There is a confluence of resourcing issues, but I think that your analysis is that it is a structural issue as well as a resourcing one. Is that your conclusion?

Yes, I would say that. Life is made a lot more difficult for a teacher if they are teaching for three qualifications in the same class. I would encourage ways of making the system more supportive of teaching and learning.

That is useful.

I want to ask about data. I was interested in your points on the annual student surveys that are carried out in other jurisdictions and countries. What kind of data do we require in Scottish education to monitor effectively the reform process? Is there sufficient data?

I do not think that I have experienced the full range of data. There is a lot of data, but the issue might be how coherent it is or how it is pulled together. There are regional data, authority data and Government data. The critical point is how that is co-ordinated.

The methodology of your paper is an international comparison. One of the contradictions for us in that is that the Government has withdrawn us from international comparative studies. Is there value in such studies for evaluating performance?

I go back to the question whether there is value for students in them or for teachers, who make little use of international comparisons. They may be useful at Government level in determining progress, but there are other ways of determining whether there is progress in a country’s assessment system. I do not want to get too involved in the politics of whether we are in or out of various international comparisons.

Convener, this is a sign of my age, but could I have a two-minute comfort break, please?

Absolutely. I am delighted to facilitate that. You might be speaking for a number of the rest of us as well. We will suspend for about five minutes.

10:36 Meeting suspended.  

10:41 On resuming—  

I welcome everyone back to this evidence-taking session with Professor Stobart. I call Oliver Mundell.

I am mindful that, during the evidence-taking session with the OECD, I was accused of being rude in my questioning. I hope, therefore, that Professor Stobart will see that I am not being rude—I asked the OECD similarly robust questions.

Leading Scottish educationalist Professor Lindsay Paterson has stated that your review is “awful”,

“ignorant of Scottish educational history”

and

“ignorant of current Scottish practice, failing to see that Highers remain the main entry requirement for university, not Advanced Highers.”

He said:

“It is not based on any systematic comparison with non-UK countries, but rather with an arbitrarily chosen group of places that seem to have been selected to make the case against exams.”

Furthermore, he notes that you fail to

“discuss the unfairness of non-exam assessment, for example the unavoidable advantage enjoyed by children from affluent homes with well-educated parents.”

How do you respond to those concerns?

I am aware of Lindsay Paterson’s work and his response, and I respect the fact that his work is concerned about equity and fairness in the system. The sample was purposive in that we chose four or five systems in the British tradition and then other systems that we thought would be instructive in this respect. We could have chosen others, but I am not sure that they would have been illuminating or useful to Scotland in the same way. We could, for example, have chosen China, India and many other places, but we tried to be representative of some broad strands.

Your point about the non-exam assessment was that we did not look at the problems of not having exams.

That was Professor Paterson’s point, and it was also highlighted by the Scottish commission on school reform, which has said:

“scrapping examinations and relying purely on teacher judgment would create a series of perverse outcomes including:

• Unintentional bias for or against certain social or demographic groups (for example against children from deprived backgrounds ... )

• Pressure put on teachers to award the grades required for university entrance, particularly in private schools and in the most affluent state school catchment areas

• Grade inflation caused by teachers wanting their children to succeed”.

You make a case against exams, but you have not touched on any of those issues.

10:45  

No, I have not, and you are right to raise them. There is a cultural issue here, in that there are cultures in which you cannot rely on teacher assessment, because there is sufficient corruption in the system and pressure on teachers to award certain grades. That happens in various countries; certainly, the ex-Soviet countries have had to deal with corrupt systems of assessment and entry to university and as a result have introduced very standardised exams. The Scottish culture is not in the same position. It has a strong professional workforce and is not, in that sense, a corrupt society.

I accept that those who come from affluent homes have advantages, but they also have advantages in the exam system in the way that they are prepared for them, the groups in which they are prepared and so on. I do not think that exams remove the problem of privilege.

What about grade inflation, which has been seen across the UK over the past two years? The argument made by some—I tend to believe it, based on my constituency experience—is that it ends up disadvantaging those who face the greatest challenges. You talk about the suite of considerations for entry to university, but these are the very young people who cannot access good-quality work experience, who do not have the same opportunities to take part in extracurricular activities and who do not have access to coaching for university entry exams. Why is grade inflation a good thing for them?

Grade inflation is not a good thing if it does not accurately reflect performance. There is an issue in that respect, and we noticed it in England during the pandemic in the pass rates and the like. That is problematic and needs to be dealt with. It might involve expectations with regard to coursework and teacher assessment, and it might mean having that mixed economy and striking a balance between examinations and teacher assessment. That is how most cultures work.

The American and Canadian example is interesting in that teacher assessment forms only part of the selection process. The British tradition relies very heavily on the grades that pupils get in an exam, and the American system makes more room for teacher assessment while putting in other checks and balances. We might need to think about, for example, the entry requirements for progressing to the next level in education.

However, I want to challenge your point. Some disadvantaged students will be further disadvantaged because of resources, the ability to do work at home and the like, but I would point out that they are also disadvantaged if they do not have any cultural access to exams and their content. It is a balancing act.

Many people in Scotland will look at the suggestion that we should become more like the US, which is probably regarded as one of the most unequal countries in the world when it comes to access to education, and think that that would be a serious departure from the Scottish education tradition. In that tradition, the aspiration at least is that every young person will leave school with a meaningful qualification. Getting rid of that seems distinctly un-Scottish. I do not see how you feel that we can achieve equality of opportunity by removing the chance to sit exams for some young people but not others. Should we not be asking why some young people are leaving school without qualifications rather than lowering the bar for a group of young people who are consistently failed? Is that not a valid point?

Yes—it is about the fairness of the system, particularly for those in disadvantaged circumstances, is it not? I realise that exams are seen as fair because everybody takes the same exams. However, not everybody has equal access to the system or the same cultural background to give them a fair chance. Therefore, I am with you on that struggle to find a fair system and to work out how to be fair to all groups in the community, but that should not narrow it down to exams, at which many of the disadvantaged are not particularly successful. Many struggle with exams, which might be for cultural and preparation reasons, in the same way that they might have problems with class work.

I would argue that a change to the system does not really change the prospects for those young people.

Why did the OECD approach you to write the review? My concern is that it asked you to conduct the review because it already knew your long-held and well-known views on the diminishing importance and validity of examinations, which are well explored in your work. Does that not create a risk of groupthink, whereby outside experts come in and tell us that curriculum for excellence is a world-leading, groundbreaking move, despite the fact that people living and learning in Scotland continue to see our once-world-leading education system being dismantled and slipping backwards?

You talk about culture, but would it not be better to work with Scottish culture to recognise the importance of Scottish educational traditions and seek to improve the examination offer, rather than going for more radical reform that might not command the trust and confidence of parents, teachers, young people and employers?

I take your point that I have a position on that, but my employment record alone will tell you that I am not anti-examination. I am very interested in fitness for purpose and in what purpose an exam serves. I have mentioned that highers and advanced highers have a clearer role and purpose. The national 5 was my particular concern with regard to what justice it does to students, which I think is your concern.

I am reading into what you say that you have traditional Scottish values, but I would have thought that Scottish culture is changing in the sense of the diversity of students and the fact that students are staying on in education—they have broader interests and motivations than they might have had 50 years ago. My concern about exam reform is how we successfully engage and cater for them. Curriculum reform is happening everywhere. We are all trying to do exam reform to match and align with the curriculum.

I just think that it is about recognising that successful learners are the people who go on to be active contributors and to make important decisions in our society. If we were to stop helping people to gain the qualifications that they need to break through the barriers that exist, that would be sad. That is what has been so powerful about Scottish education across the best part of a century.

In a sense, we have common aims, but we have very different routes to achieving them. I would be forward facing and look at how we can meet the needs of contemporary students, who are different to those of 50 years ago, and how the qualification system can do justice to them. I think that we disagree over the role of examinations. We could take the discussion further on what the difference is between an exam and a qualification.

The difference is about external assessment and the validity that comes from that. I will leave it there. I do not want to dominate the discussion.

A number of colleagues want to come in with quick-fire supplementary questions on the back of that line of questioning.

Professor Stobart, given Oliver Mundell’s ludicrous and fairly insulting assertions about your qualifications and work history, will you confirm again for the record that you have no intention of suggesting that we scrap exams totally—you have made it quite clear that there is a place for exams—and that your suggestions are about the place for exams and where they sit best?

You have said it better than I would have done. I am not averse to examinations if their purpose can be clearly demonstrated. I would also look at other forms of assessment. I take assessment more broadly than meaning simply exams.

James was obviously sharing his views on his colleague’s point of view. I hope that we will be able to maintain a degree of good manners, which I think we have done so far.

Professor Stobart, you made the point about the fact that there can be corruption as far as assessment—[Inaudible.]—so I just wanted to check that you are satisfied that Scotland’s approach—[Inaudible.]—fair and robust in that standards were checked across teachers’ approaches in schools and departments, across local authorities and nationally. I want to check that that is a good indicator and that, if we are going in that direction, you have confidence that—

I am sorry to interrupt you, Stephanie, but perhaps you would reprise your question to Professor Stobart. We lost you for a few seconds, which broke up the question.

That is absolutely fine. Professor Stobart has spoken about the fact that there has been corruption with teacher assessment. In the light of the way in which Scotland has dealt with assessments during Covid—in other words, teachers in schools checking one another’s standards and the checking of standards across local authority areas and nationally—I want to check whether he feels confident that it would be fair and robust to use teacher assessment in the future.

I am confident. My reading of it is that Scottish teachers are skilled professionals. As I said at the start of the meeting, they are asked to assess throughout school, and we trust our university lecturers and further education staff. It is a system in which there is a large degree of trust, professional recognition and professional qualification. I am comfortable in that regard.

11:00  

I have been thinking about Oliver Mundell’s point about inflated grades. I think that it is partly to do with the nature of the pandemic and lockdown, and it is about how to give the benefit of the doubt and how to make accommodations for what teachers know that their students have been through. There is clearly an element of that.

If we were looking at having more continuous teacher assessment, we should consider that it does not come automatically. We need professional development, professional discussions and networks through which we check standards. Queensland is interesting in the way that teachers there come together with their students’ work, look at other schools and get consensus about the standards and grading that might come from discussion. We need that kind of approach, with checks and balances in the system.

It is great to hear that you have such a high opinion of our teachers’ professionalism and honesty.

I, too, had taken very clearly from Professor Stobart’s paper that he is proposing not the abolition of examinations but a far more nuanced set of suggestions. Professor Stobart, what do you consider to be the benefits and disbenefits, if you like, of examinations as a method of helping young people to prepare for later life and to develop themselves under the CFE principles? We have heard from Mr Rennie and others about the potential advantages of examinations, but they also have adverse elements for some children. How do you see the overall benefits and disbenefits of examinations as a method to use in devising a system that prepares children for later life?

A lot of that depends on the quality of the examinations and the kind of teaching and learning that encourages students along the way. If it is just about the grind of facts and recall, we might question its value. The comments that I heard about advanced highers—even from students—were that they make learners think more. They are about showing students’ knowledge and how they can be flexible with it, which has positive advantages.

It is partly about the quality of the exam, which really affects the quality of the teaching and learning that go with it. I am interested in learning—the kind of learning that the system or exams require. That is double edged. It can be just a memory job, trying to get the content in plus going through past papers—that is perhaps the risk of the two-term dash. I saw the committee’s discussion with Beatriz Pont at a previous meeting, and I am interested in what we mean by knowledge. We need content, but we need more than that. It is about how students put the content together into ways that they understand and can apply to situations.

That was a very long-winded way of saying that you are right—I am not about abolishing examinations; rather, I am looking at the quality of examinations and the quality of teaching and learning in preparation for examinations, and I am raising the question of whether, if students are racing from one exam to another, that affects the quality of the learning.

My sort of guiding outcome in all this was whether the student can think for himself or herself. We are trying to help students to think for themselves. Does our system do that? Does what goes on in the classroom do that? Are students being encouraged to think for themselves?

So, examinations should not be so much about the regurgitation of facts, with no underlying purpose other than as an exercise in recollection; they should be about the promotion of better understanding, rational analysis and the ability to think for oneself. Is that the sort of thing that examinations should try to achieve, rather than fact memorisation?

You have put it a lot better than I have. Yes, exactly. I would endorse that.

I do not know whether I have put it better than you. Incidentally, I think that W B Yeats nicked his quote from Socrates, but there we are. It is just a random reflection.

That is evidence of what knowledge can empower you to contribute to a committee meeting—well done, Fergus. You did that from memory, which is probably an object study in itself.

We turn now to Ross Greer for a new line of questioning.

I would like to return to the issue of equity in the system. I am specifically interested in the impact of exams or alternative certification models on pupils with additional support needs. I realise that “additional support needs” is a term that encompasses all sorts of needs, some of which may result in a young person finding an exam easier to access and some of which may mean that a young person finds continuous assessment easier to access.

In recent years, the response to criticism of how we support pupils with additional needs through exams has been simply to extend the length of time that those pupils get to complete the exam. On the one hand, that is understandable, and it provides those pupils with an additional opportunity, but, for some young folk with additional support needs, sitting in an exam hall for three and a half or four hours is even more challenging than it would be for any other young person. Your report did not mention additional support needs specifically. Is it an issue that you touched on at all? Did the issue of providing equity to those who have a wide spectrum of additional needs come up in the alternative systems that you looked at or in the best practice from elsewhere?

I did not focus on that, either in Scotland or comparatively. Again, part of my track record is that I retrained as an educational psychologist, so I am well aware of the range of needs and of the fact that, sometimes, just giving extra time, or an amanuensis if there are transcription problems and the like, does not do the job.

We have to think very sensitively about this. It is even more complex an issue with vocational qualifications, because some of it is about asking, “Can you do this?”, and the question of how we make allowances for a competence is very difficult. I have got no clever answers. What I would say is that we need to give proper attention to the issue and not just say, “We’ll give them extra time.” That is, dare I say it, an easy cop-out. Some needs require a more skilled approach to assessment. Again, it is the idea of increasing flexibility in the system to take on diverse student needs and say, “Here’s a route that we can use to do this.”

In your conclusion, you mention the potential need to move away from the SQA’s relatively demanding quality assurance processes if we were to move towards a system that had less external assessment. There is a strong cultural attachment to external assessment and verification. Will you expand on why it is not necessarily essential? If we compare that cultural attachment in Scotland to the position in other systems, does it ultimately come back to trust in teachers being perceived differently elsewhere and to trust in the system or are there other cultural factors that we would need to work on in Scotland if we were to move away from our current system of external assessment and verification?

There are strong cultural factors in this. It is partly because the teacher contributions within the exam system have been the coursework that adds marks to the final grades, and it is about the checks that go into that.

My concern is that, if we say that we need more teacher assessment, we will have resistance from teachers. We already had that in 2016, for instance, when the units were being assessed. That was just overwhelming, so I am aware that any new system must not increase the burden of assessment on teachers unbearably. That is why we might encourage them to make the cultural shift to the alternative system of continuous assessment that is used in other countries such as the United States, Canada and Norway, in which teachers are left to make judgments about the quality of student work during the course.

It is a good question. Cultural changes would be needed and we would need to convince teachers and parents—particularly teachers—that it would not involve huge amounts of extra coursework. It would be more continuous, day-to-day assessment that was carried forward. That does not entail a great deal more work, because teachers do it anyway.

We would need a shift, and you are better placed than I am to say how you would get that change of attitude.

Part of the issue is that, if we move away from exams towards some form of continuous assessment, there is likely to be an additional workload for teachers in that system. There are other opportunities in our education system to reduce teacher workload, such as changes that we can make to curriculum for excellence, if workload increases through continuous assessment.

My impression, from speaking to a lot of teachers in recent years, is that they are inclined towards a system of continuous assessment. They see the advantages of it, but the personal workload burden holds them back from adopting a system that they otherwise understand a lot of the attractions of.

I will bring in the example from Queensland, where teachers get together and moderate the work. That is professional development at the same time. It is a check on their own assessment of the students. Perhaps such a local approach would be a possibility.

That raises an interesting point about how a change in assessment might interact with the reform of the school inspection system and what role peer assessment between teachers might have as we create a new inspectorate after the current review. The committee should keep an eye on those overlapping pieces of work.

A couple of colleagues said that they might want to ask additional questions, so I will turn to them. James, do you want to come back in or are you content?

I am fine, thank you, convener.

Bob, would you like to come back in or are you content?

11:15  

I am content, convener. I have appreciated Professor Stobart’s evidence, but thank you for the opportunity.

We really thank you, Professor Stobart. We have put you through your paces over the past hour and three quarters, and you have given us lots to think about. It has been a robust session. It has been honest, frank and fair. On behalf of all members, I thank you for your paper and your presence at the committee.

The public part of today’s meeting is now at an end. I ask members to reconvene on Microsoft Teams, which will allow us to consider our next agenda items in private.

11:16 Meeting continued in private until 11:47.