Education and Skills Committee

Meeting date: Wednesday, February 6, 2019

Official Report 769KB pdf

---

Scottish National Standardised Assessments Inquiry

Good morning and welcome to the fifth meeting in 2019 of the Education and Skills Committee. I remind everyone to turn their phones and other devices to silent for the duration of the meeting. We have received apologies from Iain Gray and Gordon MacDonald.

The first agenda item is our inquiry into the Scottish national standardised assessments. This is our fifth evidence-taking session. I warmly welcome Lindsay Law, who is the convener of Connect; James McEnaney, who is a lecturer and journalist; Darren Northcott, who is the national official for education at the National Association of Schoolmasters Union of Women Teachers; and Susan Quinn, who is the education committee convener at the Educational Institute of Scotland.

We have a big committee and a big panel. Witnesses may not want to contribute to every question that is asked, but please indicate to me and the clerks when you want to respond. For the benefit of those watching, I explain that the committee had an informal meeting with teachers this morning, about the Scottish national standardised assessments. The matters that we discussed may be raised in our formal evidence session. I record my thanks to all those who attended that informal session.

Will each witness briefly outline their experience as it relates to our inquiry, please?

Connect is a parents group and registered charity. We support parents and carers all over Scotland. We have a membership model, so parent councils and parent-teacher associations are members of Connect, and they can access additional services.

I have been on Connect’s board since, I think, 2016. Prior to that, I was the parent representative on the education committee of Edinburgh City Council, and I have been involved as a parent helper or in parent councils since my daughter started nursery school in 2007.

I am the EIS’s education committee convener, which is an elected position. I am a primary teacher by trade, and I was a primary headteacher up until the past few years. I have been involved in the work on the SNSAs, and in the wider assessment curriculum work as a member of the curriculum for excellence management board and subsequently the follow-up boards and different groups in relation to that work.

I was a secondary school English teacher and I am now a further education lecturer. I suppose that I am here because, when the policy was first announced, I tried to investigate its origins through what were at that point some of my first freedom of information requests. I spent a year looking at the issue and eventually got the information published. Since then, I have had an interest in looking at the development of the policy and how it has interacted with, for example, Government transparency and policy making.

Good morning. I am a primary teacher by background. NASUWT represents members in Scotland who are engaging directly with the SNSAs. We also represent members across the United Kingdom, and the committee might find it interesting to compare and contrast the different approaches in the UK. We operate at the international level, so we have a great deal of experience in how other jurisdictions introduce nationwide and system-wide assessments. I hope that there is something that we can share that is of use to the committee.

I am particularly interested in the purpose of the standardised assessments, which is the central dilemma that the committee faces. At the evidence sessions prior to this one, it has been put to us that there is a dilemma because, in some cases, the assessments are being used not only to measure a child’s individual performance, but for summative purposes, so that schools and local authorities can drill down into where there are underperformance issues.

I am interested in hearing your views, from your professional backgrounds, on where we can go with that dilemma. There is obviously the extremely important argument that what matters most in relation to educational assessment is the best interests of the child, but on a national basis there is also concern about underperformance and whether the assessment can be used in that context. How do we address that dilemma?

Shortly before the announcement that SNSAs would come into play, there was a national discussion about how we would gather information about education in Scotland. The Scottish survey of literacy and numeracy had been in place for a number of years and there was some challenge around it towards the end of its existence. As part of the work of the management board and wider stakeholders with the then cabinet secretary for education, there was a discussion about what was needed in the future.

At those meetings, stakeholders generally agreed that there is a body of evidence in our system that tells us everything that we need to know; the challenge is how it can be gathered to give elected politicians a national picture that is easily understood. There was no feeling in the room that there was a requirement to introduce an additional, national test into the system.

From that process, a form of assessment or test—whatever we want to call it, because we are at cross purposes on it—has been developed to take account of some of the concerns that trade unions and parent groups raised at the start of the process. We did not want to return to the national tests that we had in the 5 to 14 curriculum.

What we have is an approach that tries to do all things for all people, and it potentially cannot do everything. The SNSA that has been developed seeks to provide diagnostic information—across only about 10 per cent of the curriculum base, so it is not providing the widest information to schools—and can be used to look at individuals and groups in classes.

One of the challenges is how the SNSAs have been implemented in local authorities. Had it been left to teachers and schools to decide whether and when they required to use them to inform their professional judgment, the SNSAs might well have been a valid resource in addition to what is already there. As you have heard, schools use a broad range of assessments, including a number of standardised tests and assessments for specific purposes. Those will continue, regardless of what happens with the SNSA.

On what the SNSAs provide in the national context, the question is how reliable the information is if we use it to drill down for teachers. Assessments will be done at different times of the year, with different groups of young people who receive different support, so they are not standardised in the broad sense of the word. That brings problems.

We suggest that the information about achieving a level, which is now being gathered and that sits within the national improvement framework bank, is the kind of evidence that you should be looking at. It is information that will interest parents, because it gives the broadest picture of the young person, rather than a picture of just 10 per cent of their learning over a period of time.

As I said, the question was how reliable the information would be. The information is becoming more reliable as teachers and schools work together to moderate the information and get a better understanding of the benchmarks, which were introduced only in the past year or two—we were working with a curriculum, and then benchmarks were introduced after the fact, so a lot of work needed to be done.

That could have been resolved before the SNSAs were introduced, and I argue that it should have been resolved. Teachers should have been given the time, training and space that they needed to better understand the levels and the standards that were required to make the system more reliable. If they had, we might not have needed to have gone with the SNSA.

Thank you very much for a very full answer. Does Lindsay Law think that parents understand the purpose of the new assessments as being to inform them about how well their child is doing at school? Is that purpose clear?

No, it is not clear. That is partly because, although they are described as standardised tests, which implies to parents that they happen at a certain time and in a certain way, they have been described and communicated to parents in different ways in each local authority area—in fact, in each school. Anecdotally, some parents have told us about letters coming home that tell them that the test is mandatory, that they have no option, and that they cannot withdraw their child from it. That is in the milieu of a dysfunctional and difficult relationship between central Government and local authorities. Parents and children are being used as a playing piece in that dysfunctional relationship, and that is not helpful to the individual child in the classroom.

On the point about what parents get from schools, there has been a complete cultural change in Scottish education over the past decade. I have reflected on the reports that my parents got when I was in school—my mum kept them, so I have them. There were lines and scores on maths and English, and the words “good”, “better” and “best” were used.

My daughter’s reports, especially from primary school, are much more descriptive. They talk about the broad general education, the outcomes in the curriculum for excellence, and how the child is becoming a responsible citizen and an effective contributor. Those things cannot be measured once, in primary 1, with a very narrow focus on literacy and numeracy. Suggesting that they can be, confuses parents. When parents get school reports that simply have a whole load of descriptions about outcomes that they are not familiar with, it is difficult for teachers to get parents to a place that is removed from when they were at school and that is simply about asking, “How are you doing?”, “What level are you at?” and “What progress are you making?” against very easy to understand scores.

What parents and teachers need to do—and what we encourage at Connect—is have a conversation about the potential of the child, how the child is getting on at school, and what we can do across the whole curriculum. That narrows the focus and somewhat undoes the work that has been done on the assessment and the monitoring of progress in the curriculum for excellence.

I recall that, when curriculum for excellence was introduced, people from the Scottish Qualifications Authority and the National Parent Forum of Scotland came to the consultation committee with parents and told us that it would take some time for CFE to work through the system and for schools and teachers to become proficient at assessing where children are. Parents do not have the time. A parent has one shot at schooling, and their child has one opportunity to meet their potential. If the standardised assessments are aimed at improving the system over the long term, but are also intended to improve the experience of and outcomes for children in the classroom, it is very difficult to see how they will do either.

Thank you. I want to ask about the comment on a dysfunctional relationship between the national Government and local government. Do the witnesses think that that is because the national Government and local government are looking to the tests for different reasons and different ways of assessing?

I will respond to that briefly. I think that in your first question you said that the purpose of the tests could be to help teachers to make effective teaching and learning decisions about the next stages in the learning journeys of pupils and to get information about where the system is at the national, local and school levels. In other words, the one assessment would have a formative purpose and a summative purpose. That has never been achieved anywhere.

10:45  

I think that there is a prior step of analysing whether we want the assessments to be formative or summative. It is difficult for one assessment to achieve both aims. I was taken by the evidence that the committee received from Education Scotland that said on one page that the tests are designed to be used formatively and not as summative assessments, then on the very next page talked about how the assessments have been used to form judgments about the effectiveness of a particular department in a school.

If there is confusion at that strategic level, it is not surprising that teachers and parents question the purpose of the assessment and how it should be used. Formative and summative assessments are completely legitimate policy aims, but we have to be clear about what the assessment is trying to achieve and whether it is formative or summative. Experience tells us that trying to achieve both aims is very difficult.

As various people have said, the word “confusion” keeps coming up. My view is a bit different from other people’s because of my interaction with the issue. Before coming here, I looked through all the material that I have dug up over the past three and a half years—all the email chains, documents, policies and everything that has shifted all over the place—and “confusion” is the word that sums up everything most effectively.

As Darren Northcott said, we have a situation whereby we claim to be trying to have a testing system that will perform a summative function, which is about ensuring that teacher judgment is reliable and that we can trust it—which is a whole separate debate. Apparently, we are also using the assessments, as has been said, as a formative assessment to tell teachers more about each child in the class. I agree that the notion that we will be able to combine those two functions in a single assessment is optimistic at best. I would be surprised if there was any evidence to show that it is likely to be possible. In trying to do both jobs at once, we are probably doing neither of them well and potentially doing quite a lot of damage while we are doing that.

It goes back to the origins of it all. When the testing system was put forward initially, it was clearly conceived to be about national-level data. It was going to be a national measurement because that was what we needed. Ultimately, that is why it has incorrectly been seen as having replaced the SSLN.

We then go from there to saying, “Well, we’ll just use the test to inform teacher judgments, so that’ll be the national picture and the test will be part of the teacher judgment as well.” However, when we confuse the two purposes of the test, we are also in the position of saying, “We trust teachers to make these judgments and we’re going to rely on teacher judgment as a national measure of an education system”. Somehow, however, we do not trust teachers to make those judgments, unless they are using a standardised testing system that we think has two or three different purposes.

That goes on all the way through the testing system. There is no level of the testing system that I have looked at over the past three and half years out of which some screaming contradiction does not come. To be honest, we are in a situation now where, three and a half years ago, a lot of people said we would be: in a committee discussing what a formative assessment is and what a summative assessment is, and whether we are ever going to get any closer to having some sort of magical testing system that gives us national data.

Confusion about summative and formative systems comes up all the time, as does the argument that this is some sort of formative testing system. It came up most recently when, in response to an FOI request, the Scottish Government cited two academics as having supported the policy. When they both said that that wasn’t true, the defence was, “Well, we thought that they supported formative assessment methods, so we’re really sorry if we misquoted them.”

I contacted one of those individuals, Professor Dylan Wiliam, and put to him the First Minister’s response during First Minister’s question time. My contact was actually for a story, but there are some things going on right now making it kind of hard to get stories in the press that are not to do with Brexit. His response was quite clear: the tests, specifically in primary 1, do not provide useful formative information. Anybody who knows anything about education will probably understand why Professor Dylan Wiliam saying that is quite a big deal.

We are not going to get any further forward—and that is before we even get to ideas about closing attainment gaps and dealing with poverty through schools—until we can nail down what the testing system is supposed to be about. We are still no closer to doing that three and a half years after Nicola Sturgeon’s “judge me on my record” speech in Wester Hailes.

I do not wish to trade in professors, but a professor has been mentioned and some important points have been made about the purpose of the assessments. I am interested in the point about their potentially having more than one purpose. We have heard evidence on that in previous evidence-taking sessions. Professor Hargreaves said:

“There is a general principle that many, but not all, people accept, whereby data that is collected for one purpose should not be used for another, but that does not mean that data should not be collected for two purposes.”—[Official Report, Education and Skills Committee, 30 January 2019; c 6.]

My question is on a subject that some of you have pointed to, which is the need for, and the pressure on, local authorities to close the attainment gap and to deal with the inequalities that exist very early in children’s lives. If you do not want the test data to be used, what data should be used as a benchmark for local authorities and others in trying to address the problems?

There are benchmarks that schools and early years establishments use for young people from the very earliest stages. The widest possible assessments go on. Young people are assessed by health visitors, they are assessed their early years establishments, with a number of approaches being taken, and they are assessed in schools, where benchmarking and early intervention strategies are put in place from the beginning.

Very few primary schools do not assess young people on their entry to primary 1. Primary schools do that using a range of strategies, including teachers’ observations of the young people at play and schools’ own formal approaches to assessment. There is a host of pieces of information about the young people from the very earliest stages.

At the end of primary 1, teachers make a professional judgment of the young person’s achievement at the early level based on three years of assessment of the young person. It is not done in, as we are generally told, a single half-hour session to assess their literacy and a single half-hour session to assess their numeracy, which would not give us information for closing the attainment gap. The data will always come from teachers’ professional judgment and informal assessments, and from the informal interactions that happen day and daily between teachers and pupils, between support workers and pupils, and between parents, teachers and everybody who is involved with the young person.

That information allows us to see whether the young person is achieving the appropriate level at a point in time—at the end of primary 1, primary 4 and so on. We take into account any other specific diagnostic assessments that are identified as being potentially helpful to an individual young person, because there has been a conversation about whether the child might have additional support needs. The SNSAs will not identify whether a young person has additional support needs, for example: other assessments are required for that, and they will be done only if there is a really broad basis for doing so.

If you are asking whether the assessments are going to be the benchmark by which you can determine whether the additional money for pupil equity funding has had an impact, the answer is no. The single half-hour assessments, or however long it takes to do them, will not be that. It will be the broad assessment bank that goes on in schools day and daily, and the conversations between head teachers and their teachers about whether the young people are achieving the level based on the benchmarks that have been introduced and the experiences and outcomes that they have been planning around, that will do that.

My first point is that there is confusion among parents about the diagnostic nature of the tests: parents might expect to get a diagnosis of dyslexia or something else that might need additional support.

My second point is about gathering data and closing the attainment gap. When young people leave school, we do not talk about their literacy and numeracy; we talk about positive destinations and the people whom they become. Schools are becoming much more creative in their understanding of the broad general education and the curriculum. They are undertaking work with colleges and local enterprises, bringing the community into the school, and taking the school out into the community. That is all to create positive destinations that might or might not be academic for our young people when they leave school.

A narrow focus at each stage on literacy and numeracy detracts from the message that you have sent to the people of Scotland about the broad general education and what curriculum for excellence is supposed to attain. My concern is that schools’ attention will be drawn away from some of the great work that they are doing on positive destinations by the pressure on them from local authorities, and by the pressure from official or unofficial league tables that might spring up off the back of the data from a narrow focus on literacy and numeracy. That is exactly what curriculum for excellence was supposed to draw us away from, so that we could focus on the whole child and the whole young person and on how they move forward to become effective, happy and responsible citizens.

I accept what Ms Quinn said about the importance of conversations that take place throughout a child’s school career that include parents, the local authority and other sources of information. However, do they provide data that local authorities can use to make policy interventions? I appreciate that we are talking about two things at the same time: interventions in the life of a child and policy interventions. Do those conversations provide information that would allow local authorities to make policy interventions?

They should do that. With schools talking more outwardly to each other and across local authorities, the information is about the kinds of intervention a school has been developing and whether they have made a difference to its achieving a level, and about data in its broadest sense. The conversation might be about whether interventions are transferable elsewhere, with the acceptance and understanding that education is not a one-size-fits-all thing. You have to look at the context.

I am absolutely clear that, if the conversations are being conducted properly and there are time and space in which to have them, evidence will be based on much more than the narrow approach of the standardised assessments.

I note that the information about the standardised assessments that has been gathered nationally is being used to consider whether there are norms in specific areas. It will tell you about only 10 per cent of the curriculum at a point in time that is very different from what happens later. We do not have common approaches to issues across the country because the country is not a common space: what works in one place might not work in another.

Using the wider educational information makes a difference to local policy, and it has done so in the past. I have sat on Glasgow City Council’s assessment and curriculum group on a number of occasions over the years. We used information from our schools—the information that head teachers were telling us—to drive policy at local level.

11:00  

I will return to the first question about whether there is a need for national and local bodies to have data on educational performance. There is, because they are public bodies that are run by democratically accountable and elected people, and they need good information in order to make national and local policy. The issue is that if they try to make that policy based on a very narrow range of indicators or on a single indicator, they might not make the best policy. National assessment might have a role to play, as long as its role is clear. As we have heard, it is important to ensure that other sources of data and information are used.

It is not just about assessments; inspection has a really important role to play in giving policy makers at national and local levels an understanding of progress in the education system, the impact of interventions generally and on specific groups, and what policy needs to be implemented in order to address problems that are identified. It is about having a range of good-quality information and data on which to base policy decisions, rather than simply focusing on one standardised assessment across the system—although that might have a small part to play. We need to put it in perspective.

I will make two points, one of which goes back to the point about having national level and council-level data. We had something that gave us national-level data—the SSLN—but the information was not broken down to council level.

The University of Glasgow produced a paper entitled “Assessment at Transition” that looked at all sorts of different things. One of its recommendations, or ideas, was to expand the SSLN to include local-authority level breakdowns. That idea would have been worth looking at, and could have taken us some way towards having a national picture that we could also have looked at in a more localised fashion, without ever needing to consider how we could get school-level data about every single school. Going down to that level of data could create problems that would outweigh any benefits that we got from it.

Susan Quinn referred to something that came up in the informal session—having time and space. The idea seems to be that the testing will produce data that will allow a council or the Government to make policy interventions—along the lines of, “This bit of data is going to let us go and do this thing, which will make things better.” However, as people have said, the tests will simply not be able to do that. There is no point in kidding ourselves that they will.

If we are to get information, create improvements and have a system in which we can achieve that sort of progress, time and space are key. We need a system in which professionals are able to be professionals and in which there is time and space to have—at school level, local-authority level and national level—discussions such as have been mentioned. However, there is no convenient single way to do that, or an easy or quick answer. Getting information that is reliable enough to act on, and transferring useful and workable interventions from one place to another, is necessarily a slow and careful process.

The other kind of process focuses on getting data quickly and finding a use for it, or telling ourselves that we have a use for it because we feel that we need to be seen to be doing something. A lot of damage is done a lot of the time by the constant need to be seen to be active, instead of carefully working through slow processes, which people at every level of education say consistently is key and should be focused on.

The risk with the SNSA debate—the national testing debate—is that the more time we spend on it and, to be frank, the more time we spend in chambers like this having this kind of conversation, the less time we spend doing the things that could make a difference in addressing the impacts of poverty in schools. However, I argue that that, too, is an issue that we do not understand very well.

I want to pick up on Lindsay Law’s point about literacy and numeracy, and broad education.

I understand that you do not want to focus entirely on a very narrow set of assessments, but is it not reasonable to say that one of the things that schools need to do is give children the building blocks that enable them to access broader education? If pupils are disadvantaged in literacy and numeracy at an early stage, that feeds right through to the point at which they should be going to positive destinations—and some pupils’ destinations are not at all positive.

How do we get the balance right? I was a teacher in the late 1970s and early 1980s, when the attitude was perhaps, “You don’t need that rigour—children learn through reading,” or whatever. How do we address the point about confidence in reading and mathematical skills being a fundamental means of opening up access to education?

I do not think that Lindsay Law was suggesting that we should not focus on literacy and numeracy. Literacy and numeracy, and health and wellbeing, are at the core of what we are doing across the BGE, and the transformational work that has been done in those areas as a result of the development of curriculum for excellence is there for people to see in schools.

However, the focus of a national test on those two areas draws attention away from much of the other work that goes on in schools. I am sure that Lindsay Law will speak for herself if I am misrepresenting what she was saying. The tests will not ensure that there is more rigour in literacy and numeracy than there was before. The rigour exists, and the transformational work around literacy and numeracy, particularly at the early stages, and the use of play-based pedagogy to develop literacy and numeracy and to tackle the gaps in learning that young people have when they come to school, are there to be seen in our schools.

Members need only visit schools regularly; you will see all that. You will get information on the impact of the work, the assessments and the discussions about achieving a level. The national standardised assessments will give no more than that. That is what our teachers are telling us. In the first year of using the SNSAs, even people who said, “Aye, they’re okay” told us that the SNSA told them no more than they already knew about the young people in their class.

If SNSAs are telling teachers no more than they already knew, why waste time doing them? Why not use the time to teach the kids and get on with the rest of it? When a teacher is assessing, they are not teaching. It is said that the assessments take only half an hour, but that is not the case, because they have to be worked on and teachers have to take time away from addressing the teaching needs of the whole class—for example, to work with just a couple of people on a computer, because adequate infrastructure is not there.

If the assessment is not adding value to the system that is in place, there are all sorts of reasons why it is not worth doing. It is not that literacy and numeracy are not the focus; they absolutely are. However, I echo Lindsay Law’s point, which is that there is too much focus on narrow bits of what we do in the education system.

One way to establish whether interventions at the early stages are working is to look at the destinations of our young people. If we continue to improve in relation to positive destinations and positive experiences, that will show that we are making a difference all the way through the system.

There has to be rigour applied to what is in place, to ensure that no child is missed out. Literacy and numeracy are at the core of what we do, but there are in our education system many more things than literacy and numeracy scores at P1 and P4 that we could be looking at and promoting.

Literacy and numeracy are foundational—that has to be recognised. One of the many professors who has given evidence to the committee said that although the areas of assessment for SNSAs are narrow, they are quite important. That has to be acknowledged.

I do not think that there is a challenge to policy makers at national level or local level having a particular interest in literacy and numeracy—that interest is legitimate. The problem comes if that is all that they end up focusing on.

Take, for example, the experience from south of the border, where that has been a serious shortcoming in the education system. Even the Office for Standards in Education, which is the national inspectorate, now recognises that there has been a disproportionate emphasis on literacy and numeracy, which has been to the detriment of the rest of the curriculum. The inspection system in England is being recalibrated so that it is not just about assessing a narrow range of numeracy and literacy indicators, but about getting a broader balance.

In the context of national policy here, it is fine to focus on literacy and numeracy, but it needs to be understood that the SNSAs were supposed to be part of a broader assessment that was focused on CFE levels and an understanding of the critical role that was to be played by inspectors. Inspectors should be able to go into schools and form a judgment about whether holistic education is being provided.

The balance is difficult to achieve. To some extent, it is an iterative process. We are in the early days of the SNSAs and we have to work through the issues. The key trap to avoid is our ending up focusing in a punitively high-stakes way on a very narrow range of indicators, because that will impact on the breadth and the balance of the curriculum that children experience.

On the issue of our becoming far too focused on very narrow indicators, you would expect people to focus on literacy, numeracy and health and wellbeing as foundational aspects of education, but it is very clear that we are going in the wrong direction in Scotland, when compared with other countries, given our really extreme focus on a relatively small number of things to target in order to form our data points. That is happening for various reasons—Governments need good news stories, the Opposition needs things to batter it with in the press, and journalists want stories that can go out the next day. It is hugely frustrating. As somebody who writes in the Scottish media about education, it is massively frustrating to me how incredibly difficult it is to have the full discussion that we need about these issues.

I worry that Scotland is increasingly moving—as it certainly has over the past few years—in the wrong direction, and that our focus is becoming narrower. As I said, there are political and media reasons for that. Issues to do with the devolutionary structures of the Scottish Parliament feed in, too.

It would be something if the one good thing to come out of this work is recognition across Parliament and beyond it, across society, that we are making a mistake in becoming far too focused on narrow atomised aspects of the education system. That would, at least, take us away from a direction of travel that worries me, as someone who teaches in the education system, writes about it and has a four-year-old son. If a consequence of all the mistakes and confusion is that we have a discussion about stopping the approach, putting the brakes on and thinking through what direction we want to go in, that will be a positive outcome.

Are we in danger of overcomplicating this? Susan Quinn said that the SNSA is one of a “broad range of assessments” that are made. I am not sure why it is—

Such a big deal?

Yes, exactly.

I suppose that there are a few reasons for that. First, the principle seems to be that we trust teacher judgment to be the metric by which we will measure the education system, but—whether this is explicit or not—we really trust that judgment only if it is based on a set of standardised tests that the Government has decided it wants. I am afraid that that approach will get a reaction.

The other issue has been touched on in this session; it was also touched on several times in the informal session. There is an opportunity cost not just to the tests, but to the culture that they are likely to lead to. Ultimately, that almost certainly ends in standardised test data and versions of it becoming public. Indeed, we have heard about schools sending out SNSA reports to parents, for example.

Part of the reason why there is so much concern around the issue is because it represents a direction of travel in and of itself. In other countries that went down this road a few years before us, the tests themselves have had an impact well beyond what was intended. At this stage, that impact is relatively predictable. I direct you to Australia’s national assessment program: literacy and numeracy—NAPLAN—for example. Much of the comment that is coming from the teaching profession stems from that.

11:15  

There is a genuine concern. When there is a real focus on introducing a certain kind of testing that will tell us certain things, the concern is not just whether the tests give information that is positive or negative, but whether there is the potential for further damage down the line, once we start factoring in the opportunity cost of it all. That needs to be borne in mind.

The basic point is correct: it is a single assessment, and lots and lots of assessment is done already. However, that kind of system has a particular effect, as does the way in which it has been instituted.

Perhaps that is about the way in which it has been portrayed in the media.

Possibly—you could argue that. Ultimately, if a politician stands up and gives a speech asking to be judged on their record and trying to tie educational improvement to election cycles, that will be the starting point for how things show up in the media. A roomful of journalists took the First Minister at her word in that sense.

To be fair, I do not think that the First Minister was saying, “Judge me on the record of the SNSA.”

No—she asked to be judged on the record of educational improvement. However, given that the introduction of standardised tests was announced in the same speech, what was likely to happen?

As a result of many of the things that happened early on in the process, where we are now is where we were always going to be. That is unfortunate.

It is such a big deal because resources are limited and the resources that are placed on testing are not placed elsewhere in the system.

We have heard from parents—

I am sorry to stop you, but what do you mean by “resources”? Do you mean teachers’ time?

I am talking about money, teachers’ time and everything else. We are in a finite system. Introducing testing costs something that is not going elsewhere.

Parents are already having those conversations. Schools are already tracking their young people. My children are now at high school and the parent council has sat down with the headteacher and looked at the performance of each year group, the virtual comparators and the local authority comparators. That data exists at the school level. The tension is between its existence at school and local authority level and whether it can be gathered at national level. That tension should not be played out in children’s lives.

Primary 1 pupils have just arrived at school. We know what the data will tell you: it will tell you that children from a lower socioeconomic background will not be as advanced as children from a higher socioeconomic background. Standardised testing will not help teachers or parents get anything out that they do not already know.

At Connect, we encourage good-quality conversations between teachers, as the primary route by which the education system is delivered to children, and parents, who are the primary supporters of that—we should all be encouraging that. We know that the engagement of parents makes a massive difference to young people’s outcomes. The way to involve parents in school is not by sending them a report full of dense, complicated information that gives them a point-in-time snapshot of a small part of the curriculum but by having good-quality conversations with teachers about the young person in the context of the classroom and what support the parent can give. That requires investment, smaller class sizes and all the things that the background of austerity is preventing us from having. Parent councils are no longer about providing added value to a school; they are about providing basic infrastructure and meeting the school’s basic needs. That background means that teachers are no longer able to spend time working on a personal learning plan for a child; instead, they are more focused on ensuring that their school does not come at the bottom or middle of a league table that will be prepared from a narrow focus on the curriculum.

We need to think about the effect of all this on young people. How will it affect their classroom learning and what can we do to change that? I do not think that standardised testing will change things in the short term and I am not convinced that it will do so in the long term.

How do you advocate that we should assess children? Do you propose no tests or assessments?

We have already told you that assessments go on—

Can I respond?

My question was to Lindsay Law.

I just explained that my understanding and that of parents is that children are constantly assessed by teachers. They are assessed through local authorities’ standardised tests and they were assessed through the SSLN. Children are constantly assessed throughout their school career. It might be argued that, at high school, children are assessed too much, which takes away from teaching time.

There is a continued focus on numbers rather than children as individuals. That is driven by a national Government obsession with saying that Scotland is leading the world in X, Y or Z. Parents do not care about Scotland leading the world; parents care about their children’s education—what that means today, tomorrow and the day after that. If children are tested in primary 1 today, we will have to wait seven years for that to work through the system before anything meaningful comes out of it to help the next cohort. Our children do not have that time; they need resources in the classroom now to help them now.

As we have seen in the past, the introduction of national standardised tests makes us lazy in our conversations about educational attainment and achievement. When we had five-to-14 tests, all that we talked about was the percentages. I remember having conversations as a class teacher about how to get a class to 80 per cent when it was at 79 per cent. We discussed not the added value that would be provided for a young person but how we would get to a certain percentage point.

A test is easy, because it allows people to make graphs and all sorts of pretty pictures, but it is a lazy way to report on what happens in our system as a whole. Assessments happen every minute of every engagement that teachers have with young people. If a teacher sees that something is not working, they immediately look to the next step. A teacher does not require a formal test to decide that they will change how they work with a young person or group of young people.

Introducing something that is described as a standardised national assessment or test makes us as a country lazy about how we approach things, because it is easy for journalists to get the information and create league tables and easy for you in Parliament to say, “Look at that—the SSLN figure has dropped by 0.06 per cent, so the Government must be failing and everything’s wrong.”

Focusing on an individual thing makes us lazy. When a standardised national test is put in place, that is where the focus is directed, because that is easier than having the complicated and meaningful conversations, which have happened in every one of the 30 years that I have been teaching, between parents and teachers about what is going on with a young person. A test creates a negative narrative about the system that takes us away from the good work that is going on in our classes.

To go back to Rona Mackay’s point, there is no problem with having some form of national assessment—that is legitimate. The issue is the purposes to which the outcomes of that assessment are put. To give the Scottish Government credit, its submission to the committee makes it clear that it recognises that there are dangers in high-stakes assessment and in narrowing the curriculum—we have seen the damage that that has done elsewhere.

The trick is to ensure that, if we have a national system of assessment, we understand its limitations, what it tells us and what it does not tell us, and we act accordingly. We must give proportionate weight to what the assessment can tell us. Given some of the evidence that we have heard, the danger is that that message is not getting through and that people are attaching high-stakes purposes to the assessment.

If there is one recommendation that I would make to the Scottish Government, it is to double down on its commitment that the SNSAs are not high-stakes assessments and that it does not want to see teaching to the test. The commitment that has been given is really important, and it has to be put into practice.

To be fair to the Government, it has consistently said that the SNSAs are not high stakes.

It needs to continue to do that.

To stick with the point about confusion over purpose, I was interested in the written submission to the committee from the EIS on the union’s initial success in shifting the Government’s position in the discussions before the SNSAs were implemented. The Government started with the understanding that the purpose was summative, and the EIS shifted that.

Has the confusion come about because the Government shifted from having one relatively clear position to another relatively clear position, and in that shift the information was not cascaded properly, or is the Government still hedging its bets? Is the confusion because the Government is still sitting in the middle, or is it because the position changed and it was inevitable that during that change the information would not be transferred consistently to local authority and school level?

The reason for the confusion will depend on the individual who is confused. Some of the confusion is because, although advice on implementation was developed for local authorities by the advisory group, a significant number have chosen not to follow that advice. I cannot say why they would do that, but I surmise that there is a host of reasons.

I know from previous evidence to the committee that it is partly to do with the fact that local authorities had systems in place that they were happy with. Going back to my opening comments, at the beginning of the process, no stakeholder in the system told the then cabinet secretary that we needed to add any other form of assessment to the system; all we needed was to look at how we gathered assessment information to get a national picture. Lots of local authorities were not on board with standardised assessments so, whether they came in as originally designed or as actually happened, local authorities were never going to approach them as something that they needed.

Then there is the confusion about how information is relayed to parents and how information is shared. Throughout the negotiations and development of the standardised assessments, the policies and the advice note, mixed messages were still coming from Parliament in relation to their purpose.

The EIS national council meets five times a year. As its education convener, I am on my feet, talking about education, for an hour, on average, at each meeting. A substantial part of that time in the past two years has been taken up with members saying to me, “You told us that the assessments were going to be this,” or, “You said that they weren’t going to be that, but now they are.” That is because the messages about the assessments continue to be mixed. I do not know why that is—whether it is about one part of Government not talking to other parts of Government, or a desire for the message to be something that it was eventually negotiated not to be.

If the SNSAs had been implemented in the manner in which the advice note and the details were eventually developed, I do not believe that we would be in this position. Teachers and schools would be deciding if and when the assessments would support to them in the work that they were already doing. The school would decide that there would be would be a diagnostic benefit for each group of young people, but the school’s own evidence would already support the view that a group had achieved a certain level and did not require anything else to be put in place. However, because a little bit of the system said that some information should be gathered nationally to identify trends, that became a requirement that everyone had to do the assessments, and the question was then how that would be put in place.

11:30  

Ross Greer asked whether the problem is that we went from one relatively clear position to a different relatively clear position. I would probably take issue with the idea that we have reached a relatively clear position. We are still in a situation in which we have tests that are meant to be formative and summative, and individual, local and national, and the information is being dealt with in all sorts of different ways, so I do not think that even that is quite the explanation.

The idea has come up that, had the assessment been implemented in a certain way, or had X, Y or Z happened, things would not have been so bad. I suppose that that is the unintended consequences defence, with people saying, “We didn’t intend this testing system to become this kind of beast.” I have some sympathy for that defence, but my sympathy is limited. The defence is not as strong as it could be because many of the apparently unintended consequences were predicted at the start. We need to take that into account as well.

A lot of the issues have come from confusion and poor implementation, but a lot are what many people said from day 1 was going to happen. In so many ways, we are where we thought we would be. We are where the EIS and Connect said we would end up. The fact that we are here at the committee today and I am here instead of being in a college teaching is rather an indictment. The things that were said from the start, which were true, were not really listened to. That is why we have ended up in the situation of spending time at the committee today.

I am interested in why you think the Government’s position became more confused—or however you want to characterise it. If we assume that the tests were not the end goal in themselves, the Government must have come at the matter with another end goal in mind. A particular end goal would take us in the direction of summative testing and another would take us in the direction of more diagnostic and formative testing. The Government seems to have embarked on a path but changed its position midway through the process.

From the witnesses’ experience of engaging with the Government—or from James McEnaney’s experience of investigating after the fact—why do you think its position changed? The extent of the changes to the potential purpose and design of the tests fundamentally shifted the overall objective, but the Government must have started with an objective.

I would argue that one of the key drivers was that the EIS indicated that we would ballot our members to boycott any system of tests being put in that was as the system was described at the outset.

I would agree with that.

We were clear that we very quickly had our members behind us on the idea of a single window for every child in the country to be tested at a particular point in time with a single test. Even with trade union thresholds, we would have smashed it, particularly in our primary sector. We would argue that that was a significant driver for the negotiations that then took place.

The fact that we were unable to shift the Government from its position and the overarching data was still going to be gathered nationally to look at trends—I am still not really clear how that is going to work; it cannot work in years 1 or 2, anyway, because trends require more than a single year’s worth of data—means that we have assessments that are trying to meet the needs of multiple masters.

Do you believe that Government, of whichever political persuasion, genuinely needs to have some data about what is happening in schools?

Yes, absolutely, and it is there.

What would be the best way to achieve that?

The best way to achieve it is what we currently have in place, which is that the levels across each school are gathered—

We are told that that data is unreliable. Indeed, the Government published information saying—

That data will become no more reliable with the tests in it.

In fairness, Ms Quinn, I am not asking about that.

No. However, that data is becoming more reliable; there are ways and means. The conversations that we had at the start were, “How can we make sure that the data in the system can be more reliable than it is?”

In which year do you think that we will be able to compare ACEL data year-on-year, to understand what is happening in our schools?

I do not know. You will need to have a conversation about that with Education Scotland and the directors of education. I believe that our teachers are working hard in schools and they are doing incredible work to moderate—

I do not doubt that—

If you let me—

Wait a minute. I am trying to establish what data a Government of any political persuasion needs in order to understand what is happening. I am asking about the EIS’s view on when we will have that information—I am open to suggestions from the other panellists, too.

I believe that the data in the system is reliable.

That is absolutely not the evidence that the committee has had.

I cannot comment on where other people think that it is unreliable. At this time, an increasing number of systems has been put in place to make sure that the data from teachers’ professional judgments that is in the system is more and more reliable. Teachers are engaged in more moderation exercises and the benchmarks are now in place.

The benchmarks were not brought forward by Education Scotland until a couple of years ago, so teachers were working blind in certain aspects of the system. That was asked of them, and they continued to work to make sure that the best of evidence was there.

I would ask those groups why they believe that the evidence is not reliable, when it is based on the broadest of information. We have systems in place whereby teachers, headteachers, deputes, quality improvement officers and Education Scotland are working not just in individual schools but across multiple schools across local authorities. I believe that the system is now reliable.

Thank you. However, the Government is telling us that it is unreliable—that is its assessment. That is all that the committee can assess.

I will ask Mr McEnaney about the specific points in his submission about the SSLN. You have mentioned the potential to expand it. If it was expanded—you might want to elaborate on how that could work best—what could it tell policy makers, whether they be journalists, national policy makers or, indeed, the Government?

I am always nervous about what that kind of data could tell journalists—I say that as a journalist. The advantage of national data for Governments is that, because the SSLN is sample based, it does not go down the road of having issues around teaching to the tests. I do not know whether the couple of teachers who are here did the SSLN, but I did when I taught in secondary schools. Teachers could not teach to the SSLN.

The data that the SSLN gave was not just reliable in the sense of giving a national snapshot, but—I do not know whether committee members have looked at a report; if not, I suggest they look at the 2016 SSLN report—the level of data in it is remarkable in places. Look at the statistical tables for 2016, which show things such as 26 per cent of kids in primary 4 reporting that nobody ever read to them at home. That is the national data that you need, and the SSLN gave us a wealth of it. For the record, the SSLN is still technically available; the material for it is still sitting there.

The SSLN and a national sample is really valuable. I do not think that the case was ever made for getting rid of it; I think that it was got rid of because, if a system of national standardised testing is going to be instituted, it looks difficult to justify also doing a national sample model, especially if the case that is being made is, “We need a standardised test system because there is something wrong with the sample model.”

On the point about teacher judgment, I think I know what you are getting at. Whether or not individual teacher judgment of their pupils is reliable is a two-part question, the first of which is whether we can we trust teachers’ judgment with regard to the progress that their pupils are making. The answer is absolutely yes. I would trust a teacher every day of the week rather than trust a standardised test. My wee boy is four and he will go to primary school next year. I am not interested in seeing a standardised assessment report about how he did over 40 minutes one day, but I would love to sit with his teacher for an hour a couple of times a year.

The issue comes with whether teacher judgment can be used at a national level to give the same kind of information that the SSLN gave. In my view, it cannot, as they are two very different things. Given the point that witnesses made about trying to be clear what an assessment is for and what data is for, those are two different things.

That is not to say that I do not trust teacher-judgment data or that achievement-of-a-level data is not useful or accurate, but there are issues. We have heard that teachers are doing more and more moderation of what the levels look like, but we are still a long way from where we want to be with the time that is available to do it. I remain unconvinced that the standards—for want of a better term—have yet been properly exemplified. That comes up quite a lot in discussions on aspects of coming to judgments across more than one area.

In your submission, you also make the point that there is

“no properly agreed standard for what the ‘achievement’ of a level looks like”.

What do you mean by that?

If you look at the SSLN, a clear statement was made of what it looked like when someone was performing very well or when someone was struggling, and data was given around that. There is still no clear-cut statement for what a level 3 writing assessment looks like—that has never been achieved.

However, I must add—I was perhaps not clear about this in my submission—that part of the reason for that is that that is not really what that information is for. It is not necessarily helpful to try to view level 3 writing as, “There is a thing and that is what it is going to be.”

We are still a long way from a situation in which teachers have the time, space, professional autonomy and trust from Government, Parliament and—I happily add—journalists to do a proper moderation job. To return to Tavish Scott’s original point about the Government saying that it could not trust teachers’ judgment—as much as I and teachers trust it—that is probably the way to get to it.

Mr Northcott, do you have a view about the SSLN? Are you comfortable with the suggestion that has been made—not just today, but at previous committee meetings—that it should be altered or enhanced in some way?

All that I can do is make the obvious point that, given the advice that the Scottish Government was given by the Organisation for Economic Co-operation and Development, it was probably going to be difficult for it to continue with the SSLN in its current form. The question is what it should be replaced with. I do not share the OECD’s analysis, as I think that there were important strengths in the previous arrangement.

The SNSAs might be able to fulfil that function to some extent, but I return to the point that we made at the outset, which is that we have to be clear about what they are for. At the moment, we are not clear about that. They are described as formative assessments and then they are described as summative assessments. There is a legitimate policy debate to be had about which we want, but we cannot get one assessment to fulfil both tasks.

In a sense, it is important that national local-level policy makers have good information and data about what is happening in the education system, but they have to be able to interpret the data and information in context. The danger is that the data becomes everything; it becomes the only lens through which the education system is looked at. That is dangerous—experience from elsewhere in the UK and other parts of the world has underlined that many times.

As the dad of a nine-year-old who goes to school in Scotland, I get more information if I ask a lot of direct questions than I ever do by just reading the report, as was pointed out earlier. I have also looked at my school reports because my mum kept all of them—God help me—from all those years ago. We told parents of my parents’ generation more than we tell parents today. What needs to change about parental information from whatever system is ultimately used in schools?

Parents probably need more information earlier about the school system in Scotland, because it has fundamentally changed since we were at school. Parents arrive at school and are assaulted with lots of new terms and words, and not every parent is equipped to understand them. I loved school, but not every parent did. For a lot of parents, going back through the doors of a school reminds them of an unpleasant experience. Those reports were clear: they were clear about whether a pupil was succeeding or not succeeding, which is a stark thing to tell a nine-year-old.

11:45  

The great thing about reports today is that they tell parents loads of things about what their children are doing in class, how they are developing against certain levels, whether they are secure, and whether they are consolidating. It is not a pass-fail situation. However, even words such as “secure” and “consolidating” can be confusing for parents. We need an early years, play-based curriculum that not only introduces children to school, so that they learn how to be at school, but reintroduces parents to school.

We need to reconnect schools with communities. We should not view a school as somewhere children go to be given something by a teacher. We should see them in the context of their local community. We should invite industry into schools and we should invite young people to come out of school and visit industry and the local community. Only by doing that and sharing with industry the meaning of the terms in curriculum for excellence will we start to bridge the gap between SQA results and what those results actually mean for a young person when they go into the adult world of work.

Thank you.

Tavish Scott asked about the SSLN, and James McEnaney talked about teaching pupils who were involved in it. When I was a modern studies teacher, I had pupils removed from my class. To be quite honest, the data meant nothing to me—obviously it did not come to me—and I felt pretty disempowered by the whole process.

Susan Quinn mentioned the challenges around the SSLN. You said in your submission:

“the EIS favours the proportionate gathering of data to provide appropriate system-wide information to inform policy making”.

The SSLN sat at that level; it informed policy making.

Is there not an opportunity to use the SNSAs to empower our classroom teachers to use data more effectively and to track pupil progress? We heard in previous evidence sessions that the assessment generates information that can more readily be used, at pupil and classroom level, to track progress right the way through a child’s educational journey.

No—not in the manner in which they are designed or in which they are being used. As I said, our members tell us that they are not learning anything significantly new. They are not learning anything that they do not learn from the other assessment work that they do with young people.

The tracking of progress goes on in schools. One of the challenges is that we have a multitude of tracking systems in schools across the country. The tracking of progress against the benchmarks, using the Es and Os and the curriculum as a whole, goes on daily and has trigger points around conversations in schools, so that teachers can use the information to improve on what they are doing.

The SNSAs are not providing the national picture in the way that the SSLN did, so they are doing neither of the things that we want the system to do.

We have a system in which we have a breadth of assessment strategies in schools, which are working, and we need to find some way to collect information nationally, to give the national picture. I think that the data on achieving a level provides that, albeit that there are challenges to do with time for moderation and understanding the standards—and schools are getting better at that as time goes on.

The SSLN provided something nationally that the SNSAs will not provide. I understand your point: I was the headteacher who removed young people to administer the survey, and I was the class teacher who had pupils removed from my class. Just because something did not happen does not mean that it could not happen. There were ways and means of using the information from the SSLN and, as James McEnaney said, there were potentially ways of developing it to get a genuine picture of Scottish education at a single point in time, as well as some of that really rich information about young people and the targeted interventions that we could make.

I want to pick up on Tavish Scott’s question about the reliability of the data prior to SNSA. I noted that the EIS said that it was reliable and efficient. We know that 28 out of the 32 local authorities used some form of standardised assessment prior to the SNSA. However, that was happening in very different ways, and from the evidence that we have taken we are not assured that all those assessments were being benchmarked against CFE. That is problematic if we want to look at the reliability of the data. If we are not benchmarking against the national curriculum, what is the purpose of testing the kids?

If there is greater standardisation at a local level, is there not an opportunity to level the playing field? I think that it was Professor Sue Ellis who spoke about unethical assessment approaches at a previous evidence session. She said that taking groups of kids out of class can be quite unfair. Is not a standardised approach fairer?

We do not have a standardised approach. The SNSAs have not been introduced in a standardised manner. The agreement with and advice from the Scottish Government was that they would not be introduced in a standardised manner, because the local authority—

What do you mean by that?

Because schools and teachers should decide when the young people engage with the standardised assessment. They are not being used in a standardised way; they are being used differently, just as the current raft of standardised assessments are being used in a variety of ways and—may I add?—have not been stopped in many local authorities despite the introduction of SNSAs, even though one of the pieces of guidance that came forward said that as soon as SNSAs were introduced we were to stop doing everything else—

But all local authorities should be doing the same thing, under the SNSA, and that is not what happened previously. The system is standardised. Teachers can decide at what point in the year they want to carry out the assessment. That empowers teachers, surely. However, at least we know that what is happening in schools is, to some extent, standardised.

My concern is that if 28 local authorities were previously doing many different things, a pretty unfair playing field was being created—it was not fair to the children. Surely it is in children’s best interests that all children have the same opportunity, and that is what this is about.

Assessment is not an opportunity for a young person. It should be about informing learning and teaching. Different standardised assessments were used in different ways and for different purposes. Some local authorities said that they used them, but not necessarily across the local authority; individual schools determined what they used and when, to inform learning and teaching for the young people in their care.

The new system will not change that. The reliability of the information on achieving a level is based on teachers’ professional judgment. SNSAs will not change that—or they should not change that, unless the system is skewed to make teachers’ professional judgment simply about the SNSA scores on the doors, which takes us all the way back to the point about SNSAs being a high-stakes test.

The fact is that the way to get a standardised, equitable approach to assessing learning and teaching across the country is by looking at moderation practices and how they are inspected, to ensure that everyone is working to the same standards. The SNSA will not fix that. It will not do that. It deals with 10 per cent of the curriculum at a really narrow point in time—

But any assessment is about a narrow point in time—

But that is the whole point—

That is the nature of assessment. We are looking at a snapshot in time—

But we are not looking at a—

It is not that the assessment data is the only thing that a teacher looks at. Teachers look at a broad range of things that happen in their classrooms.

The whole point is that we are not looking at a snapshot. SNSAs are supposed to inform teachers’ professional judgment when they have conversations; they are not about confirming teachers’ professional judgment. That is the agreed language on this. SNSAs inform only a tiny bit of the curriculum at a point in time, and when that time will be in the year is not agreed.

We will need to agree to disagree. I do not believe that SNSAs will add to the system that we have in place, or deal with the issues about whether the evidence that is in the system on achieving a level is more valid for that assessment than it is for any other assessment.

SNSAs could be used for conversations, but there are no guarantees that they will make those conversations any better. The only way to do that and to understand whether those young people who are deemed to have achieved a level have genuinely done so is if we train our teachers more effectively in moderation, and that we make sure that senior managers and line managers who are having those conversations are consistent, so that the approach on the moderation of achieving a level is consistent.

As James McEnaney said, the standards have to be understood. The introduction of the test has not done that; it will not do that. It draws the focus away from that moderation exercise.

I do not want to lose that really important point. An important part of the story is that, before the SNSAs, 28 local authorities imposed standardised assessments on their schools. One element to consider is that some—I will not name anyone—of those standardised assessments were incredibly narrow.

There is an interesting contrast between the previous assessment and the SNSA, which has some technically attractive features—it is an adaptive assessment, for example. The SNSA has the potential to be a better assessment than those that it is replacing in 28 local authorities. That takes us back to the point that, if you have a reasonably good standardised assessment, the problem is not the assessment but what you do with the data that it generates. If that is used for high-stakes purposes, you undermine the formative value that that assessment would have.

An important part of the narrative is that one national approach replaces 28 different approaches to standardised assessments. There is no question but that that creates challenges, but we should not pretend that there were no imposed standardised assessments in schools before, because there was an awful lot of that.

On the standardised nature of the assessment, teachers are supposed to administer the assessment when they feel that learners are ready. Teachers and parents are telling us that, in practice, that is not happening. It happens in some local authorities, where the tests are administered in the same way that their own standardised tests used to be administered in a set window—for example, in the summer term—so that the information is available for the transition year from P7 to secondary 1. The assessment is not being administered in a standard way across Scotland; local authorities differ in their approach.

If you want to standardise something, standardise the standards, not how you measure one tiny bit of the system. I understand why that approach is attractive at Government level; I understand its attraction, as somebody who writes about Scottish education in the media and is eternally frustrated about never being able to write the 10,000-word piece that he wants to write. Ultimately, it is not only the wrong road to go down, but—it always returns to the same point about the opportunity cost—we spend so much time obsessively focusing on whether we can standardise provision and use a test to make sure that everyone is getting the same bit of data that we end up slipping further and further away from the teaching and learning system that we have in place, which gives every kid in Scotland the best possible chance in life.

12:00  

I want to focus on the practical elements of the testing. It appears to be standardised, but it strikes me that it is not even standardised in purpose. I wonder whether, given his research findings, James McEnaney can confirm where all of this came from. Am I right in saying that, basically, it started with the decision to have testing, then it was rationalised post hoc? Is there any genuine evidence that somebody sat down, said, “This is an issue” and took themselves to testing from there?

Susan Quinn’s evidence says that, in the various meetings, nobody said that they wanted a new set of tests, and that aligns with the information that I found. It is difficult to be sure, though, because all those meetings were unminuted.

That said, it is quite easy to see that, in the early stages, it was at least partly driven by a political decision. If we look at the available information and material, we can see the scarcity of written advice that led to the implementation of the system. For example, there were four emails. Given that, we can understand why people come to the conclusion that you have come to, which is that a decision was made to start with testing, and everything proceeded from that. I was not in the room and I do not know, but there is not a lot of evidence that leads us to an alternative conclusion.

Do you think that some of the confusion has come from a reluctance to pick sides in the argument? We can argue that there is a benefit from national testing—that it is rigorous—and that everybody has to understand that or we can say that it is diagnostic and is about the individual child. We can pick which view to emphasise, depending on our audience. We have certainly been asked in our own debates why we would not want a system that would identify what a child’s developmental challenges were or whether a diagnosis had been missed. What is your view on that?

It was mentioned earlier that there was an idea that the tests would be used to figure out whether children had, say, dyslexia, and I have heard people talk about the tests being used to help identify autistic children in school, kids with dyslexia and suchlike. Such an attitude is incredibly dangerous and irresponsible, because the tests will absolutely not do that, and any thinking that that will happen or that the tests are a diagnostic for additional support needs must be dealt with right now. We have a bad enough situation in Scotland with the treatment of additional support needs over the past few years, and we certainly do not need to make things any worse with that attitude.

Again, though, this is not intentional; it is not as though a minister had come forward and said that the tests would lead to kids with dyslexia being diagnosed earlier or anything like that. However, it is a line of thinking that has continued, and even omitting to say that it is wrong is irresponsible.

I cannot be certain about this, but my sense is that the argument being made to those with concerns about the testing is that surely they do not want a situation where the needs of a child with autism are not being identified. That has actually been said.

I would regard that as being very irresponsible.

I have two further questions, the first of which is about an issue that I have raised in previous evidence sessions and which I would like your view on. I was advised by those giving presentations to the committee on how the tests worked that they could be done at any time in the year, which for primary 1 children would be any time between the ages of four and a half and six. I was also advised that individual children could be rehearsed and that the information that the tests provided could make no distinction between a child who had to hear a word in order to identify a rhyme and a child who could read the word. To what extent does that range of possibilities make the word “standardised” nonsensical?

A couple of years ago, I attended an event featuring a Government statistician, at which I mentioned what you have just said and talked about effectiveness. I cannot remember the exact words that were used, but I can provide them to the committee if necessary, because I have a video recording of the event somewhere. However, I made the point about data from tests done at different points, and it was said on more than one occasion that such data was “not comparable”.

Of course, it is an issue only if we are going to try to use the data at national level. At that time, that was part of the conversation. If it is strictly formative information about each child, it is less important for the data between one point in the year and later to be directly comparable, because they are not being used to make a comparison between two points. However, the fact that all those issues are still unresolved speaks to the confusion in the entire system.

On the point about the age range from four and a half to six, primary 1 testing was overwhelmingly the major concern for parents and teachers in the feedback that we received from them. That is because P1s should be engaging in a play-based curriculum. They are learning how to be at school and how to be human, and they do not really know how to take a test. Not only is such an approach hugely variable, but the difference between doing the test at the start of the year and the end of it is a fifth of the children’s lives.

We also got feedback on resource time, by which I mean teachers’ time. It is a hugely costly approach. If there are three P1 classes, they will have three teachers, and there will be another three teachers taking children out one at a time alongside two support for learning assistants and another assistant. We are talking about a huge amount of resources and full-time equivalent teaching time being used for tests that are simply for gathering a baseline and, indeed, which are not comparable, given that teachers can pick when children between the ages of four and a half and six do them.

On the question whether the standardisation part is useful or otherwise, James McEnaney is right. It really matters only if information is being gathered for a national system or if people are looking at using it in local authorities in the way that they use their current information, which means separating out a standardised test from the overall picture of achieving a level and using that.

We have gone round and round the block with this conversation. If a single standardised test is used to determine interventions or—dare I say it—league tables, it is a narrow approach that puts us on the road to ruin. On the other hand, if the assessments are used as part of the broader bank of assessments that a school chooses to use, it does not matter whether they are used in August, September, October or whenever in the school year. People will use them to inform their decisions, and the evidence on the moderation and understanding of the standards will sit behind that.

There is a potential difficulty if we are looking at doing more than gathering information on the trends across year groups, which I understand is the national use, but we would argue that that is not what we should be looking for in the system anyway. We should be looking for something that gives us improved opportunities for young people in our classrooms.

Okay. So the issue of purpose is absolutely fundamental.

Absolutely.

And you would not use that kind of test if you were using it for national comparators.

If it was not being used for national comparators, a school would choose what it wanted to use it for. I was a primary school headteacher, and I know that my local authority has indicated that it is up to the school to determine when it will use the test in the year. I say “when it will use the test”, but actually I would always say “if and when”, just to make it clear that the test could genuinely be used as a diagnostic tool. You would be looking at which individuals you were absolutely confident were achieving the level from the evidence that you already had, which means that you would not have to waste their time or your own time with an additional assessment tool. It would genuinely be used as part of the assessment bank to inform teachers with regard to what they were doing.

Indeed, I would argue that, in some cases, it would not need to be used at all, as the body of evidence would show that the person was achieving the level. James McEnaney asked what a level 3 piece of writing looks like; I can tell him that, at level 2, a primary 7 pupil’s writing jotter will be very thick and contain evidence of all sorts of things that show that they are achieving all the bits, and that will not necessarily need to be backed up or supported by the need for additional assessment.

Do you agree that a test that cannot make a distinction between a child who has been able to read a word and a child who has had to press a button and listen to it is not a diagnostic tool, because it does not give you the information that you will be looking for?

That came up earlier this week. I think that it is problematic in relation to how teachers use the information that is generated.

Going back to previous comments, I note that we need to be clear about what we mean by “a diagnostic test”. People might have got on board with the SNSAs because they understood them to be diagnostic, but they might have misunderstood what a single diagnostic test might do. Some would understand that it was about diagnostics within the parameters of the questions that were there, while others would certainly feel that it was somehow going to diagnose other aspects of ASN. It cannot possibly do that. A wide range of diagnostic assessments are used in schools and by partners in the health service and elsewhere to ensure that young people with additional support needs have their needs met.

My final question is on the opportunity cost, which Lindsay Law referred to. A reasonable test would be a cost benefit analysis. Am I right in saying that you believe that there is little benefit but that the costs are significant? Have you been able to quantify that anecdotally?

Folk I know who are still teaching tell me that, in some cases, primary schools have spent 50 hours just delivering the assessments, and we have some evidence that additional support for learning teachers have been taken to deliver them. I know that the EIS did a survey but, more broadly, is there substantial evidence of what that looks like? If there is little benefit but significant cost, it becomes more of an issue than it would be if the assessments were not doing much good but not doing much harm either.

Some of this is difficult, because it relies on infrastructure in schools. Our members tell us that, where a school is already in an advanced stage of information and communication technology redevelopment and, for example, has iPads, the assessments take less time than a school that still has the same two old computers in the back corner of the room or where kids have to be extracted.

We know that a significant number of schools directed their whole senior management teams to administer the tests for two or three weeks. We had reports of teachers giving up their non-contact time—technically, their employers were therefore in breach of contract—so that they could deliver the assessments in the window that had been set by the local authority, because of the structures within the school.

More worrying, a good number of people suggested that support workers who were being paid for out of PEF money were being redirected to administer the assessments or to support the teachers in some way in administering them. In such cases, the money for intervening in the poverty-related gap will not have been engaged in respect of the relevant activities or interventions for that period of time. The only way in which the assessments could be delivered in the windows that were set was for staff to be redirected that way.

I am conscious of the time and I still have one committee member who has not been able to come in yet, so I must ask you to be a little more concise in your answers, if that is possible. Mr Northcott, you wanted to come in on that last point.

Yes. The points that have been made about value for money and cost benefit are really important. We know that there are costs associated with SNSAs. It is difficult to say what the cost of the alternatives would be, but it is important that we try to bear them in mind. For example, if the alternative to having SNSAs was to go back to the system that we had before, we should note that one of the costs of that system was the—quite substantial—cost of each of the 28 local authorities purchasing tests from different test providers.

If you were to replace SNSAs with some form of moderated teacher assessment, to which there would be some point, that could be workload intensive, too, and it could detract from other parts of the system. If we want to think about the value-for-money element of SNSAs or whether they are an effective use of resource, we have to think about what the costs of the alternatives would be.

12:15  

I may be very old but, in the schools that I worked in, one person ran the standardised sample testing, so you could do 30 kids at a time. Also, there was no work with iPads and that type of thing, which has added problems.

Optical mark readers and all the rest of it could be used, so the tests could be straightforward in that sense; however, they are not cheap, and local authorities had to buy them and pay a full commercial rate for them. If that were to be the alternative to SNSAs, it would have a financial cost. Local authorities would have to buy standardised tests and impose them on schools, so that would have to be weighed in the balance.

I do not have the figures to hand, but I remember that, initially, one of the big defences of introducing the national system was that councils were spending money on standardised tests, so it would save money. However, the cost of the national testing system increased a couple of times—I remember going back and forth with a press officer about that. I am sure that CommonSpace has reported on that, so it should be easy enough to check. I am not sure that the amount that the Scottish Government is spending is lower than the amount that councils were spending. It might be, but there is something in the back of my mind that tells me that that is worth checking.

It is a separate budget, though. The previous cost came out of local authority budgets.

Yes, but if we are looking at the straight opportunity cost, we would have to find out what the previous approach cost. In the way that the national approach has been framed, there has sometimes been a direct assumption in that respect, including in the press. If we are to look fully at the cost benefit issue, it is worth looking at that initial claim that the national system would cost less. I accept that there are two different budgets, but the Government made that claim. It is worth checking whether the amount that the Scottish Government is spending on the assessments is less than the amount that councils were spending, just so that we are clear about that. That is just ringing a wee bell in my head.

A related point is that the issue is not how much the scripts cost but what it means in staffing terms.

Yes. The opportunity cost is much broader than just the money for individual tests. It is just that, when that point was made, something in my head said that somebody—possibly me—should go and check that so that we have clear data on it.

Mr Mundell has been waiting very patiently to ask a question.

On that same point, correct me if I am wrong, but I think that not all local authorities have stopped purchasing other tests.

That is correct.

If the new assessments are so good, why have councils continued to spend money on something else?

That is a fair question, but it is for the councils to answer. I would surmise that they would argue that, in the first year, they were not confident of the new assessments and did not want to give up the years of data that they had. Some councils use standardised assessments in every single year of a young person’s time at primary school, and so they have not given up the in-between years. Others have—dare I say it?—introduced assessments beyond that in relation to PEF; they have started to use assessments that they never used before, which are paid for out of PEF money, to give them a benchmark for the start of the PEF process.

There are a whole lot of reasons why individual local authorities have chosen to continue with such tests, but it is absolutely clear that, certainly in the first year, very few of them have removed all the other standardised assessments while doing the SNSAs. We hear that a good number of local authorities are getting ready to do that as time goes on, but time will tell.

In a period of transition, we would expect a degree of conservatism about departing from a well-established system, whatever its shortcomings. However, in the longer term, if we persist with SNSAs, those local authorities will need to be challenged because, as has been said, one disadvantage of those systems is that they are not only relatively expensive but not at all aligned with curriculum for excellence. That brings into question the value of local authorities spending public money on standardised tests that bear no relation to the curriculum that schools should be pursuing. Perhaps we can cut them a bit of slack early on and say that they have needed a period of support or transition to SNSAs, but in the longer term it will be difficult to justify.

The other point is that, because local authorities and schools have to be accountable for PEF money, it encourages those to whom they are accountable to think carefully about the kind of indicators they want to use to make those judgments, particularly any indicators that are not linked to curriculum for excellence.

My other question is about transparency. I probably know what you will say, but surely it is unhelpful not to be transparent and allow a broader conversation about the evidence base in the early stages of developing policy, and instead to present policies and leave it to journalists to put in FOI requests for the information.

You will be shocked to hear me say yes—I think that transparency would be a good thing, but not just in the simple sense that, obviously, more information is likely to be better and to mean that journalists spend less time trying to chase down any bit of useful information.

The lack of transparency on the development of the standardised testing system fed into the way in which it had to be defended as the process went on. Part of the reason why it has been difficult to be clear about what the assessments are doing and what they are for is that all of that has become bound together. I would argue that it originates in actions such as spending a year taking me all the way to the Scottish Information Commissioner to try to prevent me from releasing the fact that four emails formed the entirety of the written advice that the Government received. I would always argue that, if the Government wants to make a case that it is introducing—I would say imposing—a new policy that should be beneficial to the education system and that, crucially, will help teachers, those are big claims and the Government needs to be clear about the evidence that it has to support them and the process that it went through to get to that stage.

There needs to be more transparency. That is a general point about policy formation, and this policy is part of that. In our experience of consultations, what usually happens is that an idea is created and then stakeholders from Scottish society are brought in as part of the consultation process. However, because the idea already exists, that sets up a naturally combative response between the people who have had the idea and those who say, “Have you thought about this and that?” We do not give much time to those people and it becomes an exercise in someone defending an idea, others knocking it down and the idea still going through, leaving a load of stakeholders feeling disempowered and disenfranchised.

In general, Government should involve stakeholders much earlier in the decision making and policy formation process, and should look to understand the root cause before it starts to develop policy. It strikes me that this policy was developed to try to understand something, whereas it probably should have been done in reverse. We should have asked what we are trying to solve in the education system and how best to solve it, rather than look straight to the measurement of something when we did not yet know what we wanted to measure.

I absolutely agree that there is a need for transparency. Even worse, in this case, is the fact that we had a meeting where stakeholders sat round a table—we are talking about a table as big as this committee table with all the seats filled—with the then education secretary and discussed what was needed for an understanding of educational standards across the country. It was generally agreed that there was a wealth of information in the system and that we needed something that would allow us to talk to each other so that there was a national understanding. We discussed whether there was a need for a national standardised test and the general viewpoint was that there was no need, and that people were comfortable and happy with what was in place, even though it was different in different places. We discussed how we needed to find a way to gather that information together.

Two weeks later, however, we attended an event at which the First Minister announced that she would introduce standardised tests. I was at that meeting—I had been at the one before—and I sat there, wondering when, between the previous meeting and that one, any of the people round the table had been spoken to again about what was decided and what was to become policy. From my point of view, that puts you on the footing of, “Oh, right—so you think so?”

We then had to have negotiations about what the tests would look like and everything else. The manner in which the tests were introduced was a real problem. An engagement exercise appeared to take place on the national improvement framework, the development of the database for that and how we would report on national standards. There was a discussion on what that might look like, but then something else came from left-field that had nothing to do with what the stakeholders and everybody else who was at that meeting had been discussing.

Let us take as a starting point the fact that there was a meeting and that the people who were at it all said that there was no need for the tests. Two weeks later, we ended up watching the First Minister announce that the tests were happening anyway. In that space, I came along and said that there should be material to look at to figure out what happened. However, it turned out that there was no material. There was a series of meetings—I cannot remember the exact number but it was in the teens—but the only material was agendas for three of them. There was no written material. Do not get me wrong: something may well be out there somewhere but, according to the FOI response that I received, the material does not exist. In that situation, what conclusion do you expect people to draw?

Lindsay Law made an entirely fair point when she asked how the Government expects stakeholder organisations to feel at the end of that process. The Government made a point of inviting people to talk to it, had the conversations, got an answer that it seemed not to like and then, two weeks later, appeared to have changed its mind. However, there is nothing to show where that change came from. There are significant issues with transparency in policy making in Scottish politics.

I also want to follow up on two points from the committee’s informal meeting this morning. On training for teachers, was there enough training in advance of the SNSAs coming in to help people to understand the data that they were producing? Was the training accessible to classroom teachers?

The difficulty was the timing of the training. Generally, if the teacher training is, as we call it, a huv-tae—something that everyone or some individuals have to do within a school—it needs to be part of a working-time agreement or in-service days. The times at which the training was made available by SCHOLAR and the Australian Council for Educational Research meant that it could not be put into working-time agreements for the first year of the assessments, which meant that renegotiations had to take place around how people could get out for training and beyond. There were therefore challenges in certain local authorities and local areas with people being released. Given the shortage of supply teachers in parts of the country, it was difficult for people to get to the training.

On the quality of the training, as our submission states, the message is mixed. The part about how to actually do the test was fairly straightforward. However, there was then a gap in time before the data literacy training took place. In some cases, a senior manager would attend the training and then cascade the information, which in itself can lead to a dilution of understanding. We would be in a much better position if, before the session in which the tests were to be implemented, there had been a period in which the assessments were developed and prepared and people were trained in their implementation.

In some local authorities, the window for training was left until the very end of the summer term. Teachers could not receive the training because the training group could deliver only a certain quantity of training at a time. Again, we would say that, if a policy is being introduced, it needs to have resource behind it. Instead of training people the day before they are about to use a system, we should do so in good time so that, before introducing the system, people can ask questions, digest the system and become familiar with it.

12:30  

In the informal session this morning, it was suggested that up to 25 local authorities are mandating a window in which the tests can take place. Why would they do that if it is for teachers to consider when individuals are ready to sit the tests?

That is because local authorities want to continue with their existing model of practice. They want to continue to use the test data in isolation from other assessment practices in order to do whatever it is that they do to inform their local politicians, in nice graphs and so on. It is a way of getting around the fact that this is not a standardised test in the way that people understand a standardised test. Local authorities will have all sorts of reasons to do it in that way.

Does that increase the stakes?

Yes, absolutely.

Do they become medium or high stakes if local authorities do that?

The stakes increase, because the results can be the subject of FOI requests or gathered by journalists in order to create school league tables and the like, which was never the intention. To be fair to the Scottish Government, when we were in discussions about the tests, it was clear that it was trying to avoid league tables by not gathering the national assessment data at national level and instead gathering only the high-level stuff about the trends. Having windows of opportunity for testing at local authority level means that there will be local league tables, which leads to the opinion that we have raised from the outset that we should focus on the bigger picture and not just this tiny wee test.

Something just popped into my head about local authorities giving windows for testing. At the event that I mentioned earlier, which I am sure was at Hampden, one of the Government officials talked about different aspects of the testing. I have the transcript of the discussion and it seemed clear that, even at that stage, the statisticians were talking about an expectation that the tests would ultimately end up being done in relatively set windows. I do not know whether that was ever set out as a policy intention. I will check the transcript for the committee to see whether that is accurate, but I am pretty confident that it is.

My understanding is that a number of local authorities use standardised testing at the end of P7 to form part of the information that a cluster primary school sends to its local high school. If local authorities are trying to reduce their spend on standardised testing, it would be logical to assume that they would prefer that P7 test to gather data for high schools to take place in a standard window, and that high schools would prefer that, too. However, that is my assumption.

If I were a teacher in a local authority that had imposed a narrow assessment window on my school, I would be sceptical about claims that the purpose of the assessment was to help me to make professional judgments about the children I was teaching. That does not help to provide the clarity that is lacking about what the assessments are supposed to be for.

I thank everyone who has given evidence. It has been a very long session.

We have an early start in the chamber this afternoon. I would not normally do this, but I ask the witnesses to leave the room quickly, as that will let us go straight into our private session.

12:34 Meeting continued in private until 12:38.