Education, Culture and Sport Committee, 26 Feb 2002

Meeting date: Tuesday, February 26, 2002

Official Report 289KB pdf

---

Scottish Qualifications Authority

I welcome back members of the committee and the witnesses from the SQA—I thank them for persevering. We are now into the second act of what will be, I hope, a play of only two acts this afternoon. I do not know what a third act might reveal.

We move on to discuss the progress that has been made on this year's examination diet. Karen Gillon has left to attend a meeting of the conveners liaison group. She will return, but may not be here for the conclusion of our discussion with the representatives of the SQA, which I will chair.

I invite John Ward to make an introductory statement, should he wish to add to the SQA's written submission. Thereafter, I will open up the discussion to allow members of the committee to ask questions.

I will ask Anton Colella to comment, but first I will put our discussion into context. Last year, we were very much involved in putting in a top-down control system and building better links with customers. This year, we are involved in building the confidence and capability of the organisation. It is vital that the SQA has confidence in itself.

I go back to some of the points that were made during our earlier discussion. As we go through the year, we will be creating a vision of and a strategy for where the organisation is going, as well as thinking about the investment that goes with that. This year is about building on last year's work and holding on to the gains that we made. As members will see from the figures that we have provided, we are running ahead of where we were last year by a substantial margin, although that does not mean that there are no problems—there are always problems.

Members of the committee will be aware that we certificated successfully in August 2001 and that we completed the appeals process successfully. On completion of the appeals process, we immediately began the formal transfer of information and the recruitment of markers. Let me update the committee. We have already recruited more than 90 per cent of our markers for this year. That is a significant improvement on the position this time last year, when we were still engaged in the recruitment process. We engaged in the process several months earlier this year and we are pleased that so many markers are already in place.

We are now matching the number of recruited markers to the uptake of subjects, identifying shortfalls, engaging reserves and, where necessary, moving staff around within levels and subjects. We have identified subjects for which the recruitment of markers proved to be problematic last year. Those subjects are problematic again this year, but we are much more prepared, in contingency terms. The SQA would like to reassure the committee that we are giving the matter due attention and that we are well ahead of schedule.

The transfer of data was a key consideration last year and we are pleased to say that, before Christmas, the bulk of entry data were in the SQA system. At that time, we were able to send out reports to centres, asking for confirmation that the data were accurate. We are now engaged in the process of centres withdrawing candidates and changing levels. Members will remember our discussions last year about ensuring that our data were accurate. I can reassure members that we are going through the process much earlier this year, to ensure that when we come to results entry and certification, the data are more robust.

A key point in last year's success was the fact that we worked to a plan. We are doing the same this year—we are holding the gains that we made through a rigorous and disciplined planning process that is subjected to clear and regular internal and external monitoring. The process is maintained through what is now called the certification planning group. We meet weekly, monitor our action plan for certification and beyond and ensure that any slippage that occurs is identified and that contingencies are in place. We have also engaged our auditors, PricewaterhouseCoopers, to conduct external monitoring, to reassure the board. PwC regularly monitors key checkpoints on the route to certification. We have a list of areas that are spot-checked through PwC interrogation. PwC then reports back to the certification planning group and to the board.

I have met every member of staff in the SQA and visited every department and I have been impressed by the air of confidence that exists in the organisation, particularly given the fact that individuals and sections faced a difficult process last year. I have heard about how they managed to cope in 2001, but this year there is a much more confident air. People say that what is happening this year is different from what happened last year.

As Anton Colella said, we are benefiting from PwC's external monitoring. I have not seen the process managed or monitored more effectively elsewhere in the public sector. It is clear that everyone, from board members down, is involved in the process. I re-emphasise my earlier point that we need to implement a different kind of process in the long term, to ensure that the pressures on staff are removed and that the process is appropriate for the 21st century.

One of the most appropriate steps to take in governance is to establish a strong audit and performance committee. We established such a committee last year and have been able to increase its strength this year. The committee comprises four of the new SQA board members. The tracking group to which Anton Colella referred reports directly to that committee, and the tracking is then validated by PwC. I repeat David Fraser's comment that the governance system that is in place is as rigorous as that of any organisation, and is probably a lot stronger than most governance systems.

It is worth putting on record my view, which I hope the committee shares, that the SQA staff worked hard and extremely well, under considerable constraints, to make the 2001 examination diet a success. I am pleased by the progress that the SQA witnesses have outlined. It is clear that the SQA is hitting milestones earlier than was anticipated. Nevertheless, confidence in the system is all. I am pleased that the SQA has undertaken a risk assessment of the plan. What risks did the assessment identify? What are the appropriate contingencies that you defined?

Last year, the SQA engaged in a risk assessment that was supported by PwC. Committee members will remember that fact from previous submissions. We identified risks at that point and defined contingencies. In the past couple of weeks, senior management and a representative group of staff have reconsidered those risks. We also reviewed our contingencies and added new risks.

The risks will not be a great surprise to the committee as they are the usual ones—data accuracy, the robustness of our IT systems and current staffing and resources. Key milestones have been achieved in those areas. I am unsure how much detail the committee wants me to go into, but I reassure the committee that the risk assessment was done not independently, but under the scrutiny of PwC at its office. We have a draft report on our risk assessment workshop.

I reassure the committee that we have learned lessons on risk and its identification, which have been built into the system. We have transferred risk identification and assessment skills to our staff. It is important to have PwC's external input, but it is also important to make it the norm throughout the organisation's operational sections, where there are critical dependencies for delivery, that risk assessment is identified and contingencies are planned. We are working towards that. It is good to be in a position to reflect on previous risks and look to the future by identifying new risks and preparing for them.

What kind of new risks have you identified? I do not want a huge amount of detail, but just a flavour of the scale of the new risks.

I will give an example of one risk. The SQA delivers a complex service and it is possible for individuals to have key responsibilities in the organisation that only they have experience of carrying out. We identified such a group as the singletons. About four months ago, we identified a significant number of them, which has been narrowed down to a critical number. Contingencies are in place to ensure that when someone is no longer there to do a singleton's job, the task is still carried out.

I have examples from a couple of meetings that Anton Colella and I have had in the past few days that have given us reassurance from those to whom we deliver the service. I spent a day at Falkirk College of Further and Higher Education, to assess its view of how we deliver our service. There was positive feedback about how the service is being delivered in the current year. Five or six staff asked me to convey to the SQA staff the fact that they feel that improvements in communication and data and so on have been of great effect.

We also had a meeting within the SQA with assistant directors from throughout Scotland, at which we briefed them on the examination diet position and got feedback from them. A couple of procedural issues were raised, but the response from the group was largely positive. Those individuals would know if there were problems with the current diet. This morning, Anton Colella and I met the Educational Institute of Scotland to discuss a range of issues, including the current examination diet. Again, some procedural issues were raised, but there was recognition that the SQA staff are addressing them positively.

That is the kind of stakeholder involvement that we need. The SQA cannot deliver the system on its own; it has to be delivered with individuals who work with us effectively. There must be two-way, open communication, and I reassure the committee that that is there.

I will let Jackie Baillie ask one more question on this subject. I guarantee Michael Russell an opportunity to come in after that. I am being generous this afternoon.

You are very lax, convener.

He is—he is being very generous to me.

I am helping to elicit information on behalf of the committee.

I want to pick up on something that David Fraser said. There has been a lot of communication with key stakeholders, but one group has been missing in the discussion: the parents and the students themselves, who have lost confidence in the system. Confidence has been partly restored, but there is still uncertainty—they are still not quite sure. Do you have any strategies in place to communicate directly to pupils and parents the progress that you are making?

Aside from the committees, I have established two lead roles on the board. One is for people in management, which I have picked up; the other is for stakeholders. I am conscious that we put a lot in place last year with the involvement of college account managers and many major organisations. That should not stop there. David Fraser and Anton Colella have that role, with the Executive. They need to think through what we might do. It is impossible to talk to everybody, but we need to ask what more we might reasonably need to do to address the gaps. We recognise that need and want to talk about it. I hope that we can talk about that further, perhaps at other meetings such as this.

Getting communication right is a complex matter, with 750,000 entries and a very large number of individual candidates. We need a mechanism that includes candidates and students in the discussion. We need to evolve how we do that over the next few months. It will be difficult to achieve that for the current diet, but we would want it to be in place for future diets.

Much of this committee's work has centred on how best we might consult young people and on best practice in that regard. It is an area that needs to be considered very seriously and pushed forward, even though that is not easy.

I refer again to the evidence from the Scottish youth parliament. It may not be entirely representative, but it is quite concerning. The stakeholders in all parts of education may be satisfied, but you have to ensure that the consumers are satisfied too.

That is very important, but remember that we are serving the whole population. It is important that we talk to young people, but we are also talking to people in work. One of the greatest skill deficits in Scotland is in the workplace. I remind you of my earlier point about using the SCOTVEC model in the south of England, which closed just such a gap for us. The issue is wider.

Nobody is denying that, but, equally, we cannot deny that young people are absolutely crucial to what you do; their confidence in the system is absolutely crucial.

Agreed.

That is a long-term issue. There is also a major issue around the fact that the currency of the certificates with which you deal is not yet sufficiently understood in Scotland. The certificates have not yet achieved the status required for people to understand them, let alone the names of the qualifications. That is something that you must work on.

Let me address a different issue—this is the one that I wanted to go into in detail. During the inquiry that followed the problems in 2000, we talked a lot about the culture of the organisation and about how it could work as a single body. We acknowledged that there were difficulties when the SEB and SCOTVEC were brought together and that the wounds had never fully healed, certainly at management level. Now, an entirely different set of people are there, who probably do not have any loyalties on either side of the divide. How has the culture developed and changed? Where is it going?

I want to introduce another issue. It seems that the three witnesses represent the triumvirate that shall be the SQA: someone with a strong background in the public sector and public service; someone with a strong background in business; and someone with a strong background in education. In the first year of the process, I was concerned by the great deal of evidence that the approach was business-oriented, not education-oriented. Where is the public service culture of the organisation going? Although the SQA's customers include businesses and others, it runs an education-oriented activity, which needs to be run well. There is no reason why a public service body should not be run supremely well. It should not have to pretend that it is something that it is not.

What are the three witnesses' perspectives on those issues?

Last year, we had no alternative but to put in place substantial and almost draconian control techniques. Anton Colella referred to those techniques as a plan, but they were far more than that, because they contained everything and were grouped under 12 success factors. If we had not introduced those techniques, we would not have delivered the result.

That was last year; this year's task is to build confidence and capability. That takes us into the field to which Mike Russell referred. The structure that has been put in place involves a new chief executive, who will give leadership to the whole organisation. He is supported by two executive directors: Anton Colella, whose background is in education, and Brian Naylor, whose background is exclusively in public service.

We have appointed eight general managers, whose job is to fit the organisation together. Previously, the organisation worked in pipes; there was no diagonal across it. The general managers who are now in place are beginning to make the organisation work horizontally.

The next challenge is to get the organisation working at the interface between the first line of management and people. As I said earlier, we very much want to work on that. There is absolute recognition of Michael Russell's point. We are trying to move the structures to fit a culture that will make the organisation cohesive. We are building on the back of the measures that we put in place last year, for which there was no alternative.

Perhaps David Fraser will add to what I have said about the cultural element and explain what he would like for the organisation.

I have a strong commitment to delivering a high-quality public service and value for money. The two things must go hand in hand. I want the SQA to take it for granted that we deliver a high-quality, value-for-money public service. However, I agree absolutely with Michael Russell's point that we should be driven by the educational and other needs that the SQA serves. Instead of having an internal culture that looks only within the SQA—which was part of the problem in 2000—we must be an outward-looking organisation that works with businesses, colleges and schools. We must be driven by an agenda to improve quality in those areas.

As was said earlier, we must ensure that we match or exceed the best in the UK and in Europe. We should be judged by those gauges, not by whether the organisation has effective governance or is delivering value for money—that should be taken for granted and should be done as a matter of course. We should be judged on whether we meet the needs of students, young people, people going through qualifications and all the users of the service. As chief executive, that is how I want to be judged.

I have little experience of the previous culture, but it is clear that the culture of the future, as David Fraser and John Ward have said, will be outward-looking. Our culture should involve asking what the users of Scottish qualifications need and working to the best of our ability to provide for those needs. We must ask what young people, teachers and managers in schools need; what teachers, managers and lecturers in further education need; and what training organisations and employers need. We must respond to those needs. The SQA might require a different culture to respond to different needs, but we must clearly identify and define those needs and structure ourselves to support them.

In the schools sector, the SQA is in a unique position to contribute significantly to the attainment of young people in Scotland. We possess, monitor and maintain the standard. We gauge the standard annually and have thousands of appointees who oversee it. It is vital that we recognise that the information that we hold can contribute significantly not only to meeting the needs that exist but to raising the level of attainment both within Scotland's schools and within colleges and workplaces, where people are looking for skills development and improvement.

I am glad that Mike Russell asked that question. I will raise more mundane matters. I would like the witnesses to reassure me about some of the points in their submission—I imagine that I will get that reassurance, but I ask for it all the same. The submission states:

"We are also looking at ways to improve the quality assurance of our qualifications, in particular moderation activity for those qualifications/units assessed internally by centres."

What is the problem, and how are you getting on with it?

To save time, I will mention a couple of the other points on which I would like reassurances. When the SQA was experiencing real problems, we talked about training for the staff who handled the data that came from schools. Those staff found problems in understanding data when they looked at information on screen. How is that training coming on?

The submission also mentions the challenge of establishing better communications across the organisation.

On moderation, one of the SQA's key responsibilities is the quality assurance of our qualifications at both the external and internal assessment stages. The SQA wants to develop and improve the quality assurance of internal assessment even more than we are doing already. We identified that area because we want to develop it even further. We are going beyond what we are doing in schools and are implementing quality assurance for our higher national qualifications, particularly in colleges. Over the past two years, our focus has been on our national qualifications in schools, but it is important that we ensure that the quality assurance regime that we establish for all our qualifications is maintained. We included that comment in our submission to inform the committee that we are taking that work forward.

On staff training, last year, we had temporary staff who came in at the peak time for data entry. This year, we have consolidated the permanent staff who have dealt with that area in the past and have identified the gaps for temporary staff. We are working with an employment agency to define the exact skills base that we are looking for and are preparing the training that we intend to put in place prior to employing staff.

On your third point, which was about internal communication, were you asking about communication within the SQA?

Your submission states:

"One of the challenges for the next year will be to continue to improve communication laterally between units and functions".

In the past, the SQA was characterised by a separation of development activity from awards. Sometimes, one activity did not communicate with the other. What is desirable as a qualification may not be deliverable, as far as the data are concerned. Both areas are now under one directorate, within which we have established three management groups to cover all functions within the organisation: the national qualification management group; the higher national qualification management group; and the vocational qualification management group. Those groups represent staff further down the management chain and ensure that all sections of the organisation are fully aware of developments. They also ensure buy-in across the organisation. That is an important development, which we highlight because we want to move forward. We think that the establishment of the management groups will go a considerable way towards meeting that challenge.

I could say a lot about communication. The committee should be aware that a key risk to organisations is when people do not raise issues and keep quiet about problems. That risk was identified in the committee's report on what happened in 2000. I gave every member of staff a message when I met them: if they have a problem or a difficulty, they must raise it, because the organisation will see that as a positive step. We have encouraged that approach, because we can ensure that we address issues only if people raise them. The challenge is getting people to be prepared to do that. [Laughter.]

The source of the mirth is the fact that we were talking about Scottish Borders Council and the fact that there were problems there. You should get in there and sort them out.

There is a job of work to be done.

What does David Fraser feel about the winter diet?

The winter diet was established at the request of the further education sector. We received a small number of entries—around 360 altogether—and that makes it difficult to administer. The costs are high; in some instances the costs for an entry can reach several thousand pounds.

Complex issues are involved in running a winter diet. In terms of the advice that will be given to ministers, a report will be prepared that includes what the SQA perceives to be the problems and the benefits of the winter diet. The structure is complex for a small number of candidates—the majority of whom were from schools, rather than from colleges, and were using the winter diet for resits.

I was sceptical about that.

As David Fraser said, the board was concerned about the cost of the diet. It was also concerned about pressure on the markers. Members will remember that the markers are the same teachers who are used throughout the year. The pressure that is put on them and on schools, who have to release them, is significant. We have to balance the cost and the viability of the diet.

The board's view is that the winter diet should be examined carefully. We will make recommendations to ministers along those lines.

One of the key issues, which was evident in the report and from discussions, was the issue of pessimism among staff. Your interim predecessor said that, because he was a Partick Thistle supporter, he was used to pessimism. Since he made those remarks, Partick Thistle has made a remarkable improvement, which is a consolation. That has been almost paralleled by similarly remarkable improvements at the SQA. How do we retain that confidence? For example, we could look forward to another Glasgow side getting into the premier league next year. From our visits to the SQA offices, we felt that more needed to be done about staff confidence.

I will end on a positive note, as that would be a useful conclusion to this afternoon's discussion.

We need to build on the success of 2001. The staff felt lifted by that and we need to ensure that that is sustained. We also need to ensure that the SQA staff see a positive, dynamic and developmental future for the SQA. They need to see that they have a leadership role throughout Scotland in the field of qualifications and awards. Everything that we have talked about this afternoon needs to be in place. The SQA staff are ready for the challenge of meeting members.

I have been impressed by the positive approach of the staff whom I have met. I could have found a group of staff that were demoralised, but that was not the case. As we move into the future, the next phase of development at the SQA will be to go beyond meeting the requirements of 2002 and future diets.

I thought that you were going to say that the secondment of John Lambie to Dalkeith would be useful.

Thank you for that contribution. Let us continue the process of progress and improvement. I hope that the 2002 diet will be delivered in the way that the diet was delivered last year.