Education, Culture and Sport Committee, 01 Nov 2000

Meeting date: Wednesday, November 1, 2000

Official Report 223KB pdf

---

School Exams

Good morning. I welcome back members of the Scottish Qualifications Authority—there are a few more of you on this occasion. Although we are missing a couple of committee members, we will make a start. I know that Bill Morton has a presentation to make this morning, but first I wish to ask a couple of questions. Could you begin by introducing your team, Bill?

Thank you, convener. Jean Blair, whom members may have met before, is a member of our staff and is involved in the project management of the operational review. Billy MacIntyre is acting director of awards and Rhona Wright is head of our moderation and assessment unit.

Thank you. You will be aware of this morning's press coverage of outstanding appeals for students of the 2000 diet. Are those reports accurate? What is the present situation? What communication was made with those pupils whose appeals are outstanding? How was that communication with pupils and their schools dealt with? Can you indicate the likely time scale for clearing the backlog once and for all?

I very much regret that, once again, in appearing before the committee, I have to start by making an apology. Unfortunately, we did not fully meet the deadline of 31 October—we missed that deadline with 196 appeals outstanding, from a total of some 40,800.

The simple reason for that delay, for which there is no excuse, is that we were checking to confirm that the results that were communicated to the centres were absolutely correct. We wanted to conduct further checks in one or two cases—that is, in 196 cases. Letters were sent to the centres yesterday and the time scale for resolving outstanding appeals completely and accurately will be either later today or tomorrow.

There were reports that students had contacted the SQA and had not been able to receive information. Had you made any arrangements for that kind of response?

We had set up the helpline. However, the main line of communication should be the presenting centres, which are the best places for students to obtain information. I am not being defensive; I just think that those centres are the best places for readily obtaining the confirmed information.

So the committee should make it known that the best route for people still having difficulties is through the centres and that you have made contact—or will be making contact—with all the centres.

We have made contact with all the centres and all the information on the outstanding 196 cases will be conveyed to them. I regret that we are still in this position; however, I am sure that the committee will understand that some significant issues must still be addressed before we can be 100 per cent confident that all will be well for the future. As I said, the best place to obtain information is directly from the centres, to which we will communicate the information as quickly as possible.

I am sorry. I missed whether you said that there was a time line for completing the outstanding 196 cases.

We are working to confirm the 196 cases today and tomorrow. We are also chasing up some information that we have been waiting for from third parties.

Are we are talking about 196 individuals or 196 appeals?

We are talking about 196 appeals.

Although I am grateful for the answers that you have given, Mr Morton, you must accept that public faith in the SQA is at an all-time low; yesterday's public relations disaster is immensely damaging to your work of bringing the organisation back to its stated task. What arrangements had you made for briefing the press, the Minister for Education, Europe and External Affairs and the committee? Committee members first became aware of the matter yesterday evening through calls from the press. To date, we have received no information apart from your statement, which is likely to create further public disquiet that the SQA has not learned its lessons.

I do not have an adequate response to that point. Yesterday, I found myself in a similar position to that of committee members. The information was conveyed to the Scottish Executive yesterday in its role in monitoring the appeals process with the directors of education. Unfortunately, there is no excuse for what happened, nor am I offering one. The information was not conveyed to me, which left colleagues in the SQA who were dealing with the press at some disadvantage. Such a situation is not acceptable and I am surprised that it happened. In my communication with SQA staff, I will reaffirm that it is unacceptable and should never happen again.

There will be a strong sense of disquiet, because much of the evidence that the committee has heard on what happened until August was couched almost exactly in the terms that you have used today. People did not know; people were not informed; information was not flowing. Many of us have welcomed your appointment and the changes that you are making. However, what happened yesterday rings very loud alarm bells for the committee and the majority of people in Scotland that the situation is recurring. There has been some talk of an external body or individual that would monitor the SQA's work to ensure that it met its deadlines and communicated information effectively to the public. When I asked you about that suggestion at a previous meeting, you were not in a position to comment. I ask you to comment on it now, because I think that the events of the past 24 hours prove the need for an honest broker who can tell us what is happening.

I am not comfortable with being cast in the position of not giving full and accurate information to the committee. That should not happen, but it has. It has caused some pain in terms of poor public relations. It has set back the SQA's recovery, which although fragile was making progress. It is not acceptable and I will have to deal with it.

I have made it clear that one of the issues that must be resolved is the availability of adequate management information. Unfortunately, several issues need to be addressed in getting the SQA back on track. We are making progress on some issues, if not most of them. Clearly what has happened is a regression, which is more than unfortunate. The price is paid not by the SQA, but by the people whom we have let down. I apologise for that.

Mike Russell asks about external scrutiny. Although it is up to others to make a final judgment on that, most of the issues that need to be addressed are internal, organisational ones that relate to the SQA getting its act together. We must sort out the process and structural issues and introduce a new set of behaviours that focus on meeting the needs of those to whom we provide services. I have found that to be lacking within the SQA, as my presentation will emphasise. My primary objective, and that of the SQA, is to get the organisation to the level of competence that it should be at and, indeed, should have been at long before now.

You are still avoiding the issue. Given that public disquiet, and disquiet among clients, must have been caused by what has happened in the past 24 hours, reassurance—I am sorry to use that word again, as it has haunted the entire inquiry—must be given to the public and to your clients that you can deliver on time and to the necessary quality. In those circumstances, is the time not right for an extra element to be put into the equation so that we have an honest broker, who can say objectively what is taking place? That could reassure the public, the Scottish Executive, this committee, teachers and, most important, pupils who have again suffered.

I still believe that if we put right—sorry, when we put right—all that needs to be fixed in the SQA, that will be the basis on which reassurance should ideally be delivered. One of the first improvements was the establishment of account management teams for all the centres, including schools and further education colleges. That has significantly improved the channels of communication. It is for others to judge whether to add a level of scrutiny, but my opinion is that when we get everything sorted in the SQA—members know that a series of issues must be resolved—that will form the basis of any reassurance as we go ahead.

Convener, does the committee want to ask for a list of key dates in next year's diet, starting from now, that the SQA believes have to be met? Such a list should be made public, so that we know the dates and can keep a close eye on them.

I am more than happy to ask for that information. It would be helpful to allow Mr Morton to give his presentation, so that members can ask questions on it. If that issue is not picked up, we will come back to it.

We would be happy to provide that information and to put it in the public domain. I hope that it would give some comfort to the public about what we are striving to achieve. We could be held accountable to those dates. Some of the dates that we are working to are still subject to consultation with various interested parties. For example, we will not be in a position to dictate on the date of information collection on internally assessed results from centres until there is a consensus on it.

I am sure that, in the information you provide, you can indicate where that is the case.

The bigger issue is that the SQA needs to be held accountable. I understand and accept the need for that and I am happy to comply.

It would be useful if we could publish the dates as an annexe to our report. That would provide a forward-rolling checklist of what lies ahead of us.

Are the 196 appeals all in one area or they spread across the range of exams?

They are spread across a number of centres and subjects. The appeals do not follow a pattern.

One of the problems that the committee has identified is the lack of an appropriate management system to set off alarm bells when a deadline is not going to be met—in other words, a reporting structure in the organisation that would allow managers to inform you that they were unable to meet a deadline. Were the problems not picked up because they affected a number of subjects and were not in a cluster? I am surprised that you do not have the systems in place that would enable people to report problems a couple of days in advance.

You raise two issues. First, the 196 cases were discovered because of the checking and double-checking that we carried out to ensure that we could confirm that all results were complete and accurate. It was in the final checks before the results were issued that we discovered that there were 196 cases with outstanding issues.

Secondly, I cannot offer any excuses on behalf of the organisation for the fact that I was not informed of the problems. I believed genuinely that lessons had been learned about accountability and communication. When I asked for assurance, that was given. I know that that is hauntingly reminiscent of recent history, but I will ensure that, when I request information, I am given the complete and accurate story. That is the management information that I require.

Indeed. As you say, there are two issues. Either the management system is not correct—in other words, instead of aiming for the deadline of 31 October, you should have aimed for 28 October, so that you would have enough time to conduct checks—or people were aware that you were not going to meet the deadline and did not tell you. That is a cultural issue that we are addressing more generally.

Sadly, criticism centres on the latter problem rather than the former. Accommodation had to be made for the fact that we were dealing with twice the normal volume of appeals—40,842 appeals against higher and sixth-year studies results, as of 9 October, from which the 196 cases that we are discussing were drawn. I believe that planning for that exercise was adequate. As Ken Macintosh rightly points out, the organisational change that needs to be managed in the SQA involves behavioural and cultural changes. We are not as far down the line with those changes as I had hoped and had a right to expect.

Was information about the 196 cases communicated by the SQA, or did it come from the other partners involved—directors of education and/or the Scottish Executive?

The information came from the SQA.

There is an issue of deadlines within deadlines. Obviously, you should have short deadlines, which act as a trip wire and which you can make public.

I have another point about the number of appeals. You will recall that 80 per cent of English departments did not do higher still, so the number of appeals would have been even greater if pupils from those departments had been involved. More elbow room will be needed next time.

I am not sure that you should make presumptions.

The culture of the SQA is a crucial issue. Over the past six weeks, we have heard much evidence to suggest that the culture of the SQA was such that, when things started to go wrong, there was no chain along which to pass information, or, if there was a chain, individuals in that chain were keen to keep information back in the hope that they might be able to rectify problems without anybody finding out about them. If that culture is still endemic in the SQA, would you admit that substantial problems lie ahead in preparing for next year's diet? Things may not have changed at all.

If that was absolutely the case, the answer would be yes. This incident has had an effect on the public's perception of us—we have slipped a little down a fairly steep slope. However, we are making progress in many areas. I would not describe our problems as endemic. One would imagine that, when a management link is put in place, it would bring accountability and effective, clear, concise, complete and accurate communication. That has clearly not advanced as far as I had a right to expect. We will have to deal with that.

I am sure that we will discuss this when you talk about the operational review—and I do not want to delay that—but you may want to talk about the carrots and sticks that are being used to change the SQA's culture. That issue may become central to our considerations.

A number of techniques may be employed in the management of change. As you will see in the presentation—and as you have already detected in evidence—a lot of ground has to be made up in a short time. We had 18 months to plan, prepare and set in train contingency plans for diet 2000. I have said that it was like having 18 hours. That is not literally true, of course, but it is not too much of an exaggeration. I used the phrase to illustrate the pace at which we have had to move. A lot needs to be done. I will not pretend that everything is fine and I will not hide the fact that we face big and serious issues. My presentation will summarise those in order to show the committee my focus—and that of the senior management team and the board—when considering the necessary changes. Most of the changes are urgent. In my experience during 14 years as a chief executive of a public body, I have found that the hardest thing to do, sadly, is to change the way people behave—their culture.

Will you furnish us with a time line, which we will add to our report?

Yes.

It will be available to anyone who is interested and can be used over the coming 12 months. Would you like to move on to your slide presentation?

Paper copies of the slides are available. I have tried to summarise the evidence that the committee has already heard on the content of the operational review.

People on the public benches cannot see.

Can the cameras pick up the slides on the screen?

They might be able to, from time to time, but we have paper copies that we could make available to the public.

We will do that, so that people on the public benches can follow the presentation. If there are not enough copies, there are a few seats at the side with a better view of the screen. I am sorry—this is not ideal, but we are trying to be accessible.

The slide seems to be on the television monitor now.

Yes, it is.

If necessary, I could run through the presentation later.

We have the presentation on the monitors now, so it should be okay. You may carry on.

In essence, I want to outline the context and the process of change, which is being managed with the sense of urgency that we all agree is necessary.

I will start by giving a quick reminder of the aims of my operational review—an exercise that was undertaken as a matter of some urgency and which was completed within four or five weeks. We wanted to identify the problems and, more importantly, get behind them to discover their causes so that we had the knowledge and understanding that would ensure that when we proposed improvements and changes, they would be based on as much reasoning as could be assembled within the time.

The scope of the exercise was such that we had to look at the organisation as a whole. It was not sufficient to conclude that, because the evidence pointed to a problem with data management, it was an issue merely of process and how we conducted our business. It was clear that there were structural issues to do with the way in which staff and the organisation's other resources were deployed. Most critically, there were—as one would find in any organisation—behavioural and cultural issues that needed to be faced so that changes could be made.

Our clear imperative is the need to deliver successfully next year. Therefore, the key is not simply to improve the processes and think that all will be well. It is clear—to put it bluntly—that new processes will not run on old structures. It is important to consider changes and to manage them across three areas—processes, structures and behaviours—sensitively and effectively.

I will summarise the findings of the review with two slides that indicate the considerable range of issues that the review raised. The list could be construed, as it has been, as a catalogue of horror—a litany of problems—at the centre of which were the failure under pressure of the data management elements of the process and the undue pressure that was placed on the staff within the structure that dealt with data management. However, I prefer to consider the list more constructively—as an agenda for the areas in which there needs to be change.

The SQA is going into this review process openly and honestly. That is why, in my previous evidence to the committee and in the presentations that I have given to staff and to the board, I have been open and have given a complete overview of what I believe the issues to be. We cannot delude ourselves by thinking that in certain areas the situation is not so bad. We have examined the situation in the cold light of day and the list shows the areas where something needs to be done. I will draw members' attention to one or two of them.

Leadership and management failings were manifest throughout, but primarily in the failure to scope effectively the scale of this year's task and to plan, prepare and communicate. Alongside that, an assessment of risk would have indicated what could go wrong. The absence of effective contingency planning meant that, unlike in the year ahead, much of the resolution of problems was attempted on an ad hoc basis.

Checks and balances have been mentioned. There is a need to ensure that everything is all right before moving on to the next stage. That is a lesson that I had hoped the organisation had learned at corporate level, but our experience during the past 24 hours suggests that there is perhaps more evidence in support of the suggestion that it needs to do that.

I have not mentioned the structure, because I intend to go into that in greater detail in a moment.

Slide 3 shows more of the issues that relate to processes, structures and behaviours. It is clear that, in certain crucial areas of the organisation, there was no sharing of knowledge or the sort of investment in training and development that should have been in place. If a member of staff was not available, for whatever reason, to play their part in a process or sequence of events, the process tended to fall into jeopardy. That is not acceptable to our organisation or to the business in which we are involved.

We operate on a split site, about which I shall say more. In my view, the SQA is not yet the organisation that it ought to be, nor has it that potential. In a sense, it is constrained by the heritage of the Scottish Vocational Education Council and the Scottish Examination Board. Some of the cultural issues that emanate from that heritage—which we must deal with—have been reinforced by the fact that the SQA operates over at least two main sites.

However, the biggest issue in relation to changing behaviours—which must be reinforced at every turn—is that the organisation must lose its internal view of itself. It must realise that it has to meet the needs and expectations of others and it must have a truly effective customer focus.

I am being a little absolute in some of my comments and I do not want members to get the impression that everything in the SQA is worthy of such criticism, because that is not the case. There is much good within the organisation on which to build. Despite the problems in the SQA's recent history, it still intends to deliver—to a high standard—much of its work, although it would be wrong to pretend that serious underlying issues could not threaten that.

I believe that the problems that we encountered in our review are serious, but that all can be fixed. Some can be fixed by doing things differently and others by doing altogether different things—we will have to replace parts of the organisation. That is a big challenge, but I believe that significant progress has been made in the two to three months during which I have been at the SQA. However, we have not made enough progress in meeting others' needs or in our performance—I understand that point.

On some of the improvements and changes that are in place, our first priorities were to produce complete and correct results and to accommodate and manage the appeals process. However, we managed to accommodate that process better than we managed to manage it, given that 196 appeals out of 40,800 remain unresolved.

It is important that the committee understands that the SQA's business is wider than its main focus, which is on national qualifications. About two thirds of the business that is measured by flow of candidates is vocational. It is also important that the committee understands the continuing value of lifelong learning and the marriage between vocational and academic learning. The SQA holds that to be a fundamental principle, but there is a risk that that principle has been compromised.

I take this opportunity to thank publicly our candidates and our partners in the vocational side of our business for their patience and tolerance while the situation has unfolded, and while the SQA has focused on solving the main problems.

We have strengthened our management by introducing account management as a demonstration model and, more importantly, as a new way of working. Schools and colleges—our centres—have complained justifiably that the SQA did not listen properly and did not provide the right information on time. Account managers will improve on that situation progressively—they will be the main conduit or point of contact between the SQA and the centres. As a result of that way of working, we are starting to pick up valuable intelligence about exactly what our customers need and expect from us.

I said that we had 18 hours to plan for certification 2001—that was an exaggeration, but it gives a sense of the pace, rather than the haste, at which we have to move. That time is under way.

Improvements and changes will include, wherever possible, rationalisation and simplification of the processes. The SQA has inherited the way in which things work and there is a certain loyalty to many of the processes that have been inherited from SCOTVEC or from the SEB. Some of those processes, because of habit and repute, are still with us. Many of the processes have to be interrogated to examine whether they are the best way of getting from where we are to where we need to be.

Perhaps the best example of that is the critical element of identifying the number of markers that we will require next year and how we will go about recruiting them. An example of how a process can be compounded or confounded by the structure is that five individual units in the SQA's current structure are involved in that task. I see no need for that—we need a much straighter path to achieve what we want to achieve and to meet people's expectations. There are clearly issues that need to be reflected in the way in which we proceed with that task next year, which will be based on the learning experiences from this year.

I will say more about my view that the organisation is not structured according to its core business. I can only conclude—this might seem slightly unjust, but I think that it is a fact—that the organisation's current structure is the product of a compromise between the two organisations that existed previously. I have been sending a strong message around the organisation that the new body is the SQA and the other organisations are no part in it—they no longer exist. That is a wake-up call or a reminder to the staff, which should allow them to move forward. That is an important aspect of the SQA's culture.

It is self-evident that strengthening customer and partner focus is a primary need. We must invest in people—I mean that in the truest sense. That is not to say that the SQA has not considered training and development issues or how people's skills and knowledge are deployed in the organisation. However, we need to involve staff and their knowledge more in how we plan for the future. Certainly we need to invest more in ensuring that we have the right numbers, capacity and capabilities of staff when and where they are needed. That relates clearly to the heavy burden that the operations group faced last year and which it must still face. The group that dealt with data management also deals with the correction of results, appeals and preparation for data collection next year. If the bulk of the work in those critical areas tends to fall on the same people time and again, that suggests that the organisation—which employs 560 people—is not properly disposed for what it needs to do.

It behoves the management of any organisation to consider issues such as having a single site. I emphasise that I am not saying that a conclusion has been reached that a single site is inevitable, but there are obviously communication problems. There are also cost and value-for-money issues that are associated with having a number of separate locations. Efficiency and effectiveness issues must again be addressed bravely, openly and honestly. We must compare the advantages and disadvantages of what we have against any alternative. One alternative would be to locate the organisation in one place.

I turn now to where we are in relation to plans for 2001. This is linked to the issue of obtaining the reference data to ensure, for example, that a centre has approval to offer a particular course in the qualifications portfolio.

More important, we need to get the registration and entry data correct this time. Many problems flowed from the fact that we failed to do that last year. That had a knock-on effect on moderation and the recruitment of markers. Instead of hurrying, we have tried to get it right. I know that there was concern about the fact that guidance went out to centres later this year. In the circumstances, that was perhaps unavoidable. However, it would have been better if we had listened to what people were telling us last year and tried to accommodate as much of that as possible. This year we have tried to ensure that the centres with whom the data originate will have the opportunity to check that the data that we hold on their behalf are complete and accurate. Last year some problems stemmed from the fact that that facility was not available to centres and that they were not aware that there were problems with the information that we held on their behalf. We are trying to simplify the process and to make it possible for centres to check data, so that they can be assured that information is complete and accurate.

The internal assessment target is the subject of consultation—I know that there are a number of views on that. We are trying to find the best way of collecting information on the internally assessed units prior to adding the externally assessed component just before results are processed. This is very much about giving centres the right and responsibility to verify the information that we hold on their behalf. Members will see that a number of the checks and balances that should have been in place last year are now being installed to ensure data confirmation by centres.

There are a variety of views on the examination timetable, which has been the subject of consultation. It appears that it would be counterproductive to move or extend the timetable. In consultation with centres, we have tried to alter the constituent elements of the timetable to take pressure off centres and candidates. We await a final resolution of that issue.

The next point relates to changed ways of working and new behaviours. Instead of continuing with our previous method of appointing markers and moderators, we have nominated one accountable project manager to deal with that. It is his responsibility to ensure that the appointments process is conducted properly and he will account to the senior management team for that. Because of the issues that relate to capacity and capability, it is important to simplify the processes within operations as much as possible. We must ensure that the additional staff and training and development that are required in this area—about which we are acutely sensitive—are put in place very quickly in preparation for next year.

Internal communications are a problem and a challenge for us. I spent yesterday giving this presentation to all the staff at the SQA, because I believed that they had a right to see it even before the committee did. However, it is clear that our external communications are not what they should be, as witnessed by the difficulties in public relations management yesterday. That issue is being examined and improvements are being made progressively.

The next part of my presentation excited most interest from staff. At present, we have a compromise between two previous organisations, which are involved in a merger that is a long way short of complete. Although the structure of 21 units looks devolved, it is in fact disaggregated, as many of those units have their own sovereignty. That is not a criticism, but the structure does not seem the best way of organising the SQA to allow it to carry out its functions.

We will adhere to three important principles. First, the SQA must realign itself to meet the needs and expectations of those whom we serve and to whom we provide qualifications and assessment services.

Secondly, we must adopt a much more corporate approach, an element of which—in terms of behavioural change—is collective responsibility. Although that does not need to be introduced as such, it certainly needs to be consolidated and expanded. However, the corporate approach means that the senior management team collectively with the board must provide the organisation with a clear and consistent strategic direction to allow the board to meet the expectations that are placed on it. Furthermore, it means that there is a difference between developing the organisation's business and running it.

Thirdly, we must introduce greater flexibility. The unit structure is one constraint but, from what I have seen so far, I believe that the organisation contains the necessary skills and knowledge to perform at a much higher level and indeed to fulfil its promise and potential. In many instances, staff are in the wrong place at the wrong time doing the wrong things. Such issues must be resolved, which entails having a much more flexible approach where staff can move more freely according to changes in our market—if I may use that term—which will allow us to respond to those changes effectively and quickly. The current structure is a constraint on that.

Perhaps this is a Freudian slip, but the phrase missing from the next slide is "Senior Management Team". The senior management team will work with the board. The slide shows how the senior management team and the board see the SQA's structure around the turn of the year. The organisation will be realigned to its core business, which is qualifications development and qualifications assessment and awards. We need to identify and focus on what activities contribute to each of those aspects. It is important that the corporate services that run across the organisation can do that effectively. For example, information technology services are locked into a relationship with operations; however, the whole organisation needs effective IT support, which I will speak about later in relation to our need to be better prepared for e-business. An organisation can make such preparations if it has a narrower focus on some of the key facilities, which should be much more corporate.

Although accreditation is an aspect of the organisation's work, in some senses that risks isolation, because a conflict of interest requires to be managed. As the SQA is an accredited organisation, we need some distance. However, there is a price to be paid for that, as I will explain.

As I said, we will be moving to this model by the turn of the year. It is designed to be sufficiently bold to ensure that the necessary process changes work. However, the danger that I have encountered many times before is that we might be more cautious about those process changes that might meet resistance.

We now get back to curing what should really have been prevented. I believe that the set-up of the SQA board and senior management team as shown on slide 8 is what is required. It sends some clear signals and will help to change behaviours for the better. For example, I made it clear to the staff that the fact that we had 21 units was part of the history of the organisation. In the short term, however, a lot of the activities will still be recognisable in terms of the groups of people involved. We cannot risk having too much disruption at a critical stage in the work of the organisation, which will be the next nine months to a year.

At stage 2, the senior management team is retained. If there is a family of activity around how we deliver our services—we could, in a business model, talk about near-market and far-market activities, although I do not think that such phraseology is appropriate for the SQA—and if this is where we deliver, it makes sense for the family of activity to include everything that we do in qualifications.

About this time next year, everything that we do with qualifications, including how we deliver directly to our centres and to our candidates, will come under one senior management responsibility and will be within the one family of activity. Qualifications and assessment development will sit alongside the quality assurance of effective certification.

What we have done on incremental policy developments and on the development of new qualifications and means of assessment will effectively be addressed through a proper approach to strategic planning. That will mean that the organisation has a view on what it needs to do and on how it needs to position itself, perhaps three to five years hence, rather than within the short-term horizons that tend to be set annually. That would engender an effective planning process, benchmarked against best practice wherever it is to be found. The long-term objective is not to make good the damage that has been done or even to come alongside any competitor organisations; it is to move towards and aspire to something that suggests that the SQA delivers its promise—to be the best qualifications organisation.

That will mean that we will have not only to do things differently, but to do different things. Some things that the SQA does will have to be abandoned on a reasoned basis because they are not adding value to meeting our core business, but still taking up a lot of resources. We need to think ahead and develop new types of qualification, perhaps including new means of assessment. It does not take much wit and wisdom to suggest that a lot of the assessment could, not far down the line, be web enabled. Some organisations in the same business world as the SQA are already going down that path. The SQA is not. That reinforces the point that the IT resources need to be more than just the focus of data management. We need to start to think in terms of the bigger picture, as appropriate to the method, rate and scale of the change in our field of activity.

Any organisation in the public sector, particularly in following the principles of the Turnbull report, needs to pay explicit attention to management information and to the identification and assessment of risk, alongside effective internal audit. To me, internal audit means finding out how we do things now so that they can be made better through a greater understanding of strategy and planning. The SQA perhaps needs that more than most similar organisations.

We maintain a distance with accreditation, as is necessary. However, accreditation is also a tremendous source of knowledge about what is required. It is important that accreditation can feed into our strategy and planning focus, which will be more established by this time next year, without any conflict of interest

By this time next year, corporate services will be an awful lot stronger, which will have a cementing effect across the whole organisation. What interested the staff most was the fact that the 21 units and 21 unit managers—part of a rigid hierarchy in an organisation that is still too fond of status, process and procedure for its own good if it wants to become more flexible and responsive—will be replaced by six networked general managers. Earlier, I spoke about a corporate approach. If the board and the senior management team develop the business and provide clear strategic direction using effective best-practice corporate governance, the general managers working with staff, stakeholders, partners and customers will ensure that the organisation runs on the basis of a corporate approach.

Thank you for that presentation. Some of our questions may now have been answered, but I am sure that there are more. It is appropriate that you ended by talking about the organisational structure, because that was one of our first concerns.

The structure is now much clearer, which is encouraging. There is one area that you did not cover: what is the role of the board? How will, and how should, the board operate?

There is one thing that I did not say that perhaps I should have. When I gave this presentation to board members and recommended these changes—which, although fundamental, will be managed effectively over the next year—their reaction was that they too would need to realign. They have volunteered to do so; they are considering the ways in which they will engage with the organisation in future. The role of the board in any organisation such as ours is non-executive. Its role is to provide us with clear and strategic leadership, showing us the direction that we have to go in. The board is also the final arbiter to ensure that the standards of corporate governance adhere to best practice at all times.

From my short experience of the board, I know that it contains a wealth of knowledge and experience. Its members have been willing to help us to rectify the problems of the past and, more important, to move us forward so that people's trust in us is restored. It would be naive and arrogant for us to suggest that we can restore our credibility by ourselves. However, we can meet the needs and expectations that are placed on us and so gradually restore people's trust in us.

It may not be appropriate for you and I to debate the role of the board, but a number of us may think that, if the board has allowed the current situation to develop, it would be better for its members to be replaced.

There may be some concern that your new structure puts information technology and finance and administration in one grouping, because IT is absolutely crucial to the delivery of the SQA's services. We have heard that the management of IT has been weakened over a number of years, which may have contributed to your problems. Will you reconsider the grouping? Will you consider strengthening the IT department, in terms of the qualifications and experience of the people who are running it, to ensure that it can deliver?

We would do that in any event. When we considered all the functions in terms of their synergy—how they sat together best—we felt that IT and finance were complementary in terms of what we needed to do and what we would be held accountable for. The general manager who will be appointed will have to have complementary skills that reflect both those big responsibilities.

One of the problems in that area in the SQA is the skills of those who are managing particular sections or delivering on particular matters. When we visited the SQA last week, we discussed with you the fact that there had been comparatively little investment in staff training. Although IT is central to the delivery of your function and IT issues have been crucial in the problems with the SQA, would it not be best if IT was seen as a core function and was independently managed again?

I am sure that you will remember my response when you raised that issue, which was that training and development is available and has been taken up in many parts of the organisation. Unfortunately it was not taken up in the operations division, despite the fact that it was available.

I hear what you say on IT and will reflect further on it. We need to think further about having the correct management capability in place. The new structure reflects that. We also need to consider everybody's skills, not only in IT but in the organisation as a whole. The experience of the past year suggests that some areas are critical to successful delivery next year. We are paying a lot of attention to those areas. Unfortunately, I thought that one of them was management accountability and communications—I will have to reconsider that.

You drew attention at an early stage of the review to certain innovations—for example, account management for schools and colleges. Although customer relations is always a corporate service, it has to infuse the whole corporation. How will you ensure that, in the crucial qualifications area, the quality assurance and certification division, for example, is customer oriented? How will qualifications and assessment development be customer oriented?

You are right—that has to seep constructively through the whole organisation. I will not pretend that that is not a challenge. There are aspects of the SQA that operate almost as though they had a divine right, rather than behaving with awareness or responsiveness to centres and candidates. There is some way to go on that. I can achieve that focus by repeating the message, to ensure that everyone who needs to hear it does so. As director of awards, my colleague, Billy MacIntyre, is responsible for ensuring that we do things effectively. Where things need to be done differently, they will be.

You are right to draw a distinction between customer responsiveness—communications and meeting needs—and customer-relationship management. It is not enough to conclude that if we communicate with customers, we are looking after them. I am more interested in the looking after them part—my colleague understands that only too well.

On Monday, Mr Galbraith indicated to us that he thought that a board consisting of such a large number of stakeholders from so many organisations was neither necessary nor desirable. Is it your experience of the organisation that that range of organisations and the unwieldy board mirror the unwieldy structure of the SQA and that that could usefully be changed?

I am sure that you realise that that is a matter more for the chairman and the board than for me.

Can I tempt you to have a view on it?

I have a view on it. Suffice it to say that I am content that the board represents a useful reservoir of skill and knowledge, which helps me to get to where we need to be.

The conflict among individuals over difficulties with data processing and the management of exams troubled this committee and, no doubt, a number of the people who have been observing our inquiry. We were also concerned that a number of people might have been blamed or made responsible—however one might wish to put it—for those difficulties. Even after your appointment as interim chief executive, a number of actions were taken, by mutual agreement, in relation to David Elliot, and disciplinary action was taken against Jack Greig. I am trying to get at the extent to which problems of lack of training, experience or communication contributed to the situation, in comparison with problems with the structural set-up. For example, it is clear that, even after Jack Greig was relieved of his duties in June, communication problems continued down the line.

We visited the SQA and met the people involved and I have read your statements. There is no doubt about the commitment of the SQA's staff—no one would challenge their commitment now, even if they had challenged it before. However, we have not been able to put our finger on the balance between individual and structural difficulties. Today, you set out clearly how you are seeking to address the structural problems. Now that you have been able to look back with hindsight—a facility that all of us have enjoyed—what is your interpretation of the balance between individual responsibility and structural responsibility? Given that there must be some individual responsibility, how will you help people to improve their management and communication skills?

The structure of an organisation is a bit abstract—the issue is more to do with how that organisation works, which is down to people. In any management structure, managers need support, training or development in particular issues. It is difficult for me to comment on specific instances from the past, but it is clear that, in certain areas, individual managers did not pay enough heed to the welfare of their staff or provide supportive and enabling management. I would like to develop that sort of management behaviour in the SQA. Perhaps there was too much emphasis on the importance of status, control and direction.

I deduce from my view of staff capabilities that, in many instances, they would be able to make good decisions on behalf of the SQA if they were empowered to do so. One cannot tell an organisation of 550 people that, from today, things have changed and that they are empowered, even though management's role is to guide and support. However, we are simplifying the processes and constructing a different, more effective management structure—one that operates corporately across the organisation, rather than occupying individual niches within it—which will gradually change a series of behaviours. Those behaviours must change to alter the organisation's culture. We must face the reality that faces a number of public organisations: the culture of an organisation must shift as its market changes. That shift is achievable within the SQA, but it will not be achieved overnight.

Members have not seen an end state; they have not seen the structure that will exist this time next year. Rather, they have seen the first two substantive stages of the design of a planned change management process. That process will be continuous and I expect it to be based on growing improvements.

Although you answered my question, you missed one component. I would like to know whether you have a finger on the costs. What are the cost implications of that process to the SQA? Can you sustain those costs within the organisation or will you need additional public funding?

We are examining that at the moment. I have initiated an exercise to reforecast completely the budgetary position of the SQA. That is not mentioned in current activities, because it is very current. I want to know exactly where we are on the additional costs that were inevitably incurred as a consequence of the events of the summer, relative to any savings that we can generate, and how that compares to our income. In many instances, when one does the calculations in such an exercise, one discovers that, although there may be some short-term cost, that will be an investment, because one generates longer-term savings.

For example, I mentioned that in operations we were strengthening the staff and their capabilities through effective training. That will mean that another 24 people are involved. Some of them will be redeployed from other parts of the organisation as part of a more flexible way of working. However, it is more cost-effective—and therefore better value for money—to have those 24 permanent staff in place than it is to pay endless amounts of staff overtime to plug the gap or to bring in temporary staff who, by definition, are not readily or quickly up to speed. I am not going to pretend that there will not be costs associated with the exercise, but my expectation is that those will be offset by longer-term savings.

I have another question on the structure. Thank you for the report. I see some sense to the way in which things have been moved around, particularly in corporate services. You talked about the culture of the organisation and the time scale that might be needed to bring about changes. Can those changes be brought about before people are on a single site and have a corporate identity?

I believe that they can be. The changes have been designed to deliver two things: the successful certification of 2001—I am sure that we all share the view that that is the clear priority—and a process of change management that will allow the SQA to deliver the promise that that process represents. Much of the capability of the organisation is constrained by processes and structural and behavioural issues, which do not have much place in a modern organisation that is moving forward. I expect that, if we make changes and manage them well, we will fulfil all the aspirations and expectations that people have of us.

What about the time scale?

The time scale will be very quick. By January—by the time we achieve some of the improvements to processes—we will know exactly where we are on verified data, which will be the basis for entering certification 2001. I hope that certification 2001 will happen with an assured group of centres—through which we have the candidates alongside us—which we have had a chance to check. That is not to say that we are waiting for that to happen before we undertake exercises such as the recruitment of markers; we must go on our best judgment. By around January, a lot of the processes will have been simplified, we will be clear about where we are going, we will know what data we have and we will be assured that those data are robust. Structural changes will sit alongside that. If we do more than that, we start to run the risk of disrupting the organisation at a time when that could be dangerous. We have examined the proposed changes and have judged those risks to be ones that must be taken. We are not going too far.

If there are no other questions on structure, we will move on. I hope that it was indicated to you that there would be questions about quality assurance and so on. Ken Macintosh will lead on that.

There are 196 outstanding appeals. Are you now in a position to tell us how quality assurance went this year? Have you reviewed your quality assurance mechanisms?

Are you referring to quality assurance of the marking process?

Of the marking process, in particular. Can you say whether marks were up or down, and so on? Previously you were unable to answer questions on this subject, as you had not gone through the appeals and did not have the figures. There will be some statistical analysis. I assume that you also examine sample papers or interview markers to check your procedures.

I do not think that we are yet in a position to offer a definitive judgment on the pattern of results. There are some early indications, but I am wary about sharing those with the committee, particularly in the light of the experience of the past 24 hours. I would like the opportunity to verify the information. I should not have to do that, but I believe that it is necessary. The broad pattern of results does not appear to differ greatly from previous experience, but I would prefer to reserve my position on that. We will provide the information directly to the committee.

The quality assurance checks that were run in the context of the appeals on marking generally indicate that the administration of marking was very poor. There is no pretence that it was not. However, the marking itself remains robust. That is the conclusion of the principal assessors and the markers who have had the opportunity to revisit these issues.

Given that last year's markers were the same markers as usual, I imagine that the marking would have been of a similar standard. What concerns us is standardisation. Are you sure that markers are marking to the same standard across the board? Have you been able to review the procedures that are supposed to ensure that? Were they found wanting or were they up to scratch?

By and large, the principal assessors ensure, by a process of checks, that equitable standards are maintained across the board by subject. Rhona Wright and Jean Blair may want to add to that.

There is a statistical breakdown that will give us the overall figures. I understand that you do not want to give us information until you are sure of it—although I would like to know when you intend to do that.

Would either of the other witnesses like to add to what Bill Morton said?

I also want to ask about how you ensure standards are maintained. I assume that you check sample papers and individual markers. Those checks should have been done already, so you should be able to say something about them now.

Marker checks were carried out before the marks were finalised. During the appeals process we have carried out more checks, which have shown that marker reliability is at the same level as in previous years.

A specific issue that we identified previously was the use of unqualified and probationary teachers as markers.

I wrote to the committee about that.

Yes, you gave us an update on that. You said that although these mistakes are unfortunate, they have happened before, and that the probationary teachers who served as markers were not of a poor standard.

They were not of a poor standard. We used 11 probationary teachers and another two markers whose background we do not yet know. We are currently investigating that. However, overall the probationary teachers were found to be of a good standard.

What about the fact that, although markers meetings are compulsory, some people did not attend them?

Eighty-five per cent of markers are required to attend a markers meeting. There were 13 absences of markers who were required to attend meetings, seven of which were unauthorised and six of which were authorised. The people involved were briefed subsequently by members of the examining team.

That is from a total population of 7,006 markers.

Why 85 per cent?

There can be various reasons. For example, in the standard grade English portfolio, there are markers who are very experienced and are therefore not required to attend the markers' meeting. That is allowed by prior agreement.

I thought that one of the reasons for markers' meetings is to discuss grading levels and how marking will be done.

In maths, for example, there is a very close set of marking instructions.

So there is no room for subjectivity.

If markers are new or they are changing to a new qualification, they have to attend markers' meetings, but if markers are experienced and the subject is one on which there is a close set of marking instructions, it is by agreement that they do not attend.

Have you reviewed procedures for markers for next year? Apart from maintaining your rigour, it is important to rebuild faith in the SQA. Eleven probationary teachers out of 7,000 markers is not a huge proportion, but it is a worrying factor. Thirteen markers missing meetings, some of them without permission, is not a huge figure, but again it is worrying. What strictures are you introducing so that that does not happen and reports that standards are slipping do not appear?

As part of our project management for appointees, we will examine our procedures. As you say, it is a breach of our procedure for 11 probationary teachers to be used in the marking process and for there to be seven unauthorised absences.

Most worrying of all was the timetable. There was a problem in what you describe as the administrative part of the marking procedure. You had a problem identifying and recruiting markers in enough time. Have you assessed that matter and scheduled enough time for the recruitment of markers? Have you set deadlines that you will keep, rather than let slip, as you did last year?

There has been a limited consultation on the examination timetable. We have moved some subjects that have a lighter population to the start of the diet.

I do not know whether you can answer some questions at this stage if you do not have the figures and have not broken them down. One worry has been the fact that concordancy was dropped this year, for reasons that were explained at a previous meeting. You did not have the history that was needed to establish concordancy. In your analysis so far, has the absence of concordancy been a factor? That is difficult for you to answer if you do not have the final data to show whether there is any variation in the standards. Were you worried about the absence of concordancy? Will you reintroduce it next year?

We plan to run concordancy checks next year. Concordancy took place on standard grades, Scottish certificate of education highers and certificates of sixth year studies. It did not take place on the national qualifications.

It did not take place in the new highers.

Sorry. Will you be able to introduce concordancy for higher still, given that you will have information on only one year?

Building in assurance depends to some extent on having a statistical pattern. The absence of concordancy checks seems to be an area of residual confusion. As Jean Blair says, the checks were run on standard grade, CSYS and the old SCE higher. There were perhaps instances when, again because of the SQA's past inability to deal with some of the estimates that were received from schools, that was not done. Concordancy checks did not run in the new highers, largely because there is no statistical track record. I have checked again this morning and can say that it is intended that full concordancy will be run this year.

One of the reports that we received was that schools were asked for information on concordancy several times before the decision was eventually taken not to run concordancy.

I think that schools were asked to submit information on internal assessment results rather than estimates.

I want to ask about moderation. This relates to internal assessments as well as the exams. In previous testimony we were told that moderation was successful for internal assessments. How much of a factor was moderation as an additional burden on teachers, schools and the SQA?

We know that the passing back and forward to the SQA of information about internal assessments was one of the major problems this year. Was moderation also part of that problem? Was it an additional burden?

I do not think that the moderation exercise was a huge burden on centres. The SQA did not meet its planned moderation targets. We achieved quite a coverage, but no centre was burdened with huge requests in all the subject areas. The problem had more to do with the submission of results. Whether or not a centre was selected for moderation, it still had to furnish us with the results of the internal assessments that were carried out.

We wanted to target our moderation resources where they were most needed, so we used our track record information. We did not select centres for which we had information to say that they were okay in a subject area; we were not going to put them through hoops, just to carry out the moderation exercise. However, those centres still had to furnish us with the internal assessment results.

What would that mean for next year? Was that the case because moderation was carried out on a sample basis?

Yes. Selection for moderation is carried out on a sample basis.

You do not think that that was a factor in the additional burden that teachers and schools have talked about?

It could not have been a huge factor because of the level of moderation activity that was undertaken. With everything else that was going on in centres, selected centres could say that it was yet another factor but, across the piece, the level of moderation could not have caused huge distress.

Given the way in which we are going to restructure the internal assessments and set a deadline for the transfer of information, are there any plans to restructure how moderation is carried out? Was it deemed fairly successful last year? Will it be carried out in the same manner this year?

We are not trying to impose a deadline on the centres, in any absolute sense, for the provision of information. We have consulted on a target date when the majority of the information that is required at the end of the course process can be furnished to the SQA in one data transfer, which could be verifiable by each centre. I know that there are many views on that, and the debate continues.

I would not want the committee to gain the impression that there is any imposed, fixed deadline that acts as a sort of drop-dead date. Apart from anything else, the colleges operate differently from schools; they do not operate on the same annual cycle of the diet.

My second point was—I cannot remember what my second point was.

It was whether the SQA was going to change the moderation for this year.

Yes: whether, because the moderation was deemed a success last year, it would remain the same this year.

The moderation was successful last year to the extent that we managed to carry it out, albeit with the data problems that we experienced. Technically, moderation ought to be triggered by the receipt of entries, and it should be a slick process. The moderation exercise should be carried out reasonably early and should be over and done with before centres know it. Because of our data processing problems, moderation resorted to contingency measures for diet 2000 and the activity was undertaken later in the day. It is therefore difficult to say where we are going to go with moderation for 2001, as we did not put the proper system into place—the selection based on entry data and what have you.

Moderation should not be onerous on centres, and central moderation is not as intrusive as putting visiting moderators in centres. The nature of the evidence that is generated determines the type of moderation that we use. We will ensure that the moderation process is as burden-free as possible for centres.

I would like to pick up a point that Bill Morton made about the submission date. Although it is out to consultation, we would continue to accept unit assessment results beyond 30 April. For all parties, the main advantage in having a target submission date would be the ability to process all data during May and confirm to centres what is missing.

Data are missing either because they should have been submitted or because they were unit assessment results that did not materialise until May. One of the main factors of the summer problems was the missing unit assessment results. This is one of the main checks and balances that it will, in our opinion, be possible to implement. We can draw a line in the sand: from our and the centres' perspective, we know that we can do a quality check to ensure that the data that should be present are present.

We have no desire to make it harder for the centres. We believe that it is in everyone's interests—particularly candidates' interests, and bearing in mind the experience of last year—to have an opportunity to verify the information to the satisfaction of everybody before the actual processes are run. I believe that to be a reasonable and necessary check or balance to introduce.

I had more questions, but I can see that other members are dying to get in.

I was wondering about moderation. This point ties in with the discussion about internal assessment. Is the SQA content that the moderation of internal assessment will be manageable? It will be a massive task if it is done properly.

I worry, for example, about how the moderation of the spoken English element of standard grade English is currently conducted. It is not terribly robust in ensuring equality of standards across the board. If it will be introduced on a large scale because of the significant increase in internal assessment, that leads me to think that a great burden will be created for the authority and a high demand will be placed on teachers to go out of their schools and do the moderation. That demand is generated by all the internal assessment and it is a big problem. If it is not done properly, it is not worth doing; if it cannot be done properly because you cannot afford to do it, we need consider why you are having to do it in the first place.

I think that you are quite right.

Good.

On your point about standard English, it could be argued that moderation is more of a training exercise than an actual moderation exercise. In higher still, more than 85 per cent of the units and courses involved are subject to central moderation, not visiting moderation. The central moderation exercise involves a sample of completed candidate evidence being put in an envelope and submitted to the offices. I hope that the fact that we are seeing the completed candidate evidence, as well as the fact that most evidence generated from the unit lends itself to that type of moderation, will help make the system more robust.

Visiting moderation is much more costly. It is fortunate that not many of the units require visiting moderation. Not many of them are performance based or produce huge artefacts that cannot be transported to the offices. I hope that that gives us scope to undertake successful moderation.

If there are problems with the standard grade spoken English element, would there be similar problems with the spoken element in higher still?

Potentially, there could be, but we do not have evidence on that yet.

I was going to ask about quality assurance but I have one question for Rhona Wright on moderation, as we are on that subject. When we took evidence at our meeting in Hamilton, we heard about head teachers' and principals' experience of moderation. They said that, when they sent information, it was returned to them, apparently unused. Was that common, or was it an exception to the general rule?

That is definitely not common. It is not something that would happen—it probably never happened before diet 2000. That incident probably occurred because of the problems associated with diet 2000.

Ken Macintosh brought up the subject of quality assurance. With regard to quality assurance of the marking, I have not heard adequate explanation of the fact that there were considerably more appeals. We know that some people were appealing for the sake of appealing, or were advised to put in appeals in the hope that they might improve their marks because of the perception of the difficulties and the fact that there might have been a problem with their paper. That aside, there was a large number of appeals. I would have expected that, if the standard had stayed the same, the percentage of successful appeals would have fallen because the volume would suggest that a lot of appeals that would not be successful had been entered. In fact, allowing for an acceptable margin of error, the percentage of successful appeals remained more or less constant. That would suggest that there is a concern about quality. I would like to hear your comments on that.

Not only would one expect, for the reasons that I have given, the proportion of successful appeals to have fallen, but we have been told that the higher still set-up and the introduction of unit assessments should have resulted in the percentage of successful appeals falling, even if the number of appeals had not risen. For those two complementary reasons, I am surprised by the consistency of successful appeals. Can you tell us why such a high number of appeals has been successful when you say that the quality assurance has been maintained?

I am not able to give an explanation in definitive terms. I understand your analysis, which is sound. I have asked some questions in that vein but I have not yet arrived at the final process—we are still in the midst of dealing with the standard grade and intermediate 1 and 2 appeals. We do not have a statistical analysis of the overall picture.

There is a higher volume of appeals this year, for all sorts of reasons. Perhaps that has brought in a component of appeals that has never been seen before and which might be aspirationally motivated.

I am not an expert in this field, and I would want to wait for detailed analysis before I answered your question, but a suggestion that has been put to me is that the internally assessed units represent a sounder base of evidence on which to make judgements on appeals than was possible before.

I offer those suggestions, but I do not have a statistical analysis that would explain what you ask about.

If there are reasons to support the second of your suggestions, that would suggest a welcome improvement in the quality of the exam.

I want to talk about marking. We have heard that people who have been keen to be markers over the years are more cynical now and say that they never want to be markers again. Related to that is how markers are paid. Bill Morton, you have said that you are interested in exploring those issues. What progress have you made? What will the situation be in the coming year?

I do not think that we have been quick enough to thank the markers for what they did in the past year, although that is now in hand. The project manager, Tom Hamilton, who is considering what we will do next year, faces a big task.

There is a lot of understandable resistance and scepticism among markers. Much of that results from the events of this year, which was more than unfortunate for many markers. Gauging by some of the letters that I have received, I think that the situation verged on being a professional affront. Apart from the poor administration, the late date at which contingency plans were established meant that markers had to deal with a greater volume of examination papers than they had expected.

All those matters must be examined. People are also clearly vexed about payment. I understand that, and we will consider the issue. We have explored such issues before and there are normally two sides to the equation. If it costs more to recruit the markers, the SQA must somehow reflect that in its income base and its charges to education authorities. I expect that to be a vexed issue this year, which will have to be resolved in a common-sense and practical way.

Consultation with all the other stakeholders must take place, particularly and first with the Convention of Scottish Local Authorities. We are happy to engage in that. I hope and expect that we will be able to develop a reasonable proposition, because we cannot afford to disenchant or disfranchise the markers more. We need to concentrate on some constructive and urgent remedies. That is why, instead of leaving the outcome to past practice and risking a muddled result, we have identified an accountable project manager who will fix all the issues.

I will pick up another issue that arose from your presentation. You said that the organisation would have to abandon some things. Will you elaborate on that? What will you abandon?

I hoped that you would not ask that, because I stated the principle but I do not have an explicit answer about what we will abandon. The simplest answer is that I would abandon many of our ways of working and the conventions that have become a bit institutionalised. I would probably abandon the unit structure, which does not reflect what the organisation needs to be as it moves forward. I do not rule out examining some of the qualifications in our portfolio. To move with the demands on us, we may have to create space in staff schedules, to introduce brand new and more effective qualifications. That means that some qualifications may fall by the wayside. I cannot give an explicit answer. I simply wanted to show that we are not afraid to consider such matters.

I am interested in your comment because, according to some of the evidence this committee and the Enterprise and Lifelong Learning Committee have heard, so much was happening at the SQA that, at times, it took its eye off the ball and did not deliver the core work. Would abandoning some things mean that the emphasis could be placed on the core work?

The one thing that we will not do is jeopardise the core business, because that is based on what people need from and expect of us. That is how our performance will be judged. It would be illogical to suggest that we would focus on that and not impair some of our activities.

I also said that the SQA still delivers well on a wide range of its activities, despite recent problems. That is the truth. We can build on elements of that performance. A better way of expressing my point may be to say that we need to ensure that we understand first principles, get back to them and ensure that they are right. Then, we can consider the rest of the issues. In realigning the organisation, we must ensure that the resources are targeted correctly. Staff may be undertaking some tasks that are not contributing effectively enough to our core work.

We have covered the major issues, but I know that members have some miscellaneous questions that are designed to tie up loose ends. I will go round the table again to ensure that everybody is clear on where we go from here.

I have three points.

I am counting.

I am sorry.

I want to return to your point about there being not a deadline, but a target for the transfer of information about unit assessments. The committee may take its own view on unit assessments and data. However, I would be concerned if you were to say that it does not matter if people fail to meet the target. If rigour is not applied, nobody will meet the target and we will end up in the same situation as last year. I hope that you are aware of that.

That is not my objective. The objective is for everybody to invest in ensuring that the candidates' interests are best protected. I am not saying that we will do what we did previously, when centres had incrementally to provide the information to the SQA as units were completed. This year, when courses are completed—by and large around a consistent window in the year—information will be transmitted once to the SQA. I do not want to create the impression that we are saying that we are striving to do that but do not mean it, and that the information can come in at any time. I do not want to impose additional burdens on centres, but I am sure they will understand that what we are proposing is a simpler way of submitting information that builds in checks and balances to prevent a recurrence of the problems that we had last year.

Earlier we accepted that point fully.

You were not able to comment on the picture for this year, but you hinted that it was similar to that for previous years. Obviously, that is a great reassurance to us all, as there are many unanswered questions about quality assurance, how we should value these exams and how meaningful they are to pupils. However, you did not say when we would get the full statistical picture. Will we have to wait until all the standard grades are awarded, which will be around Christmas, or will you be able to do an assessment once all the highers are out of the way?

Rhona Wright talked about the burden that moderation imposes on the SQA. You said that there was a sample burden in schools that was not especially onerous, but I am concerned that moderation is a very heavy burden on the SQA that gets in the way of its other functions. If, for example, concordancy—which this year was not carried out for higher still—were introduced, that would place further burdens on the SQA, which some people have described as already overburdened.

However, I agree with what you and Ian Jenkins said about moderation being necessary. Tests must be meaningful, or they are worthless to pupils. I bring to your attention comments that were made about the core awards by a young woman from Stirling who gave evidence to the committee. She said that to her the core awards were meaningless because they did not reflect her abilities. She had studied three languages to higher level, but had no qualification for communication, which was ludicrous. Having core awards and trying to give pupils an assessment of their abilities is a good idea, but if the awards are meaningless, and moderation, quality assurance and standards are lacking, the exercise is worthless. I would like you to comment on that.

Rhona Wright will answer on moderation and Jean Blair will answer on core skills.

Moderation is a burden on the SQA. It should not be an onerous burden, because it is part and parcel of what we are about. Moderation is about undertaking quality assurance to ensure that national standards are being maintained within centres. I hope that it would never be viewed as a burden. If we get a robust set-up for our data and management, moderation should be less of a burden, because then we will have the information that we require to carry it out accurately and as planned.

Concordancy would help moderation because it is a criterion for selection or non-selection, whichever way we choose to play it. If we have a centre where the external and internal track record is good, we could use that to say whether a centre should be targeted. I hope that, everything being well, things can evolve in time and we can lighten the touch so that the burden is lighter than it seems to be at the moment.

The five core skills are made up of various components, which can be awarded at the same level as they had been for national qualifications—higher, intermediate 1 and 2, and access 1, 2 and 3. There are two routes to core skills certification. One is by assessment of core skills as discrete core skills units. For a communication higher, for example, the components would be awarded at higher level. Alternatively, what we call a carrier subject can be taken. For example, drama might include working with others and communication as core skills that are embedded into that subject. Those core skills would therefore be awarded automatically to the candidate.

This year, for the first time, we have introduced a core skills profile, which records core skills that have been awarded as discrete units or as embedded components within subjects. Victoria MacDuff's qualifications portfolio must contain some subjects that carry components of core skills, which will not necessarily be at the same level as the subjects that she sat. It could be the case that she sat a subject at higher but that the component subject is at intermediate 1 or 2. I understand that a public relations exercise must be carried out to explain to schools and to pupils the merits of having a core skills profile.

We understand that the core skills awards are generated automatically by the subjects that pupils study. What concerns us is the fact that, if there is no rigour about the awards, they are worthless to Victoria MacDuff or to any other pupil. If she gets a B in communication, generated by an exam that she sat but which is not standardised, that does not mean anything. It is not just a question of having a public relations exercise to explain it.

We will want to pursue that issue, but I am not sure that we should do so at this moment.

I also want to ask about the date for statistics.

That is a post-clear-up, pre-appeal matter and I hope that that information will be available within the next few weeks. It must be put back into our main computer system prior to recertification, where appropriate, at the end of November. That will obviously be subject to rigorous scrutiny. As soon as those data are available, we will let you know.

All I am concerned about is that we get them before our report. It is crucial that we have that information before we finalise our report. At the very least, we will need some sort of information. Without those data, there will be key areas of our inquiry on which we will not be able to comment accurately.

We shall provide you with those data at the earliest possible opportunity.

Thank you.

I have two questions, the first of which is about unit assessment. We have heard your explanation of the changing approach and structure and how you intend to deliver the 2001 diet without the problems that we had in the 2000 diet. Obviously, there is a commitment to bring in the unit assessments and consultation is taking place about the timing of that. That is all about practicality and, as we have heard from a number of witnesses, the SQA is a can-do organisation. You have shown us that, under your management, you intend to make it a can-do-and-deliver organisation.

Considering the exam chaos with hindsight, do you have any view, as managers, of the philosophy, rather than the practicality, of maintaining the current unit assessment regime? Would you suggest any changes? Changes are suggested constantly that would mean that you would not have to meet some of the time scales that you are setting yourselves. Would you suggest any philosophical amendments?

That is a question where I really will have to hide behind the fact that I am not an educationalist, which is the answer that members may have grown to expect. I would answer the question slightly differently from the way in which it was posed. I do not mean to avoid the question, because it is a big policy issue, and I would be happy to be guided by the view that the committee and others reach. If we can simplify the transaction of the data, in so far as it remains valid and required in order to certificate completely the awards to candidates, our responsibility is a practical and pragmatic one: it is to ensure that we understand our responsibility, that we have planned and prepared for it, that we communicate, that we have assessed all the risks and that we have contingency plans. "Can-do organisation" is not a phrase that I coined. I like to think that we are a will-do organisation.

My next question concerns communications. We asked for copies of press statements issued by the SQA. I notice that we seem to have received only those that go back until 7 August, although we had asked for statements going back to June. I do not know whether none was issued before then or whether we have not been sent them. Nevertheless, that does not change the matter that I want to raise.

I appreciate that Mr Morton was not in his role when the press statement of 7 August was issued, but I think that it will be helpful to him. That statement makes no mention of the problems that were about to be faced. The press statements issued on 9 August, of which there were two, still do not mention any of the expected problems. They talk about standards being maintained, announcements coming out and the countdown to the exams results being issued. Not until 11 August is there a press statement that makes an announcement about the helpdesk being available to help with the problem of incomplete information on certificates. On 12 August, a short statement was issued, saying that there should be no challenge to integrity. There is an undated statement, which I presume is from either 12 or 13 August, about Ron Tuck's departure. Only on 13 August is there a joint statement from the SQA, the Committee of Scottish Higher Education Principals and the Scottish Executive about validations being about to be run following a meeting.

I wanted to read all that out because if one examines the press coverage in the week up until the Sunday before the issuing of the certificates, and even right up until 13 August, it is clear that SQA spokesmen were giving information—officially, not as unnamed sources. For example, the Daily Mail of 8 August reported that about 400 candidates would be disappointed. We know that the SQA was saying to the Executive the next day—so I presume that it already knew the figure on 8 August—that the figure would be 1,500. That suggests that the SQA had been communicating using the telephone—giving off-the-record briefings to journalists rather than making official statements—and that attention had not been given to providing full and transparent information. Given yesterday's experience, do you agree that the usual way for any public body to impart information in future should be to give as much information as honestly as possible, and that when there is a crisis, bodies should try even harder to be proactive and to help by giving information in advance, so that the perception of the problem can be played down? It is clear that that did not happen.

I agree.

I commend answers that are shorter than the question. Given what has happened in the past 24 hours and the huge public interest in the matter, what is your assessment for the next 12 months? Will the same thing happen again? What will take place? Can you reassure parents, pupils and the committee that diet 2001 will not be affected by the same problems? Each of you comes to the situation from a different perspective, so I would like each of you to answer.

Whom would you like to go first?

Rhona Wright?

I give the assurance that every effort is being made, by everyone in the SQA, to ensure that it does not happen again.

I agree with Rhona. Clearly there are things that we still need to do, and our approach to planning for next year is to take each step as it comes and not run before we can walk. We will plan properly and we will assess risks adequately. Assuming that that assessment is undertaken with the rigour that I intend to apply to it, I have every confidence that we will deliver successfully next year.

I am optimistic that the situation will not recur.

Internal communications in the SQA will have to be improved. If concerns are listened to at the right time, we should be on track next year.

Ian, do you have a final, very short question?

Can schools feel reassured that there will not be shifting sands—that you are working with them? Obviously, you want to be flexible, but can you reassure schools that you will not change things under their feet so that they feel uncertain, and that they will know what you need from them and when they must send it? For this year, we should try to keep the system simple and secure.

You said that you might listen to what the committee recommended for the future. I am worried about the committee recommending things as though we know better than the teachers and everybody else. What would be the process of change in the longer term?

Finally, after the fiasco and all the discussions that we have had, is it not a shame that we are talking about the fifth and sixth years of youngsters' schooling as if they were about only testing? Should we not be talking about education?

Yes, I agree, and I acknowledge the SQA's responsibility for skewing your agenda.

Keep it simple—those are the watchwords, and that is what we are trying to do. We are making a genuine effort to be much more responsive, by listening. That is why, although some centres were concerned that we seemed to be late in giving out information, we have tried to get the process right rather than rush things. The guidance is now with the centres, and we have introduced a simplified way in which to get the registrations data. We have introduced the principle of getting confirmation back to the centre, to give the centre—whether it is a school or college—the reassurance that it needs that the information we hold on its behalf is complete and accurate. The checks and balances are being introduced right from the beginning.

I have said openly, and emphasised to the staff in the organisation, that as this is the SQA's problem, we must have the room to fix it. We need to move quickly—not with haste, but with a necessary pace, given the urgency, scale and complexity of what we need to do. I do not pretend that there is an easy fix that can be taken off a shelf and simply applied, especially as some of the issues are cultural. I acknowledge, freely and openly, that a great many people have a view on the situation and a right to express it, but the responsibility to put in place the necessary improvements and changes is ours. I am not asking for anybody's indulgence, but the SQA must be given the right to fix the system.

Thanks very much. Today may be the last time that we see you in the context of this inquiry. I am sure that I speak for the whole committee when I say that we wish you well and hope that things work out as they should.

Thanks very much.

That concludes our questioning, you will be glad to hear. The committee invited you and your team back to speak to us because we wanted to see your presentation. We thank you for that, as it has cleared up some outstanding questions. It is unfortunate that you returned on a day on which there has been further disappointment for students throughout Scotland. I am sure that you are aware that it will take extra work from you to reassure people that you are heading in the right direction.

We needed reassurance that changes were taking place, as we do not want to be in the same situation next year. You have shown us that some changes are taking place, but—in the light of this morning's news—there is still room for improvement. We acknowledge that you will provide us with a timetable of events for the coming 12 months, and the committee will take great interest in that on behalf of all those who will sit exams in 2001. I thank you again for coming this morning and for the information that you have given us.