Official Report 229KB pdf
Before we begin our formal business, I want to raise an issue that has come up since the committee started to take evidence at last week's meeting. It relates to further clarification of a question that I asked the civil service witnesses.
I note Mr Ewing's point. There may be merit in what he says, as there may be omissions in the evidence that we have taken. I am also conscious that we have been deluged with a huge volume of information to examine.
I am happy to do that, if it is the will of the committee.
Do committee members have views on the point raised by Mr Ewing? Would it be acceptable for the clerk to write to the department, given that it would be for the department to consider whether it could release further information?
Would not it be more appropriate for the convener of the Enterprise and Lifelong Learning Committee to raise the issue with the convener of the Education, Culture and Sport Committee, given that both committees are involved in similar inquiries? That would prevent one committee from going off at a tangent.
In fairness, the Education, Culture and Sport Committee will not take evidence from the department of enterprise and lifelong learning or from the three witnesses from whom we took evidence.
That is the case, as far as I am aware.
Is it acceptable for a letter to be sent to the department seeking further information, which is within the province of the department to determine whether it can release?
On occasion, the Education, Culture and Sport Committee will be able to take evidence from civil servants. We are talking about a code that applies across the board, irrespective of department. I would prefer there to be liaison between the two committees before our committee goes off at a tangent in relation to the matter raised by Fergus Ewing, which also has an impact on the work of the Education, Culture and Sport Committee.
In fairness to Mr Ewing, his point refers specifically to the department of enterprise and lifelong learning and to the three witnesses from whom we have taken evidence. Is not that the case, Mr Ewing?
The point also applies to the Education, Culture and Sport Committee. However, this committee is concerned with the activities of the enterprise and lifelong learning department. If the suggestion is that the requests for information in the letter that is to be sent by the convener, on behalf of this committee, should be restricted to advice that was given to the Minister for Enterprise and Lifelong Learning, that seems entirely proper. The Education, Culture and Sport Committee could, even before taking evidence, write a similar letter—if so advised—to the Minister for Children and Education.
Mr Ewing has made a request that we ask the enterprise and lifelong learning department for the further information that he seeks. Does the committee accept that request?
By way of an amendment to Mr Ewing's proposal, I suggest that the convener of this committee and the convener of the Education, Culture and Sport Committee should liaise with each other before any request is made for additional evidence.
Would it be acceptable for a copy of the letter that I write, as acting convener, to be copied to the convener of the Education, Culture and Sport Committee?
No. I am suggesting that you should liaise with the convener of the Education, Culture and Sport Committee in advance of any request for additional information from any civil servant in relation to this inquiry. That is fairly standard procedure.
Equally, it could be argued that this committee should be free-standing and should act independently in relation to the remit that it is being asked to discharge. This committee has no competence to control what the Education, Culture and Sport Committee does; likewise, that committee has no competence to control what this committee does.
Convener, I propose that a letter be sent to the enterprise and lifelong learning department.
As no one else seems prepared to second Mr Ewing's proposal, and in the interest of assisting democracy, I use my casting vote to second the proposal. Is there any other proposal to put to the committee?
One of the members of the Education, Culture and Sport Committee was indicating that she wanted to comment.
I am sorry. Mrs Peattie, would you like to speak?
I realise the importance of this to the committee, as a similar issue will arise for the Education, Culture and Sport Committee. Getting the information is a part of the parliamentary process and should not be dependent on a whim of the minister. It might be helpful for the convener of this committee to liaise with the convener of the Education, Culture and Sport Committee to ensure continuity on this issue.
I have listened to what Mrs Peattie has said. Mr Ewing, do you still wish to press your proposal?
It is a matter of principle. This inquiry must have all relevant information before it, and we do not have that information. That is wrong in principle. I therefore adhere to my proposal that a letter—along the lines that you have described, convener—be sent to the Minister for Enterprise and Lifelong Learning, urging the Executive to reconsider the decision that was intimated by Mr Galbraith last week at question time to withhold internal advice from both committees' inquiries.
I move, as an amendment to that proposal, that, prior to any letter being issued, the conveners of the two committees that are conducting the inquiries should liaise with each other.
Is there a seconder for Mr Wilson's amendment?
Yes.
Can I have a show of hands on the amendment?
For
The result of the division is: For 5, Against 2, Abstentions 0. The amendment is therefore agreed to.
For
The result of the division is: For 5, Against 2, Abstentions 0. Therefore, the proposal, as amended, is agreed to.
I understand that the committee has now taken the decision that the conveners of this committee and the Education, Culture and Sport Committee will liaise with each other. As we are to resume this inquiry on Friday, I request that the liaison be concluded before that meeting, so that, if appropriate, this issue can be revisited on Friday morning. That gives 48 hours for the two individuals to have a chat, which would presumably be sufficient time.
I hear what Mr Ewing is saying. As deputy convener and acting convener for this meeting, I will try to ensure that those wishes are met. However, as the committee will understand, we are unclear about who will be convening Friday's meeting. Therefore, I am in an impasse in terms of determining who is responsible for doing what. You will have to leave it with me to consult my successor or to determine who will be convening Friday's meeting. We will try to ensure that contact is made with the convener of the Education, Culture and Sport Committee.
No, I do not think so. I used to be the chairman of the Scottish Vocational Education Council—that is now quite a long time ago. The arrangement was that we saw the minister once a year, after which the minister would write a letter of guidance to us on what he expected us to do in the following 12 months.
I notice that one of the refinements that you suggest in your submission is for six-monthly meetings with ministers, at which plans and targets could be agreed. Was it the intention that attendance at that meeting should be restricted to the chairman of the board and ministers, or was it anticipated that other members of the board would be included?
Usually it has been the chairman and the chief executive. That should continue, unless the minister indicates otherwise. That is appropriate.
Thank you. At the moment, liaison meetings are held between members of the department of enterprise and lifelong learning and the chief executive and some members of staff of the SQA. Did the minutes of those meetings go to board members?
No, they did not. On every occasion, there was a chief executive's report on what had taken place during the meetings, but we did not see the minutes.
Does that explain why one of the recommendations in your submission is that there should be
Yes. If members can stand one more minute of history, I will explain. When the SEB and SCOTVEC were wound up, there was a hiatus, as neither chief executive was going to continue. We at the SEB and SCOTVEC decided that we should take the initiative and try to describe the kind of organisation that we would like the SQA to be. We had regular meetings with the Scottish Office, which I attended on behalf of the incipient SQA, to talk about the investment that would be necessary in computers and so on. When the chief executive was appointed, I withdrew.
I am grateful. The other broad question that I want to ask is perhaps more appropriately addressed to the members of the board. I want to identify what they consider to be the role of members of the board.
The nature of our role was defined by the former Secretary of State for Scotland in our letters of appointment. It was to set out the SQA's strategy and objectives, to approve its corporate plan and estimates, to make decisions on major issues of policy and to review the authority's performance. It was very much about vision and ensuring that the necessary framework was in place.
Is your view similar, Mr Minto?
It is. However, I have an additional function on the board, in that I convene the finance committee. I see my main role on the board as being to monitor the authority's finances, to discuss budgets with the officers and to report to the board on the financial information that the officers present.
I was invited to sit on the board because of my background in the Scottish School Board Association. I see the position of non-executive board members as a safeguard of policy, more than anything else. We bring with us the concerns of the constituency with which we are familiar—in my case, the concerns of parents. Those have been brought to the attention of the board and the officers. Basically, our role is to ask questions and to find out information, to ensure that we safeguard the service.
I open the meeting to questions from members.
I want to examine the information that was available to you about the awards processing system, a subject that was brought up at your board meetings on a number of occasions. Of the meeting on 23 September last year, you say:
Yes it did—that and a number of other comments that were made prompted us to ask questions. Members will see—from the minutes of one of the finance and general purposes committee meetings in June—that we asked David Elliot to come to that committee and that we cross-questioned him closely and offered to make available additional resources. Contrary to popular opinion, the SQA is not bankrupt and could have provided more resources. We were assured that the situation would be all right.
Would you say, therefore, that you felt reassured by the various statements that were made to you about the—
I felt reassured about the level of missing data. By that stage, we had established that the missing data were the problem, but that the results would be publishable on the given date, although the data were not perfect.
What would you have done had you not been reassured of that? What course of action would have been open to you?
That is part of the difficulty. The question is perfectly logical, but it is difficult to answer because the time scales were—as they are in general as far as the SQA is concerned—extremely short. I also had in mind—I suspect that the other board members did, too—that intervention at a very late stage would be a complicating, rather than helpful, factor.
I wish to turn to your written evidence to the committee, specifically the evidence on governance. You conclude that issues of governance did not contribute to the current difficulties, but that governance worked well. You go on to ask whether there is a better model for the governance of the Scottish Qualifications Authority—the primary matter that we aim to determine. You conclude that, at the time of the establishment of the SQA, there was cross-party support for its establishment as a non-departmental public body, rather than an agency or arm of Government. You also conclude that there was wide support in education and training circles for that. Does support still exist in those circles for the SQA's continuation as an NDPB?
We should consider the level of board input, the contribution of the people at the SQA board table and the areas from which those people come. They are not representatives—they are all on the board as individuals. Iain McMillan, for example, is there as an individual, not because he is a member of the Confederation of British Industry. I believe strongly that the contribution of the board members—whether they are from schools, colleges, universities, industry or whatever—would be required by whatever body ran the system.
We have heard from officials in the enterprise and lifelong learning department about the "Management Statement and Financial Memorandum", which seeks in several respects to amplify statutory provision relating to the governance of the authority. You refer to that in your submission.
On the first question, if I felt that there were problems of which I was aware in detail, of course I would have gone to the minister. The SQA executive has regular contact with the Scottish Executive and it would be quite improper for me to hide from the minister any problem of which I was aware.
To clarify Mr Wilson's point, paragraph 6.3 of the "Management Statement and Financial Memorandum" states:
Every December we produce a corporate plan—I presume members have a copy of that—which contains our mission, corporate goals, key objectives and so on. It is created by the executive and approved by the board—not without comment and amendment. It goes then to the Scottish Executive for approval. We review the SQA's performance against that document, as per paragraph 6.3 of the "Management Statement and Financial Memorandum". When we
I accept that. The board of management did its best with the ingredients that were available. Given that, if a significant problem—about which the board was completely unaware—arose in a substratum, should some further safeguard be considered to try to avoid such a situation being repeated?
With hindsight, that must be the case. In our submission, we suggest that the attendance at board meetings of senior representatives of the Scottish Executive would be helpful and that we ought to institute a committee that consists of the vice-chairman, the chief executive and me. That committee would examine specifically the detail of the operational side of the business. It would meet monthly and report to the finance and general purposes committee and the board of the SQA. I regret that we did not have something as straightforward as that in place—I believe it would have solved the problem.
When the chief executive and the director of awards reassured the board and the finance, planning and general purposes committee on the points on which they were being probed, we believed that they were reassuring us according to their certain knowledge. Since then, it has transpired—the former chief executive alludes to this in his evidence—that the board was being reassured on the basis of best of knowledge, and not certain knowledge.
In your written evidence, you refer to a proposition to recommend the appointment of assessors, as happened with the SEB and SCOTVEC. However you referred to advisers in your oral evidence.
I am sorry. Assessors is the right word. I am sometimes all at sea with such expressions. I mean assessors on the board, who will give prompts, rather than advice.
I want to address the issue of where the breakdown occurred. You say that it occurred below chief executive level, at the level of the officers who reported to the board.
I do not think so. I understand what George Lyon says, and with hindsight we might have done things differently. However, as far as the board was concerned, the staff who dealt with the examinations were the same staff who had dealt immaculately and in a world-class way with previous examinations. Hordes of people from abroad were beating a path to our door to get our assistance in developing their programmes. We were reassured intellectually by the fact that the people who were involved this year were those who had been involved on all previous occasions, which were hugely successful.
Ann Hill said that your role was to ask questions and find answers, but clearly you asked the questions and got the wrong answers—you were not being told what was going on inside the organisation.
I will address your first question first. When somebody is concerned about something, they seek reassurances. We sought reassurances, which we were given. I do not know whether my colleagues have anything to add to that.
I can speak only for myself. No one in the organisation approached me to tell me that problems were expected.
No one approached me, either.
People expressed concern. The chief executive expressed concern. However—and I believe that the chief executive is beyond criticism in the matter—the requisite assurances were given.
I take it from that, that at no time during the past 12 months did the board discuss the possibility of reporting back to the minister before the final stage of the process, to say that there was a serious problem and to express reservations about what the board was being told by senior management.
No. The assurances that the board was given implied that the situation was under control, so we did not go to the minister.
I will try to keep my questions brief. Elaine Murray asked about the APS. In your submission you say that one of the root causes of the problem was a failure
The assurance that we received was given correctly, because the software worked. It is not unusual for software to be written late in the development of a programme. One of the difficulties—which we thought was a software problem—turned out to be a data management problem. The assurances that we were given about the software, late though its development was, were absolutely correct. The computer system did what was asked of it.
I want to return to your corporate plan, because the process of agreeing that plan is vital. Did you feel that the agreement that you had made with the Executive was appropriate?
I am sorry—I missed that.
Did you feel that the process by which you agreed your corporate plan with the Executive was appropriate? Were you happy with it?
Yes. Detailed work was done internally on the plan before it reached the finance and general purposes committee and the board. It was not plucked out of the air; it was the result of careful consideration. The board provided input to the plan and it then went to the Scottish Executive for approval. It was, in due course, approved.
You have said that you do not, as a matter of course, get the notes of the meetings between the Executive and the SQA liaison group. Were you aware of concerns about difficulties that were expressed by outside bodies?
We were aware that such bodies were telling us—especially in relation to the registration of students—that there were considerable strains on schools, which we caused. As a result of lateness, a knock-on effect ran through the system. We were aware of that. That is why we asked so many questions.
Did the answers to those questions satisfy you that everything was under control?
Yes. I take no satisfaction from it, but I sat in front of the press on the day that we were supposed to put the envelopes in the post. The figures that I used then were, as far as I was concerned, a continuum of the figures that we received over a period; that 1,500 candidates and 0.5 per cent of courses were affected. Those figures were given to us in good faith. We dealt with the matter based on the figures being the logical extension of those that had come out during the previous days and weeks. When I went to Dalkeith the next morning, the figures turned out to be nonsense—unposted certificates were still lying in the exam room.
Concerns were expressed to me in mid-May about registration in the further education sector. I asked for a report on the awards processing system to be put on the agenda of the finance committee, which was done on 1 June. We got an extremely full report from the director of awards. We were given assurances that contingency plans had been prepared for every eventuality. We accepted that things were in order and that problems were being solved. That was subsequently reported to the board on 22 June. Again, the board was reassured that problems were being solved.
I would not like any of what we are saying to be taken as excusive. My colleagues and I have spent an enormous amount of time since the event on trains, aeroplanes or wherever we are, asking ourselves what we could have done to prevent the events happening. The changes that we suggested are things that we want to put in place and might have put in place sooner, had we known that this was going to happen.
I have listened to that clarification and I thank Mr Miller for it.
We have one, on which I will get clarification from my official.
Would it be fair to conclude that the whistle was not being blown very loudly? Certainly, it was not being blown in an audible sense, so that people on the board or elsewhere could hear it.
I am not being facetious, convener, but while one can have a whistle-blowing policy, I do not think that one can be responsible for making people blow the whistle: if people choose not to do so, they will not do so.
Rather than people who work for the SQA being encouraged to whistle-blow, which suggests involvement in an activity that may endanger career prospects, should they be encouraged to offer positive criticism and comment?
Yes.
I want to ask about the root of the problem, according to Mr Tuck, whose written submission says:
In some cases, information sent from schools was not entered: it existed within the organisation and had either been lost or had simply not been entered. It seems extraordinary to me that an organisation that had managed data so efficiently before got into such a mess without anyone saying anything to us on this occasion.
I spoke to people in schools—
May I intervene, Mr Ewing, to remind you of the terms of our remit. I think that we are straying into matters that are properly the concern of the Education, Culture and Sport Committee's inquiry.
Your interruptions are always extremely courteous, convener.
Mr Ewing, if your questions on the governance of the SQA concern the board or how it operates, it is entirely proper that you should put those questions. However, I would like to know what your questions are.
Must I submit my questions in advance and in writing?
No, you may articulate them orally. I will let you know whether I approve of them.
It seems to me that the problem lay in part with the SQA board, which—I believe—is composed of between 15 and 24 members. More of the board's members should be directly involved at the chalkface. They should be people who know about the problems that schools were having with inputting information and about the difficulties that occurred throughout the year with obtaining parts of the higher still course.
We have the president of the Educational Institute of Scotland, who is a teacher in St Andrews. We have two head teachers, one of whom became a head teacher after we had appointed him to the board. I am not being critical of him when I say that we thought we had recruited a teacher, but in the interval he became a head teacher. We have a good deal of school-based input. We have a member of the Association of Directors of Education in Scotland. We have Ann Hill. The board is a lifelong learning body that covers school education, further education, some universities and industry. It has to be composed of people of the sort that I have described. The representation on the board is good, bearing in mind the constituencies that we serve.
I want to move on and refer to paragraph 19 of the minutes of the meeting on 8 February between the SQA and the Scottish Executive. The paragraph, which is headed "ICT Funding", may or may not go to the root of the problem that we are trying to solve. It states:
Before Mr Miller replies, would Mr Ewing say again to which meeting he is referring, and to which paragraph? Is it the liaison group meeting of 8 February?
Yes. Paragraph 19 is on page 5.
The point that we were raising—and it was raised with the minister—has nothing to do with the current operations of the SQA. If we want a system that will respond instantly to assessment on demand, a different range of things has to be available to achieve that. We need a sufficient number of national assessment banks. You will have to ask Ron Tuck about that, as he will have much more information than I do.
You stated that you warned ministers about data management problems earlier on. Can you recall specifically when those warnings were first issued to ministers, either directly or through civil servants?
I honestly cannot say when warnings were issued through civil servants, but I met the minister on 25 July, quite close to the production of the exam results. We had a discussion about whether the results should be delayed, and I am glad that we did not delay them, as we would not have been any better off in a week's time than we were then.
We know that the non-executive board members of the SQA, including those who are here today, are distinguished in other walks of life—in chartered accountancy and as director of the Confederation of British Industry Scotland, for example. There are also many other distinguished board members, including Malcolm Green, who is not here today. All of you are busy people, engaged, in most cases, in full-time employment or activity in other important walks of life. Is the best way to run an important body such as the SQA to have a series of non-executive members, for all of whom membership is bound to be, at best, part-time?
Yes. If you want around the board table the kind of expertise that can make a sensible contribution to the development of an organisation, you must have people with that experience. I have no doubt that all of us are involved in a number of things, but I do not think that any of us are so busy that we cannot put our minds to the SQA. I invite Iain McMillan to comment.
That was the first time, to my recollection, that Mr Ewing has ever described me as distinguished, and I welcome that. To answer his question, I think that David Miller is absolutely right. The model normally works well. Members of the board come from various walks of life, bring skills and insights to the board and, in the main, give a result that is greater than the sum of its parts.
I have another question.
We are slightly pushed for time, Mr Ewing, and I am anxious that other members should participate. Is it a brief question?
Yes.
The answer, I think, is yes.
Can you recall who that was?
I hesitate to answer, as it involves a choice, but we have a banker and a chartered accountant, to say nothing of my vice-chairman. Those people are involved with data management and are among those who expressed concern.
I want to comment on two of Mr Ewing's points. I have some sympathy with his request that we have more teachers on the board. We need to have more dialogue with teachers, because they can give us the necessary expertise. It may not have solved the problem if everyone had listened to teachers earlier, but we might have been able to cope with it better.
I want to return to the issue of risk assessment. Ron Tuck's submission states:
I cannot say more than that we asked a lot of questions and that we got reassurances.
Was the risk assessment internal or external?
It was an internal assessment. You would have to ask Mr Tuck, but I am pretty sure that it was only an internal risk assessment.
Perhaps I could add to that. My understanding is that the risk assessment was carried out by the executive directors with the chairman of the audit committee and that they were greatly assisted by internal auditors. Those matters were reported to the board and, as far as we were concerned, the assessment had been carried out very professionally.
A lot of the questions that I wanted to ask concerned areas that have been explored. We are trying to ascertain whether the governance of the SQA needs to be improved to enable it to operate better. That is where all our questions are coming from. Given that a lot of new measures have been implemented in the past year—higher still going live for the first time, a new IT system, the SQA having to manage all the data, the new certificates—and given the history in the public sector of IT disasters of one sort or another, did you recognise that you were in a high-risk environment? Are you sure that everything was done that needed to be done to assess the level of risk? If it was recognised that you were in a high-risk situation, what extra steps were taken to address the risks?
I repeat that, amazingly, the problem was not an IT failure, but the management of data. As you know, the software produced more than 90 per cent of the results correctly; it did what was asked of it. It really was a data management problem. If you listened to and read reports you would hardly believe that the figure was 90 per cent. That figure is not nearly good enough—I am not excusing myself—but you would hardly believe that we had a result that was as high as 90 per cent. The software worked remarkably well. I take my hat off to the two computer people who were involved in the development of the system, who were working very hard against the clock.
You said that data management was the source of the problem. Are you saying that the overall project management was inadequate because it had allowed the data management to go wrong? It has been said today that you were assured that the data transfer from the schools and further education colleges would be all right in the end. Was that proved? Was user acceptance testing or systems testing used to prove that the data flow was adequate and that the communications worked, as well as the software?
I was aware—and I think the board was aware, because this is when the questions started to be asked—that there was an incompatibility between some of the schools' systems, the further education establishments' systems and our systems and that we were chasing our tails from registration onwards.
When you became aware of that, was it too late for you to do anything?
The problem started at the point of registration, which happens again this year in three or four weeks' time, I think. We were aware that registration had been delayed and was causing difficulties. When we visited schools, that issue was raised with us. The registration was completed eventually, but not in the way that one would have liked, especially with regard to the data transfer arrangements.
I want to explore further the governance role. I have heard about non-executive members and the problems that people have felt they have with operational issues.
The first answer would be that hindsight is a wonderful thing. Meetings were being held all the time between our executive and the Scottish Executive. We heard reports of those meetings at each board meeting, but we did not see the minutes. We felt that a lot of the operational debate was taking place between the Scottish Executive and our executive. Now that we have seen the minutes, we can see that that was not the case. The discussions were much more general than we thought and, where they were specific, they dealt with the introduction of higher still.
I was involved with SCOTVEC for the whole of its existence. At that time, we had assessors from the Scottish Office. I, and the board of SCOTVEC, found that helpful and I am sure that the Scottish Office found it helpful as well. That is why we are making our suggestion. We think that it will work and that it will help.
I will try to keep within your rules on governance, convener.
Try also to keep it brief.
In respect of the governance of the SQA, if the problems had been not data management but marking, would the board have been aware of that?
We were made aware that there was a shortage of markers and that that was because of the fact that some registrations were not complete. Again, we were given statistics about the number of papers that needed markers and how those numbers were falling all the time. We were assured—as was the case, I think—that the marker situation had been solved, although it had been solved late. I think that 88 per cent of this year's markers had been markers last year.
There has been much talk of the different cultures in the old SCOTVEC and the SEB. Given that David Miller was involved with SCOTVEC, can he tell me whether elements of the culture of governance in those organisations came into the SQA? If so, what were they? In the governance of the SQA, did one cultural view of how to govern an organisation dominate the other?
I was also chairman of the SEB, by the way.
Is that physical dichotomy still a difficulty for the good governance of the SQA?
I shall be careful in answering that question. It would be more advantageous for everybody to be in a single building. The problems of getting there would be huge, in terms of cost and of losing the valuable input of a vast number of people such as those who move the paper every year, who live mostly in Dalkeith. We would lose a huge core of knowledge if we said, "Let's go to Stirling" or wherever. However, if we came down from Mars, we would undoubtedly choose to have one building.
On behalf of the committee, I thank you for attending this afternoon and answering our questions. Are there any concluding remarks that you would like to make?
You have been very courteous in giving us a hearing. Thank you very much.
Our next witness is Mr Ron Tuck. Good afternoon, Mr Tuck. On behalf of the committee, I welcome you and thank you for attending. I thank you also for making your submission available to the committee. It has been most helpful. You have listened to my earlier comments about the remit of this inquiry; it is concerned solely with the governance of the SQA. That is the issue towards which we shall direct our thoughts and questions.
I was responsible for leading in all operational management matters and, in conjunction with senior colleagues, for proposing a strategy. However, responsibility for the approval of that strategy rested with the board of management. That happened through our corporate planning process. A fairly detailed corporate plan was put to the board for approval and was subsequently submitted to the Scottish Executive for ministerial approval. The additional line of communication was directly with Scottish Executive officials. Latterly, that would be officials from the enterprise and lifelong learning department and the education department, as both had an interest in SQA business. There were also annual meetings with ministers. I never met the minister alone, but always in conjunction with the chairman.
Do you have a view on whether there were adequate safeguards in the system of governance to allow the emergence of a potentially dangerous situation to be identified? If such safeguards did not exist, what safeguards should be put in place now?
Since 10 August, I have thought about that question a great deal. It is a very hard question to answer. What happened in the SQA was so extreme and unusual that one wonders what governance arrangements would have made a difference in that situation.
I am grateful to you for your candour, Mr Tuck. The question that the committee is wrestling with is, was this such a phenomenon in the exercise of the SQA's functions and objectives that nobody at managerial level could have been aware of an emerging problem.
I would not go so far as to say that—clearly not. My difficulty, as you will understand, is that on Wednesday 9 August I went public to the press and to everyone else with what I believed to be the situation. It transpires that the problems were far worse than I had understood them to be, but as I have not been in the employment of the SQA since 11 August, I have not been privy to the investigation. Given the information that has emerged since then about the extent of the problems, it would have to be said that somebody at some level within operations unit must have understood that the problems were far worse than senior management had been led to believe. I am not in a position to answer why that was the case.
Does that point to some deficiency in line management, communication systems and accountability?
I would guess so. I would presume that that is correct.
Thank you. I believe that Mr Lyon has a question. Is it relevant to the point that we are discussing?
Yes. I would like further clarification of that point. Either the process was wrong, because you were not being given the information about the problems in data processing, or you were being lied to by the staff below you. Which was it?
I have no reason to distrust or doubt the good faith and integrity of any member of staff. There is no evidence of that, with one possible exception. I am therefore unable to explain why data management problems were worse than I had been led to believe.
On that point, I want to return to your evidence. You say that the information that you gave to the board at the time was, to the best of your knowledge, correct, and that you and your senior management colleagues "were not fully aware" of the difficulties. We have heard evidence from the board. What is the difference between information being correct to the best of your knowledge and a guarantee that information is correct? In future, how can information be guaranteed to be correct?
The management information difficulty we had was that to grapple with centre-held data that were apparently missing we needed our data to be organised by centre and candidate. Information is not built up in that way for the purpose of issuing certificates; it is assembled by subject and reaggregated by centre and candidate only in the latter stages. We therefore found ourselves having some difficulty getting information in which we could have total confidence or that could be produced quickly. That was a difficulty that we should perhaps have anticipated, but I would have to say that had we embarked on trying to build in that level of sophistication to the IT system we might have been led to advise the Government that implementation of higher still was not doable.
You are quite categorical in your evidence to the committee that you do not feel that
By and large, attendance at board meetings was good and we had lively and good discussion. In the past year in particular, I found them quite probing. Members of the board asked the kind of questions that I would have asked had I been a board member. They asked the right kind of questions.
I seek clarification of your statement that the problem was in data management, that the problem did not surface until June and that you were not aware of it in its full entirety until 10 August. Are you saying that the problem was not related to the APS developments that the board discussed, the issues on which the Scottish Executive offered help in March 2000, or the electronic transfer of data between schools and the SQA, to which the SSBA, according to its publication, alerted the SQA and the minister in March and April 1999? Are you saying that something else went wrong? What would be the channels by which you would find out what went wrong in another department?
I would not go quite as far as you do in your summary. The lateness of software development added to pressures and therefore contributed to a data management problem. We were alerted to electronic transfer problems through our contacts with schools. I do not remember learning of the SSBA's concerns until we read them on the pages of The Herald, at which point the chairman wrote to Ann Hill for clarification. I have no recollection of Ann Hill contacting us directly in that regard.
Did you inform ministers of any of those discussions or the fact that concerns had been raised on any of those issues?
I did not have a direct line to the minister. My line of contact was with officials at the education and lifelong learning department. In March, for example, we had discussions about software, which was why the department kindly gave us the services for a day of a leading IT expert from the Scottish Executive, who came in and looked at what we were doing. There was that kind of open dialogue. As we became aware of problems, we informed Scottish Executive officials. The killer problem was one that we were not aware of until it was rather too late.
I am keeping an eye on the time. I do not want to inhibit members' proper lines of questioning, but I ask you to keep your questions as brief as possible and to let Mr Tuck answer as concisely as he can.
We heard the chairman, Mr Miller, say that he informed the Executive last year that new computer systems require to work properly or there will be serious problems. You did not seem to answer Dr Murray's question directly, so I pose it again. Can you recall when you first informed the civil servants about the emerging problems that you were discovering in data management?
It would have been late June.
I see.
It was the operations unit.
Was the particular problem related to the input of information from schools?
The problem was related to the fact that data sent to the unit from schools—and indeed from further education colleges—was not being successfully processed in the system. Therefore, there were missing data.
What puzzles me is that if there was an effective system of governance—[Interruption.] I am sorry, I am a bit distracted, as Mr Lyon is making remarks to the convener. If there was a problem of—
My Lyon is seeking a point of clarification.
May I continue?
What do you want to clarify, Mr Lyon?
Did the problem in that most recent reply relate to paper or electronic data?
Primarily paper, but the question would need to be asked directly of the SQA, because it has conducted an internal inquiry. I can give you evidence only on the basis of my knowledge and understanding which, as you know, proved to be less than perfect. The problems were greater than I had been led to understand by 9 and 10 August. I am not evading your question; I am saying that I am not necessarily able to provide a precise answer to it.
What we are trying to establish is whether the issue of governance is of any relevance to this matter. I believe that last year you set up focus groups in schools, on issues related to assessment. Was that related to higher still development rather than data management?
No, it covered all aspects. It also covered administrative and information technology issues.
Did you receive feedback from that focus group process, which I think you instituted quite early last year?
Yes.
When exactly did you institute it?
I cannot recall exactly when. It might even have been late 1998, but I am working from memory.
What puzzles me is that in addition to all the feedback you got from the focus groups, which were part of the governance of the SQA, as soon as schools began to send information about internal assessment and the three units which I believe were the required component, they informed the SQA of problems with the correct recording of those data. When did you become aware of such problems?
We have to distinguish between different kinds of data. Earlier in the session, centres must send in registrations, which is what happens when a candidate first becomes involved with the SQA. That is a one-off process. The next piece of data is entry, which states that the candidate is taking these units, that course, this group award or whatever. Results can be submitted only after a candidate has completed a unit or a course. In the main, you do not expect to get the data in any volume until April or May. It is in the nature of the system that you would not come across problems of that missing data at any early stage.
Yes, but what puzzles me is what seems to be, on the face of it, a contradiction between Ann Hill's evidence and Mr Miller's evidence. The thrust of Ann Hill's evidence is that teachers' concerns were made manifest in October 1999. Mr Miller's first warnings to ministers were on 25 July 2000. There is an apparent conflict of evidence. It seems surprising that warnings were not given to ministers earlier than has been indicated so far in the evidence.
To understand that, we need to recognise that there were different issues. Warnings that were given were about different matters at different times. The primary concern of the assessment focus group—as you would expect from a body representing teachers—was unit assessments in schools. As you know, that aspect of higher still has been the most controversial. A body of teachers' opinion is critical of the idea of unit assessments in schools. We were keen to monitor feedback from teachers on how unit assessment was operating so that we could make necessary adjustments. Administrative issues were discussed, but they were about, for example, difficulties with centres' own software submitting data to us.
I suggested earlier that one deficiency in the governance of the SQA may be that there is not enough teacher involvement, especially those who are teaching in the classrooms. Do you share that concern?
If I were still in my former position, I would welcome the involvement of more teachers on the board—who could be against it? Having said that, we had multiple channels of communication with teachers. You mentioned the schools assessment focus group. We had the same arrangement with further education colleagues.
I would like to return to the issue of data processing, to ensure that I have it clear in my mind. You were saying that the major flow of data did not start until April and May of this year.
Are you talking about the results?
I am talking about the results coming into your system. Only then did you start to identify that there were data processing problems. You said that those problems were mostly on the paper side. What form did those problems take? Was the information physically going missing, was it not being entered into the IT system, or was it being mislaid? I find it incredible that you had no idea where these bits of paper were disappearing to.
When we became aware that a massive amount of data was outstanding, the first advice that I received was that it had not been submitted. Various explanations for why that might credibly be the case were given to me. When we contacted schools, they informed us that they had submitted the data already. At that point we began to look for other possible explanations. In the somewhat more detailed paper that I submitted to the Education, Culture and Sport Committee, I describe some of the audit trails that we followed. If you wish, convener, I can rehearse those explanations now.
I am anxious that we stick to the issue of governance. I appreciate that some of the incidents that we have been discussing will help the committee to come to an opinion on the mechanics of governance, but we are more concerned with the terms of line management and accountability.
In conjunction with the director of the awards division and in discussion with colleagues, I attempted to identify a number of potential reasons for data going missing and to carry out an audit. The aim was to ascertain the extent to which a particular problem might be the cause of data going missing. We followed five or six such audit trails; in terms of staff time, that is quite an expensive activity. Each of the trails produced an unsatisfyingly partial explanation.
I return to my earlier question: did the process fail or was there an attempt to circumvent the audit trails to ensure that management was misled about what was happening in this part of the operation?
I can respond only as I responded previously. The answer to your question will come from the SQA's internal inquiry. Because of the pressure that we were under to deliver certification, we did not have time to carry out an exhaustive inquiry into what had gone wrong. My understanding is that the new chief executive has led such an inquiry. I am sure that that will shed more light on the matter than I can.
You said of the things that were going on in the SQA last year, such as the new computer system or the implementation of higher still, that none of them alone caused the knock-on effects, but that various things combined to impact on data management. You said that you were unable to forecast that.
The overall time scale did not allow for user-acceptance testing, if I understand what you mean by that.
Is it true that, other than unit testing of the software to verify that it did more or less what you expected it to do, there was no end-to-end testing of the whole system as a process, from the point of registering people to sit exams through to the point at which the certificate is produced?
It would have been beyond the scope of our powers to do that, given that such testing would also have involved the seven companies that supply software to our centres. To have ensured that such national whole-system testing took place would have required an initiative outside the SQA. The software developed for centres was not the responsibility of the SQA; it was our responsibility to supply certain specifications to the software suppliers, but we could not require them to do anything.
In retrospect, would you say that your project management was inadequate or that you did not have adequate powers to carry out the required testing?
That is a reasonable question to ask. I do not necessarily feel in a position to agree or disagree with what you suggest, but it strikes me as a fair question.
You spoke of people raising several issues, as opposed to one big issue, with you over the summer. I am confused. Why did you continually feel that you could meet the goals in the time scale? Why was the SQA board unaware of the on-going issues?
Even by the time the board met on 22 June, I did not know that there were data management problems. The data issue that was reported to me was one of staffing, arising out of the fact that data were due to arrive in a large flux instead of in the more even submission that we had expected. I had not been alerted to the existence of data management problems. I can now only surmise that the procedures that had worked well for years broke down under pressure and that the difficulties simply went unreported, for whatever reason. I know that Mr Lyon has been keen for me to identify that reason and I honestly wish that I could. I am sure that subsequent investigation will have identified what the reasons were.
As far as we know, the SQA board was still unaware of the situation at the time. This inquiry is about governance and about how the SQA operates. You are telling us that various problems were occurring—I accept that it was difficult to pinpoint them and their source exactly—while the board was in ignorance of the issues. Why were the issues not reported to the board? Is there any way in which the board can meet or be briefed between meetings, given the severity of the issues that were on the table?
Are you talking about the problems that became apparent after 22 June?
Yes.
The chairman wrote to board members more than once to inform them of the situation. I could not tell you the exact dates, but he wrote to them at least twice.
Was any consideration given to pushing back the date on which examination results would be mailed out?
Yes. That was considered as late as 25 July. The reason why it was agreed not to delay the issuing of results was that, as you will recall, we believed that the problem was confined to about 1,500 candidates. The choice that we saw was whether to issue results on 9 August, knowing that, as we thought, 1,500 candidates would be affected by data gaps that could be resolved as soon as data could be retrieved from schools—which would be after the summer holiday—or to delay all certification by a week to the detriment of 99 per cent of candidates. We could not be sure that that week would be enough time to pick up the missing data. On the basis of the information that we had, it seemed best to proceed with certification on 9 August. Clearly, had I been aware of the true scale of data management problems, the proposal to the minister would have been to postpone certification.
The SQA has made a submission. In answering the question whether there is a better model for the governance of the SQA, the SQA has come up with three suggestions: the appointment of assessors; six-monthly meetings with ministers to cover key issues and agree plans and targets; and regular management information for the Scottish Executive through bi-monthly senior management meetings. Do you think that those suggestions would improve the current situation and remedy some of the deficiencies that have been disclosed?
They are helpful suggestions and I would support their implementation. The form of governance in relation to school qualifications in Scotland has not changed since 1963. The Scottish Certificate of Education Board was set up in the early 1960s under a form of governance that was similar to the one we have now. In that sense, the SQA is not a modern quango; it is a 37-year-old quango. In all that time, it is only this year in which things have gone badly wrong. I accept responsibility for that. I do not think that it raises questions about a form of governance that has, as I say, generally served the country well for more than 35 years.
In relation to the governance of the SQA, your submission explains a great deal about the relationship that the SQA had with the Executive and the board. However, it does not say a lot about the relationship between the management and the staff. You will have heard me ask about possible cultural differences. Could you explain whether there were problems of governance in the structure of the SQA that might have contributed to the problem? Did the inheritance from the two organisations contribute to the difficulties that were encountered?
David Miller summed up the situation well and I would not disagree with him on that issue. Obviously, it would have been helpful to have had the whole organisation on one site, but that would have been risky because of the staff losses that it could have incurred.
Did you have any form of official staff consultation, such as staff consultative meetings or meetings with staff representatives, that might have thrown up additional information that otherwise was inaccessible to you?
We had regular meetings with our recognised trade unions, which represent around 35 per cent of staff. Those meetings were useful. They were always chaired either by me or by the director of corporate services. We did not have a staff consultative forum as such, although that was under consideration, but we had a variety of means, such as team briefings and staff meetings, of communicating with staff. We tried to use a range of communication media to keep in touch with the views of staff, and indeed to consult them on matters that were of great concern to them. For example, we had a number of consultations on how staff wanted the restructuring of the SQA to proceed.
Mr Ewing, you have one brief question.
I understood you to say that the SQA has had a seamless existence since 1963, but it was set up in 1996 from a merger of two organisations—the Scottish Vocational Education Council and the Scottish Examination Board. If the SEB had been left to get on with things, would we be here now or would it have produced the exam results with meticulous accuracy, as it did previously?
I did not say that there was a seamless line from 1963 until now. I said that the form of governance—that is, the powers that the Government has over the organisation—that was adopted for the SQA was much closer to the one for the SEB than the one for SCOTVEC; it represented pretty much the status quo for the SEB, but a tightening of control for SCOTVEC.
This is just a statement—major Government policy change can cause extreme and unforeseen problems.
We are here to ask questions, Mr Ewing. Have you concluded your interrogation?
Yes, convener.
Mr Tuck, on behalf of the committee, I thank you for attending this afternoon. We realise that you have been under significant pressure in recent months and that you may have viewed this occasion with some anxiety, so we are particularly appreciative of your attendance. I think I speak for us all when I say that the candour and fullness of your answers has been extremely helpful. Our remit is to try to improve circumstances, if such improvement is possible, and your evidence will greatly assist us in our consideration of the circumstances.
I was just going to say that it is nice to see parliamentary democracy in action.
In concluding the proceedings, I remind members that the next meeting will be on Friday at 9.30 am in committee room 1. I look forward to members being available at that time.
Meeting closed at 16:16.