Scottish Qualifications Authority
Item 2 is on the Scottish Qualifications Authority, of which we have a number of representatives with us. The committee continues to review the SQA's progress in implementing our recommendations and in preparing for this year's exam diet. I ask Bill Morton, chief executive of the SQA, to introduce his colleagues and to make a short opening statement.
Bill Morton (Scottish Qualifications Authority):
On my left are Don Giles, who looks after our external affairs and corporate services, and Dennis Gunning, director of the development division. On my right are Billy MacIntyre, acting director of the awards division, and Jean Blair, who is involved in our project planning work.
I have copies of my opening statement, which I will make available to the committee and for broader distribution if the committee is in agreement with that.
That would be helpful.
For the committee's benefit, I will summarise the information that I have already submitted. In the next few minutes, I will cover our progress on: improved management information; our project or action plan; replacing the operations unit; markers; consultation on further improvements that reflect the committee's recommendations; the stability of our information technology systems; and improved communications.
I believe that the SQA now has in place many of the changes that are required for certification in 2001. Improved management of information allows us to track the progress of key stages in our preparation for the summer. For example, we now share information on registrations, entries and appointments with ministers, the Scottish Executive, the ministerial review group, our staff, education community stakeholders and the wider public through the media.
Knowing just where we are is a major improvement on last year. It also means that we can ask for help when we need it. We are grateful to the schools and colleges—and to the directors of education—for their co-operation in, for example, data collection and the recruitment and release of markers and the other appointees who are required.
We are finalising a project or action plan. It details the actions that are required for successful certification in August 2001. That identifies everything that we must do, the extent to which we indeed rely on the support of others and how progress in those areas can be measured. The plan also covers who is responsible for doing what and by when, plus an assessment of risk and the identification of contingency measures, should they be required. As the committee knows, our internal auditors, PricewaterhouseCoopers, are assisting us to track the plan and verify its achievements objectively.
With the support of ministers, we are setting specific targets against which our overall progress can be measured. The targets, which have still to be finalised with our board, will cover such matters as certificates being issued accurately and on time, results being sent to the Universities and Colleges Admissions Service on time and the efficiency of our handling of the appeals process. The targets—or benchmarks—will be published shortly.
Our immediate task has been to improve the processes that directly support certification. The former operations unit has been replaced with a new unit that focuses explicitly on data management and certification. The new unit comprises four dedicated teams: one for software development testing; one each for school and college and for employer and training provider data processing; and one for certification. As well as having additional staff and strengthened management support, the teams have worked hard to address the many procedural weaknesses that contributed to last year's problems. One example of that is the way in which data are now submitted, recorded, logged, processed and confirmed back to the centres. A system of strict validation of data is now in place to ensure that the originating centres can verify the information that is held by the SQA and that any errors can be identified and rectified quickly.
We also have a team that is involved in resolving outstanding cases from last year. In the main, those are connected with reconciling college data and with anomalies such as very late appeals, late absentee considerations and misdirection.
Since 5 March, when we last reported publicly on markers, we have started to use the number of actual course entries rather than estimates as the basis for recruitment. As a result, the number of markers that we now need is around 8,000. Given last year's difficulties, I am sure that the committee will understand that we are adding a further 10 per cent to the total to provide extra comfort. In particular, we are making sure that we have enough markers in 15 subjects—including drama and administration—that are either usually difficult to recruit markers for or are new to the diet this year.
I can confirm that 6,052 markers have been appointed and that responses are awaited from another 1,307 recent invitees. With a pool of a further 500 potential markers in reserve, I am confident that our progress is satisfactory in this important area. The recent 50 per cent addition to fees is undoubtedly helping, but it is clear that the fundamental review of appointments, which is due to take place later this year, will be welcomed by all concerned, including ourselves. I am confident that the SQA board will support the simplification of fee structures and a movement towards the prospect of longer-term appointments and improvements.
The Association of Directors of Education in Scotland's inquiry and the Scottish Executive's independent review have both confirmed the robustness of our 2000 appeals process. Clearly, however, there are lessons to be learned and the SQA will be conducting its own analysis with a view to improving our appeals handling for 2001 and beyond.
Further changes have been made following consultation with schools, colleges and other interests, including the recently established ministerial review group. The one major stipulation is that such changes must not jeopardise the prospect of successful certification this year. For example, the Scottish qualification certificate will be simplified as far as is possible this year. Core skills information will be moved to the back of the certificate and there will be a much clearer explanation of what core skills mean and how they were attained.
We consulted on minor adjustments to the 2001 examination timetable, such as advancing some higher still national qualifications into the first week. A group has been set up to consider the 2002 timetable with a view to consulting on any proposals emerging before the summer.
As part of a move towards greater openness, we will consult on various measures concerning assessment and awarding procedures and on piloting the return of examination scripts for the entire winter diet in January 2002.
I stress that preparations for such changes must take place in the context of safely delivering this year. For example, we, like many others, hoped that the default pass model would be a way of decreasing the quantity of data moving between the SQA and its centres. I wrote to the committee on that point. However, our feasibility study revealed that its introduction this year would pose too great a risk and could destabilise our system. We have not lost sight of the need to simplify and we intend to consult and to prepare fully for simplification of the collection of internal assessment data in 2002. Similarly, any changes to the awards processing system this year will be strictly as required. Since various inquiries concluded that the system did what it was supposed to do, in the main, but that it used flawed data, our board has decided that its stability is to be maintained at all costs. PricewaterhouseCoopers, our internal auditors, will help us to monitor and to report on the development of the system at each stage, up to and including certification.
As part of our efforts to improve communications with stakeholders, we are nearing the end of a series of seminars with SQA co-ordinators in schools and colleges. To keep centres up to date, we publish a regular newsletter, the "NQ Digest", which provides useful information on national qualifications such as reminders of procedures and time scales for submitting information.
A team of school account managers is in place and is improving greatly our communication with schools. The appointment of a communications manager will be confirmed shortly. That will undoubtedly help to improve the effectiveness of our communications inside and outside the SQA.
As indicated previously, secondees from schools and colleges will join our senior management team. That will bring valuable first-hand experience of how our relations and dealings with those important customers might best be improved further.
Although there is still a lot to do, I guarantee that we are doing all that we can to ensure that candidates and their families suffer no repetition of last year's problems.
Thank you for your statement. I am sure that members will want to pick up on a number of issues.
I want to ask about markers and visiting examiners. Do you have enough visiting examiners to carry out the tasks that they are required to do? If so, that is good. If not, how are you going about rectifying that problem? You indicated in February that you are about 400 moderators short. Has that shortfall been met?
If you are not going to have all your markers in place, what kind of support will be available to people who come into the system later? We picked up on the fact that people were not attending marking meetings or getting proper briefings, which caused more problems in the system. We are interested in those issues; perhaps you could touch on them first.
Briefings and markers meetings still lie ahead of us, but last year's experience has taught us that we must ensure that the administration of markers meetings is as it should be. I shall hand over to Don Giles to answer your question on visiting examiners and moderators. However, I must say that one of the improvements in the SQA is knowing where we are and sharing that information with everybody who has an interest. When we flag up concerns about not having enough moderators at a point in time, that is us being open in order to be accountable. We are not giving rise to concern over a problem that cannot be managed; we are illustrating the fact that we are managing the problem.
Don Giles (Scottish Qualifications Authority):
Let me clarify the position on examiners and moderators. We are 75 per cent there with examiners and 88 per cent there with moderators, but the position changes according to the entry data that we have received. We are still firming up precise numbers. Indeed, early indications from the entry data suggest that we may not require the full complement of moderators that we have identified this year.
The issue of markers is topical, given today's press coverage. Let me brief the committee on the position. We started marker recruitment somewhat earlier this year, as you know, and the process has been assisted considerably by the 50 per cent increase in fees awarded by the Minister for Education, Europe and External Affairs.
Until recently, we have had to work on estimates of our likely marker requirements, as we had no definitive entry data—on course entries, subjects and levels—to draw on. The estimate that we were working on, which has been drawn to your attention, was that we required 7,500 appointments, on the basis that each of those appointees would mark about 90 scripts. We also planned for a contingency of 2,500 appointments and we have been seeking to top that up with a further 500 would-be markers, who are currently in the pipeline and are being vetted for eligibility. What we have wanted throughout is to have healthy contingency arrangements; that has been very much part of our strategy for this year, to prevent a repetition of last year.
In recent days, we have moved from using estimates to using actual figures, because we now have definitive entry data. Those data are encouraging, as they show a healthy uptake of intermediate 1 and 2. However, because there is an upturn, we require additional markers. Furthermore, a new trend has emerged from the markers who have agreed to mark for us this year. They do not want to mark as many scripts as in previous years; the number is down to about 60 or 70 scripts per appointment.
In the light of the entry information and in the light of the declared intentions of markers, we have decided to carry out a re-evaluation of the marker process. We normally undertake that process in April; we are therefore a month ahead. In the light of that re-evaluation, we can revise our targets. The data indicate that we now need 8,000 markers. We will add 10 per cent to that for contingency—to give us a comfort zone—which means that we are aiming for 8,800 markers. As Bill Morton said, we currently have 6,052 people in the bag. Invitations have gone out to another 1,307 willing teachers and something like 1,200 people are on our reserve list. As I said, 500 people are currently being vetted.
Media speculation over the past two or three days has resulted in our marker helpline being inundated by calls from teachers across Scotland offering to mark for us. Further recruitment is therefore in hand to top up the numbers that we require—not just to 8,000, which is the number that we definitely need, but to 8,000 plus a buffer of 10 per cent.
Mr Giles, you said that you have 77 per cent of the visiting examiners that you need. As I understand it, those examiners, if they are not already visiting schools, should be preparing to visit them. If you do not have a full complement, how does that impact on the examiners' work?
The visiting examiner process goes on over a period of time. We may have only 77 per cent of the examiners, but names will be identified and release will be sought to make up the residue.
Do you therefore see no problems in that regard?
No serious ones.
This is a rolling programme—it is possible to bring in visiting markers according to subjects as the programme progresses.
What is the position with moderators? I understand that the moderation in modern languages has been affected and I understand from former colleagues that the moderation of oral skills in English appears to be a bit behind schedule—or, at least, people are not being given as much warning as they might have expected that moderation is going to take place. Are local authorities comfortable about letting moderators away from their own classes in order to perform this task?
I do not have data to hand on moderation in those areas, but I can source them and supply them to you directly.
Is it correct to say that moderation in modern languages has been technically suspended—I am not sure whether that is the right term—or that it has not been going ahead as everyone expected?
Are you perhaps thinking of visiting examining rather than moderation?
Yes, possibly.
That could be the source of the confusion.
Jean Blair (Scottish Qualifications Authority):
I would like to clear up some of the figures that have been mentioned. By 5 March, we had appointed 85 per cent of the moderators that we needed; the most up-to-date figure is 88 per cent. As Don Giles has said, the moderation section may be doing some number crunching and we may not need the full complement of moderators that we have on our books.
You asked about the visiting examiners for modern languages. You are right to raise that issue, because it was a concern. We had a contingency arrangement for getting in contact with schools and ensuring that we could deploy people for modern languages as needed. If a subject is early in the programme, it is given priority in the appointments made.
Are you worried that the process is not fully operational?
We recognise that the whole area of moderation and visiting examination is very important—as are many other areas. We are monitoring it diligently and regularly and will respond to any concerns quickly. Some months ago we were concerned about the appointment of moderators. We focused our activities on that and addressed the issue. I am not unduly concerned about it at this time.
Are you comfortable with the moderation of spoken English, for which the numbers are bigger?
We are happy to come back with a detailed written response on the subject areas for the committee, giving you chapter and verse, if that is acceptable.
Yes, thank you.
We are familiar with the fact that you rely vastly on the support of teachers for marking. The committee has been told a number of times that the fee for marking, although important, is not crucial, because many teachers value the marking process in itself. However, since we took evidence on that, there has been the McCrone pay settlement. Has that had any impact? Do teachers feel that the increase in pay means that they do not have to mark, or will it cause problems in future as higher rewards are offered to teachers? Will you have to revisit the issue of the markers' fee?
I will give you a personal view, which Don Giles can supplement. It is too early to make that judgment in terms of McCrone. We are probably better placed to judge the process that we are embarked on against some of last year's difficulties. We acknowledge the support of schools, education authorities and colleges in releasing teachers and of the teachers and lecturers who are coming forward.
The matter could perhaps be properly addressed in the context of the upcoming review, which will help to simplify the entire fee structure. The opportunity exists to do that in the context of further information about the implementation of the McCrone proposals. The issue might also be important as we consider whether SQA appointees should be on some form of rolling contract rather than have to go through the recruitment process from start to finish annually. The short answer is that I have not seen any evidence that would enable me to offer an answer with conviction.
It is true that many teachers get involved in the SQA awarding body in order to further their career through professional development. Certainly, there was disillusionment last year with the fee rates. We have done a comparison with a similar body in England, which suggests that the new fee rates, which have been increased by 50 per cent, are roughly the same as what the body in England is paying.
The minister has dealt with that issue, but we are committed to a fundamental review of fee levels. We will be working closely with the Scottish Executive and we will be factoring in McCrone and the length of time that is available for marking, which is two weeks. In previous years, the period was three weeks, which meant that the window of opportunity was wider and that there was less pressure on markers. Such considerations will be built into the review and recommendations will be made to the Executive and the SQA for implementation in 2002.
Would you rule out an increase in the fee in future because of McCrone?
That is an interesting question, which I am sure you appreciate I am in no position to answer. I cannot speculate on the findings of a review that is yet to take place.
I want to talk a little about communication and public confidence. Mr Morton, you were quoted in the Daily Mail on 1 February as saying:
"We are behind where we would like to be, but this is a problem not a crisis."
This is 20 March. What is the situation at the moment?
It would be wrong to pretend that we are exactly where would we like to be after catching up with the tasks in the aftermath of what happened in the summer. However, that is not to suggest that our current position should cause major concern. The position today—20 March—is that we have sound foundations in place and we have made many improvements and changes that should allow us to prepare effectively for certification in the summer. I am not concerned about the criticism that inevitably has been levelled at the SQA; however, I am intent on doing all that we can to put the facts into the public domain in such a way that the candidates preparing for exams will be reassured that we are doing all that we possibly can do.
That is the key issue. The people who must be made happier are the young people who are about to go through the diet, their parents and their families. Teachers and others involved would follow them in the hierarchy of comfort.
In its report last year, the committee recommended:
"SQA should ensure that it puts in place a greatly improved internal and external communications system
In 2000 SQA was found to be woefully wanting both in internal and external communications between its Committees and its staff; its staff and other staff; and with centres, candidates, the Executive, education authorities and interested national bodies. This must be redressed quickly to ensure that the system works and that credibility is restored."
I do not know you feel about the press releases and press coverage over the past 10 days. There have been headlines such as "Taxpayers face huge bill to meet the cost of SQA exams fiasco", "Court action after exam results chaos", "Fears of second exams fiasco" and "‘Nervousness' as exams loom". According to The Press and Journal, there are "Ominous signs of exams fiasco". Such headlines are not restoring confidence. Quite the reverse—people feel on a knife-edge about what is going to happen.
What is your strategy for dealing with such publicity? Do you accept that the headlines are highly damaging to young people at this time? Finally, how can we be assured that the situation will not continue? If it does, I am afraid that we will just get more and more worried.
I am sure that you will accept that I cannot account for how the media choose to address the issues, although I perfectly understand that if they have concerns, they should raise them.
I can answer your question in the light of the approach that we have taken. A major improvement on the situation last year is that the organisation's management information now allows us to know exactly where we are. We have also recently introduced a method of sending that information to everyone who needs to know it. We regularly brief the ministers, the education community, the ministerial review group and the general public through the media. We are certainly doing our best to put across the facts, which lead me to believe that I can guarantee that we are doing everything we can to offer reassurance to candidates.
You mentioned internal communications. Although we brief the staff too, I will not pretend to the committee that we have addressed internal communications satisfactorily. The people in the SQA work very hard and are dedicated to what we are trying to do, but they are still suffering the aftermath of the shock of last year's events. As a result, there is—understandably—a very strong appetite for information about everything that is going on. We are making continuous improvements to internal communications.
You also raised the specific issue of schools. In my opening statement, I mentioned school account managers, who were introduced in late September and early October. They seem to have been popular with schools, which now have a single point of contact for exchanging information with the SQA. The "NQ Digest" newsletter also provides schools with useful information, which sits alongside the usual channels of communication, such as the guide for the heads of centre. That is all helping.
It is almost inevitable that we are having difficulty reaching the constituency that comprises candidates and their families, because the means of communication are not direct and people are relying on the media for information. A week or so ago, I attended and spoke at a Headteachers Association of Scotland conference, at which one of the head teachers suggested that if we were going to send out a newsletter to inform teachers, it would be a good idea to send one, at a point in time not too far away, that would provide information in a user-friendly way to all the candidates who are about to sit exams. Personally, I am impressed with that idea—I thought it was good and I have asked our staff to look into implementing it.
Sending a newsletter is one solution, but another would be to write directly to the candidates, to say to them, "We are doing our very best to make this work."
Do you accept that we are getting very close indeed to the start of the diet—it is only a month or so away—yet it is still not possible for you or your colleagues to say categorically that you believe that everything will work in the way in which it must work if you are to succeed? As you know—I believe that you said this to the committee previously—99 per cent success is not enough. I am not pushing you on this, but until people hear those words from you or your chairman, there will be difficulties. How close are you to being able to state confidently that everything will work?
I am not trying to avoid answering that reasonable question, but I am sure that you understand that it is extremely difficult to answer it. At this stage of the process, the improvements that we have put in place seem to be having the desired effect. However, they are not perfect. For example, the verification report back to the schools could be much more user-friendly, but to make it so would entail risky software development. We will make progress on that in future. As I said before, I can guarantee that the steps that we have taken so far are the right ones. I firmly believe that and my view is supported by our board. I can guarantee that we will do absolutely everything that we can to ensure that there is safe, complete and accurate certification on 14 August.
May I press you on that?
This is your final point, Mike.
I understand that you cannot say that there will be 100 per cent success. Could you go so far as to say, either to the committee today or to each candidate, that if problems arise this year, they will be far smaller than the problems that arose last year? I do not like using the word "reassurance", because it has been over-used, but it would mean a lot if some came from you, given the authority of your office, and was backed up by your staff.
I understand your point and I really would love to be in a position to offer such a reassurance. However, I think that it would be disingenuous of me to do so—it would be cosmetic. I am content to place what you say in the context of our ambitions, aspirations, aims or objectives—you may choose the term. We are working towards ensuring that there is complete and accurate certification on 14 August. I know that that falls short—
You will appreciate my disappointment at the fact that you have not taken that final step.
I understand that, but there remain steps to be taken and improvements to be put in place, which must have an effect. My ambition is identical to the guarantee that you are asking me to give. There will come a point in time, further down the track, when we will know that the steps that we have taken are having the desired effect. If you ask me that question again then, I might be prepared to answer it more definitively.
When?
Some time in the near future.
I am tempted to suggest that, after the events of last year, the currency of an assurance from the SQA might not be that strong.
What is morale in the SQA like at the moment? What evaluation have you made of morale?
Morale inside the organisation is not good; it is low and people are working extremely hard. Mike Russell talked about the effect of the media on public opinion, by causing disquiet. I understand that the media are chronicling the various issues; the media are also seeking the reassurance that Mike Russell sought. As I am sure members will appreciate, that has an effect inside the organisation that is hard to counteract. At the same time that we have to get volumes of information out, staff have to have real knowledge of where we are and need to know that our progress is achieving tangible success. Staff are not comfortable with all the changes that are either taking place or yet to be refined. It would be wrong to suggest that a sunny gloss could be applied to the situation at the SQA. We have an issue with morale, but we are addressing it. That we have it is a matter of concern to me.
Has the appointment of PricewaterhouseCoopers helped the situation?
There is a difference between fact and perception. There may be a perception that PwC has been appointed to snoop on the staff, but that is untrue. PwC is our internal auditor and it has been appointed to work alongside staff, and with the SQA board, to ensure that the detail of the project plan is progressed properly and achieves the desired result. That should reassure the staff, as PwC is not there to sit on staff shoulders, to second-guess staff or to snoop. It is good business practice to ensure objective corroboration that what is said to happen is indeed happening on the ground. I am sure that staff would not want it any other way.
Would PricewaterhouseCoopers be able to help us to identify the insider who was quoted in the Daily Mail article of 14 March?
I am encouraged to believe that there is only one individual, but I suspect that there is probably more than one.
Do you feel that the resources that you have been given—or that the minister with responsibility for education has indicated that you will be given—are sufficient to deliver the outcomes that most people seek? The insider did not seem to think so and that is what I am worried about.
As far as I can judge, the Scottish Executive has given us every support. I have absolutely no concern about resources. Two choices are available to us: either to make fairly radical changes, which could be costly in the short and long term, or to make changes that are absolutely essential but do not risk any downside to the safe delivery of certification 2001.
In August last year, we had to go from a poor organisational state to one in which we will be proficient enough to deliver this summer's results safely, completely and accurately. We put the essential changes in place and we have been able to resource them. I have heard reference to a £6 million deficit. I am not sure where that figure came from as the deficit that I am aware of, and which we have made public, is £3 million. That sum has been partly expended on resolving issues from last year and on putting in place changes for next year.
Is the alleged individual a unique example of staff disillusionment or are the individual's concerns reflective of wider concerns?
It is difficult to speculate on that, as it is true to say that the staff have concerns. My colleagues, the board and I do as much as we can to tease out those concerns and to resolve them. I believe that our internal communications will get better. I mentioned that we are soon to appoint a communications manager, which will help. I am not trying to place the burden of finding a solution on the shoulders of one individual, in the complacent belief that that will be the end of the matter. We are all busy doing many other things, but there is a clear priority to spend more time with the staff, to explain things, to listen to their concerns and to respond to them. We have some ideas that we are about to implement.
I will ask about some of the serious and specific concerns that have been raised in the press and elsewhere, which I do not believe you have addressed properly in your opening statement or in the written submission that we received. For instance, last year's difficulties were largely attributable to data processing software, yet we understand that the computer system is largely the same as it was last year. I would like to hear your views on that, particularly because we understand that testing of the system has shown a higher error rate than last year—it is up from 2.7 per cent to 3 per cent, and a 2.7 per cent error rate was enough to have an adverse impact on 17,000 results.
I would like your comments also on how you think the system will cope with the projected 40 per cent increase in data to be processed this year. Those are specific concerns and people would like you to comment on them.
Those are indeed specific concerns, but I suggest that they are false. They are derived from the media coverage rather than from the facts. Again, I emphasise that I am not criticising the media; they are doing what they have to do.
All the inquiries found that last year, by and large, the computer system had done what it was intended to do, only it did so with flawed data—if one puts flawed data in, one gets a flawed outcome at the end of the process. The computer system was largely okay. I will ask my colleague Billy MacIntyre to say more about data volume in a moment or two.
Comparing apples and pears—the idea that a 3 per cent error rate over here is greater than the 2.7 per cent error rate in last year's certification—has caused considerable disquiet, especially among candidates. If I were a candidate, I would be worried about it. In fact, the 3 per cent error rate related to entries from the Strathclyde educational establishment management information system—SEEMIS—group of schools. That error rate was tangible evidence that the verification and validation process was doing exactly what it was intended to do: flag up errors so that they could be fixed.
A tremendous leap is required to say that a 3 per cent error rate in the transfer to us of data on entries, which is an important part of the process but a small component of the big picture, from a group of schools in the west of Scotland is the same as a 2.7 per cent error rate in the big picture. The 2.7 per cent error rate last year was for the entire certification. I understand how such correlations are made, but they are quite misleading and, I imagine, distressing for candidates.
I accept what you say, but have you done any testing to allow you to compare apples with apples? If so, what was the margin of error?
Billy MacIntyre will answer on the data volume issue and also on the processes that are now in place to log, confirm and check errors at every stage. Last year, errors tended not to be detected and the problem was compounded at each stage—basically, a bow-wave built up. This time, we have checks and balances in place at every stage so that errors can be identified and remedied and are not allowed to compound.
Billy MacIntyre (Scottish Qualifications Authority):
The 40 per cent increase in data volume that has been cited in the press is the increase that we anticipate in the number of units within courses. In absolute terms, that approximates to an increase of 160,000 compared with last year. The units that contribute to courses are only one relatively small aspect of the significant volumes of data that the SQA handles. My best estimate of the increase in course entries in the current year is about 11 per cent to 12 per cent across the board. The 40 per cent relates to a specific aspect of the data that we deal with—in relative terms, it is a smaller proportion.
Bill Morton mentioned comparing apples with pears. The process for centres submitting data to us is iterative. The centre will submit a range of data to us and we allow into the system those data that we can check and validate. The word error is quite emotive, but those data with which we feel there may be problems will be rejected and returned to the centre for checking. It is difficult to draw any comparisons with last year, as one of the main problems that was encountered then was that the controls that were required in the system—that would have confirmed the errors that had been reported—did not exist. We have reintroduced the controls and they are resulting in rejections from the system. Moreover, the initial controls are stopping potentially erroneous or problematic data getting into the system in the first place.
As it is an iterative process, the proof of the pudding is essentially in the final product, which is the number of course entries and the number of unit entries. Where we can draw comparisons with last year's entries, we are confident that the data that we have in the system now stack up.
An error is not necessarily a mistake; it is just something that has been flagged up as needing to be checked. It might be a change of address or it might be an entry with a completion date of 2002, when it should say 2001. The important point is that the verification and validation checks are improved. At every stage, the schools and colleges have the opportunity to check the data that we hold on their behalf. They should be comfortable that the data are complete and accurate and that last year's problems do not repeat themselves. I appreciate the additional work that that might mean for schools, but at the end of the day, it is in their and our best interests—and, fundamentally, in the interests of the candidates—to know that the data are correct at every stage, up to and including certification.
You could not give any guarantee to my colleague Michael Russell, but could you go so far as to say that, whatever else might go wrong this year, it will not be the computer system or the data processing?
That is almost the same question, but by a different route. I believe that the improvements are firmly in place and are doing what they are supposed to do to check that the data are complete and accurate, and that the system runs with complete and accurate data at every stage. I would be enormously disappointed, for the candidates, the people in the SQA, who have worked extremely hard, and the people in the schools who are supporting the improvements, if the improvements produced a less than satisfactory outcome.
You say that registrations are pretty well complete. I understand that the record sheets from schools, which give final unit information and schools' estimates of the candidates, are not in yet and will not come in until April. Is that correct?
My colleague will answer that one.
We expect to receive the estimates for standard grade, Scottish certificate of education higher and sixth-year studies in early April. We expect to receive the estimates for the remaining qualifications—higher, intermediate 1 and 2 and advanced higher—from centres towards the end of April or in early May.
I understand that the sheets on which those data are to be recorded are still being sent to schools and have not reached all schools yet. Is that correct?
The main forms that have been issued so far are for standard grade estimates and the collection of the internal assessment grades for standard grade. We had intended to issue those forms to centres by the first week in March. We experienced delays, so we were not in a position to issue the forms to schools and colleges until 16 March. That is a week behind the time scale that we had expected.
And the stuff for higher still?
Our intention is to issue the forms for higher still to schools in early April for completion by the end of April.
A fortnight's Easter holiday falls between early April and the end of April.
We are examining the time scale for issuing forms to ensure that we can fit in more successfully with the schools' work loads and schedules. That is one of the lessons that we learned from the delay in standard grade. We made a mistake with that. We must improve on that aspect for the next run of forms.
We delayed the standard grade forms for the right reason—to ensure that the data that were sent to schools were correct, in the sense of having been properly checked. I recognise that people hold us to deadlines. Deadline is an absolute term. In the interests of the candidates and everyone else, it is more important to ensure that the information that is sent out is right, rather than rushed.
Through a programme of national SQA co-ordinator seminars for schools and colleges, which have run in the past month, we have explained all that directly to the staff in schools who must deal with the data. We wrote to the directors of education to make them aware of the issue. To acknowledge the work load pressures, we said that we would make adjustments to the way in which schools can respond to us.
Some teachers to whom I have spoken are a bit edgy about that and other matters such as the flyleaves that are sent out and returned with the folio. They have been a wee bit late, which is putting teachers under a wee bit of pressure.
I want to clarify one question, although I suspect that you answered it earlier. Will the data that the SQA receives be final, or will they still contain uncertainties—will they be like shifting sands? For instance, what is the latest point at which someone can decide that they will be entered not for higher still but for intermediate? If a candidate fails to complete a unit near the end of the course, they may not be in the right position to be presented for higher still. It used to be possible to delay the later unit until after the examination had taken place, or to drop the candidate down to intermediate. Even when the SQA has received a sheet from a school, does it still have a problem with shifting data?
We are aware of that issue. The ability to alter levels according to the candidate's expected attainment is a central component and is part and parcel of the new, improved process. Billy MacIntyre will expand on that.
At every stage of the process, the data return to the centres. The process is a bit iterative, and this is the first time that we have used it. There are teething troubles, but we will iron them out and we are improving the system all the time. Registrations, entries, the submission of estimates and the submission of the internal assessment results will all be verified back to the schools.
Some people might have issues to deal with, such as that which Ian Jenkins mentioned, which involve essential flexibilities, but at each stage in the process, the bulk of the data that will condition the outcome of certification in the summer will be—to everyone's knowledge and satisfaction—verified and validated as complete and accurate through an open process.
One of the committee's recommendations was that we should check for what is missing; those checks were not undertaken last year. I shall highlight some of the main checks that we propose to undertake.
Once we have reached the point at which we expect to have received from the centres all the information on standard grade internal assessment marks and estimates, we will run checks in the system to identify where there might be gaps. We will follow up those gaps with the centres, through our account managers and the staff in our data processing section, to identify how they can be filled.
We plan to carry out checks on unit assessments in two stages, based on entries and results. At the entry stage, in April, we intend to write to all centres with a report showing all the entries, at both course and unit level, that we hold on the system on their behalf at 31 March.
I mentioned earlier the iterative process of submitting data to us. We have built in a check to confirm to the centres exactly what data we have processed. However, that process of confirmation is based on each piece of data that is submitted to us, and there is no opportunity for the centre to see the cumulative picture of entries. That cumulative picture will be sent back to centres during April, to allow them a big-picture check to ensure that the entries record is complete and accurate.
We will also run checks on the system to ensure that the combination of courses and units stacks up to a relevant qualification. Some of our initial checks have shown that there are courses for which we would expect three units to be entered, but for which only two have been entered. That is understandable. During the process of the teaching session, candidates might have options on some units and the optional units that candidates will take might be identified only as the examination approaches.
In April, we will highlight those potential gaps to centres, to ensure that nothing slips through the cracks in the pavement. We will repeat that check in early June, when results are known. We have asked centres to send us the unit assessment results by no later than 31 May. In June, we will run checks in the system to identify where we have an entry for a unit but no relevant results. Again, a follow-up process will be undertaken with the centres, to ensure that any gaps that need to be filled are filled. We must perform that follow-up process prior to the schools' breaking up, which happens usually towards the end of June.
You probably detect that we are taking your recommendations on data validation extremely seriously and that we are putting in place some firm and rigorous checks and balances. Those give rise to work inside and outside the organisation, but the changes that we are making are essential and in the best interests of the candidates. I invite Dennis Gunning to answer the specific point on flexibility regarding levels and entries up to the exams.
Dennis Gunning (Scottish Qualifications Authority):
A lot of flexibility is built into the system. For example, a candidate can take one of the units in a course at a higher level than the course itself. It is not uncommon for youngsters studying physical education to take the course at intermediate 1 level, but to do the sport performance unit at a higher level. The system allows such flexibility.
You also mentioned the scenario of a candidate finding, at the end of a course, that they have not completed a unit. The system also allows for that. The completion of units is open-ended, and the youngster can complete the unit even in the following year. They would obviously not get a certificate for the course in August, but they would get one for it the next time that we issued certificates. That flexibility exists in the system, but, as Billy MacIntyre said, we need confirmation of the data at every stage. We need to talk to colleges and schools about those flexibilities and build them into the system.
That is right. However, such complexities and permutations offer massive opportunities for something to be done incorrectly and, if I were you, I would be delighted to return to the former system.
We are acutely aware of those opportunities.
An aspect of the system that was not available to candidates in the old system is the scale of entry that Bill Morton and Billy MacIntyre mentioned for intermediate 1 and intermediate 2. There was no coherent provision for that level previously.
Absolutely. That is the virtue of the higher still programme. I accept that.
I have a final question about staff training. Last year, there was a feeling that the people who were looking at the computer screens did not know quite what they were looking at and that the grading numbers from 1 to 5 or 1 to 7 meant different things at different levels—at standard grade, higher and so on. Even after the results debacle, when people were on the phone to someone at the SQA, numbers were misinterpreted at the SQA end, perhaps because the staff were unfamiliar with what they were doing. Have efforts been made to ensure that that situation will not be repeated?
Yes, but there is no complacency about it. We have replaced the former operations unit with the new data management certification unit and we now have a team that deals specifically with data that come in from the schools, colleges, training providers and employers, and with software development testing and certification.
The schools data processing team contains four elements, including people who deal specifically with the schools that provide information to us electronically, on paper or by computer file. There is a team that assists with all that. We have not only established a better distribution of labour within the organisation, but increased our staff, strengthened our supervisory management structure and strengthened the management structure above that level. In Dalkeith, we are piloting the idea of a location manager. Everything that pertains to the way the staff feel about how things are going in their specific location is dealt with by the location manager. That idea shows signs of significant progress and, if it proves its worth, it will be implemented throughout the organisation.
We are also simplifying the processes and working alongside the staff to ensure that they understand all the processes. There is still the odd glitch, whereby a piece of data comes in that should be logged and receipted, but is receipted before it is logged. However, we are trying to put in place mechanisms to ensure that the proper training and support is provided. Jean Blair may want to say something about our training activities. I will not pretend that, since last summer, we have completely overhauled the skill levels in the organisation through universal training. Training must be customised and targeted to the areas that matter most and that is what we have been trying to do.
We need to examine training in more detail. The specific problem that you mentioned arose in the immediate aftermath of certification in 2000, when candidate helplines were set up and people phoned in asking for confirmation of the results. Plans are afoot to recruit and train staff to operate those candidate helplines, as well as the schools and colleges technical helplines. Our plans for that are well under way, and training will be a big part of them. We must ensure that the right information is given when candidates phone in.
Let us move on to the topic of the exam timetable. A pupil has informed me that he faces examinations in drama and modern studies on the same day, 1 July. Is there a possibility that the exam timetable is too compressed? Is that a genuine problem?
Dennis Gunning will answer that, but I should point out that there have been long-term preparations for the exam timetable and we have consulted widely on the matter. In September, we consulted on the fine tuning of the timetable for this year and, as a result, we brought forward to the first week some higher still exams to try to deal with the problem. In essence, however, if the teaching time is extended, the exam timetable will have to be shortened so that the exams can be undertaken at the end of term.
In a sense, this is a no-win situation. The problem goes all the way back to 1996, if my memory serves me correctly, when we consulted centres on how the new timetable, which would operate with the new national qualifications, should operate. We asked how long the timetable should be, what its structure should be and so on. The dilemma that we face is that, although a shorter exam timetable lengthens the teaching time that is available, it also increases the chance of a person experiencing Brian Monteith's constituent's problem, which is exams falling on successive days or on the same day.
The model that we construct for the timetable takes account of popular combinations of subjects. One would not expect physics and chemistry to be timetabled together because that is a common combination of subjects. However, it is more difficult to take account of such considerations with a short timetable. For 2001, we consulted on the possibility of opening up the first week of the timetable—in which only standard grade exams were scheduled—to some of the new national qualifications. That will ease the pressure, but only marginally.
We are in the middle of work on the 2002 timetable. Obviously, the timetable must be published so far in advance that it is difficult to make adjustments late on. We have consulted on a proposal to extend the timetable to six weeks, instead of keeping it at four weeks. So far, the opinion of our schools and colleges is divided—as their opinions on every consultation on the subject have been. Some say that the timetable must be kept at four weeks to lengthen the teaching time and others say that it must be stretched to six weeks to reduce pressure on candidates. We must find a way of squaring that circle by next year. We plan to pull together a small working group of teachers, lecturers and—importantly—a couple of invigilators, who face daily pressure when the exam diet is running. The challenge is to re-examine the timetable model using the outcome of the consultation to determine whether the model fits the new national qualifications.
In a sense, parts of the timetable still look back to the old highers. Next year, we will be helped by the fact that there are fewer exams to accommodate. One of the problems over the past two years has been the fact that the old higher, the new higher, the certificate of sixth year studies and the advanced higher were being examined at the same time. Pressure will be eased next year, but we still need some kind of consensus on whether we should shorten the timetable and increase the teaching time or lengthen the timetable and reduce the pressure on candidates. I hope that the working group can come up with a formula that might help us with that.
Do you allow for a mechanism that brings the pupils into the consultation?
We have not allowed for that in the past but one of the interesting experiences arising from the ministerial review group is that we have received direct feedback from the pupils on that group. As you can appreciate, it is difficult to get a representative view from the candidates, their parents and other interested parties. That is why we tend to canvass their opinions through schools and colleges.
Recently, you have completed the independent assessment of the appeals process. It has been brought to my attention that some difficulties still need to be resolved. I understand that an appeal by pupils from—ironically—Bearsden had been refused. The pupils felt that they had received the wrong mark for their human biology exam because it had been marked as if it were a higher, even though the pupils' course was a higher still course.
Can you quantify the extent to which such problems still need to be resolved? Can such problems be resolved speedily?
The independent review of appeals was run—with our administrative support—by the Scottish Executive. As I understand the results, the review demonstrated that our appeals process is robust.
Teachers and lecturers assess the appeals, not the SQA; it acts merely as a broker. It is difficult to comment on the independent review specifically, because it is not really our problem but, in the best interests of the candidates, I—as chief examiner—have undertaken to accept the findings and have them certificated.
The two pupils that Brian Monteith mentioned attend Clydebank College, I believe. I ask Jean Blair to comment further.
I understand that the appeals were submitted using the wrong estimate bands. Staff must be better informed about bands and we must ensure that we give out the right information about bands and so on.
We believe that the correct information was conveyed to the college in December.
The date for the issuing of certificates has been moved by a week. You mentioned that deadlines are an absolute term and that it is important that the job is done right, rather than it being rushed. With the new deadline, however, it seems that you must still be both right and rushed.
You will be familiar with Parkinson's law that work expands to fill the space that is available to it. Is there a danger that the extra week will not be used to provide extra time to build in extra checks and so on? Is there a fear that the extra week will simply be swallowed up?
There might be a fear of that but, unlike in my response to Michael Russell's earlier question, I am prepared to give a categorical assurance that that week will not be swallowed up.
The extra week includes a weekend and has been included on the principle that we should get the job right, not rushed. There is no fixed date for the issuing of certificates. We felt that by extending the period for processing and verification by a few days, we could ensure that the certificates would go out completely and accurately. I hope that the candidates will not regret that investment of time.
We consulted the major representative bodies throughout the education community. Three quarters of the consultees supported the move to certificates on the 13 August, which would mean that they arrived on 14 August. The SQA will use the extra week to full effect and with due diligence.
I should add that the real deadline is 7 August, because that is when the results are sent to UCAS and the higher education processing begins. The aim of having that deadline is to allow the universities to brief admissions officers and prepare offers so that, when the candidates get their results the following week, university staff are ready for the thousands of phone calls that are made the minute candidates get their certificates. The process is carefully staged.
As Bill Morton said, our best judgment this year was that the extra weekend—of printing time rather than processing—would be important to us. If we are confident about the way that things are going next year, 2002, we will move the date back again.
You mentioned the further education sector. A growing feeling among members is that the FE sector is becoming a cinderella part of the SQA organisation. A concern is that a number of students have still not received their certificates from last year. Many people who go back into further education are doing so after a negative experience of education. If they do not receive their certificates, which affect what they can do in future, that is another negative experience.
What are you doing to ensure that the FE sector is brought into line in the same way as school-based candidates?
That is an important issue. I will ask Dennis Gunning to expand on my initial response.
If one speaks to the school lobby, one will hear that the SQA is biased towards further education, but if one speaks to the further education sector, one will hear that we are biased towards schools. We are trying to give a much-improved service to our customers, and we are trying to do that equitably and across the qualification range.
We have been working hard with the colleges. I have thanked the schools and the teachers and I must also thank the colleges—specifically those that are members of the Association of Scottish Colleges, although I thank them all, whether they are members or not—for their patience and support. We have had to do a major reconciliation of the data, because there was a series of duplicate numbers. That can happen when a pupil who has a candidate number from school forgets that number and gets another one; the same candidate can have two or more numbers. That must all be sorted out through a data reconciliation exercise, which is on-going; Dennis Gunning will be able to tell the committee where we are with that.
Since 1 January, we have issued about 26,000 certificates to that important group of candidates.
The matter has several aspects. One is that the data reconciliation exercise reflects the fact that the problems that we had last summer affected all data, not only data for school candidates. We have been working with colleges, especially with the Association of Scottish Colleges—as Bill Morton said—to do a data match exercise with every college. That is now complete for 42 colleges, although there might still be one or two errors to pick up from it. We are well on our way through the process of matching the data from last year. In the period to 14 March, we issued more than 28,000 group award certificates for higher national certificates, higher national diplomas and Scottish vocational qualifications. The process of trying to work our way through last summer's problems is under way with colleges, but we have a bit to do yet.
We also try to meet the specific needs of the FE sector. There are about 150,000 candidates for SQA qualifications in our system from that sector each year, so it is an important sector for us. Next winter—in January 2002—we will run the first limited diet of examinations in a few subjects, primarily to meet the needs of colleges for flexibility of timing. As members know, not all college students finish their programmes in the summer. We are also working on the development of project-based courses in the new higher still system, primarily in vocational areas, which is meeting colleges' needs. One of our priorities is to balance the needs of these two very different sectors. Our aim is to meet the needs of both.
I will correct an error that I made. I think that I said that we have issued about 26,000 certificates. Dennis Gunning is correct; we have issued more than 28,000 group award certificates.
Although we have certificated 28,000 group awards, we must still issue certificates for candidates who have received free-standing units. We will not be in a position to certificate those candidates until we have completed the full reconciliation between the data that we hold and those that the colleges hold. We are intent on releasing the remaining certificates as soon as possible.
I hope that you accept that that situation is not acceptable and that it should not be repeated next year. It is unacceptable that students have still not received their certificates, almost a year after they sat the exams. That would not have been allowed to happen in a school-based situation and it should not have been allowed to happen in the FE sector.
You are quite right; it is not acceptable. That situation is largely a product of the reconciliation of data between the SQA and the colleges. In some instances, few in number, misunderstanding can also be an issue. Somebody might have completed a course, but the course entry suggests that the completion date is a year later—that point triggers the certification. We must have a more responsive recording in the system of when work is complete and certificated.
Does not that confirm the scepticism in the FE sector about the relationship that it has with the SQA? Several principals with whom I have dealt are concerned that the door is closing on the role that they play and that they might be asked to deal with more vocational qualifications—they are not comfortable with that. I hope to hear some reassurances on that matter.
Secondly, as the clear objective is not to repeat last year's situation, when do you expect to have cleared the hurdles? It is now March, so when would the alarm bells ring so that you would know that there was a problem—perhaps not on the scale of last year—and could consider plan B?
A lot of teachers in the college that I am in contact with are still concerned about the information flow and inaccuracies. It will be helpful to know Bill Morton's views on that.
Mr McAveety is quite right and I heed what he says about getting much closer to college principals. They have been very patient and supportive. We need to improve our communications but, more important, we must deliver the service that they have every right to expect from us. I concede that there have been problems, but we are working hard to ensure that the situation is back on track.
Sorry, I have momentarily forgotten your second point. I am being distracted by the musical accompaniment from the piper outside.
You should try listening to it every day.
We are in the middle of March and already we are talking about delays. When will the alarm bells ring?
I cannot put a measure on how much better the management information is now than it was, because—as I understand it—there was previously no reliable management information. Nobody knew at what stage the SQA's process was, or which problems were compounding other problems. We now know those things. In the interests of what is a good and understandable discipline, and given what happened to us, we are attempting to be much more open so that we are accountable. We report on all the management information that we produce so that we can track where we are.
We have a traffic light system; as the committee would imagine it uses the colours green, amber and red. It flags up the stage that we are at in each of the processes and it highlights the extent to which we must focus on an issue to fix it. That information is designed to be preventive, although it has a curative element. If we have concerns, we openly and honestly flag them up. We report to the Scottish Executive, to the ministers and to the ministerial review groups. We share that information with the major education community stakeholders and with the public, via the media. As the information spreads there will be a better understanding of the progress that we are making on behalf of candidates. If any matters were to reach the alarm stage, the SQA would be the first to express concern.
I have two specific questions and one general one. On the further education sector, let me read out an e-mail that I received this morning. I cannot name the college, and it will become obvious why. The e-mail begins:
"Heard Bill Morton on Radio Scotland this morning doing the usual oil on troubled waters routine".
It goes on to say that, at that college,
"and I assume this is the same the length and breadth of the country for FE colleges at least, the SQA still have not got last year sorted.
At the tail end of the last academic year (July 2000), the last few days were spent re-registering students for SQA NC modules and HN units for students that the SQA system had lost.
We are 8 months into the 2nd year of their new system and many of those 2000 students have either received no certificate or only part of what they did is listed.
SQA have passed the administration work for last year back to the colleges for us to sort out entries missed, errors made etc.
A neat twist is that as colleges have been registering students for this years work from September 2000, SQA have mixed this years students and last years students together which has made trying to administer the system even harder ... it passes the time!"
That e-mail came from somebody who works in a college and who is clearly immensely frustrated. What the convener said is absolutely true: it simply will not do. Give a brief answer, please. What are you going to do to ensure that the problem is fixed and that it never recurs?
I am disappointed that people believe that, when I talk about what we are trying to do, that is pouring oil on troubled waters. That implies a degree of complacency that I do not feel. It also implies that we are less accountable than I feel we are, especially today. If there are outstanding issues in relation to a particular college, and if Michael Russell will name the college to me privately after the meeting, I will give an undertaking to examine the matter to see whether a resolution can be put in place quickly.
I have talked in general terms about the issues around data reconciliation. I have talked about the extent to which that has constrained the recertification process. It is very difficult to respond to a general question other than by being general. However, if Michael Russell can give me the specifics of the problem that he cited, I shall give an undertaking on my intent to fix it.
I have a problem with what has happened this afternoon and I must say this quite straight. Many of us supported your appointment last year and said, when we met you on a number of occasions, that it was essential that changes were made. I am sure that the SQA is making those changes. They are immensely detailed and all your colleagues know a great deal about the detail. I could go through a list of issues that I think should concern the committee. For instance, the SQA's redesigned certificate is a mess and has caused quite a lot of resentment among people who have seen it. I do not see much point in that redesigned certificate.
There is a matter to which all this boils down. I can understand and speak with some sympathy about Sam Galbraith's difficulty last year, if he was receiving information without being absolutely certain whether it was right or wrong. The big question—which needs an accurate answer that not only we, but every young person in Scotland, can understand—is this: what went wrong last year and has it been fixed? There is no other question. I am sure that the SQA is doing its very best and trying hard but, unless you can answer that question, we will all leave here immensely dissatisfied and very worried. The press that we have talked about will continue.
There is no answer to that question. Michael Russell knows, perhaps as well as I do, the answer to the first part of the question—on what went wrong—because the committee conducted an inquiry that came up with a series of findings and recommendations. It would be fair to say that we could agree without too much difficulty that there was an expansive range of issues to address. The expectation that they can all be addressed in one year is not realistic. That means that we must focus on the basis of priorities and fix them one at a time.
Is the problem fixed? It is being fixed, and I can guarantee that we will do absolutely everything in our power to ensure that it is fixed sufficiently for a successful certification in the summer. The question that Michael Russell has asked is, in effect, the guarantee question by another route. I have already attempted to explain that. I have no desire to further disquiet candidates. If anything, I try to use any platform that is afforded to me to reassure candidates about the progress that we are making on their behalf.
I believe—I would not otherwise give this guarantee—that we are on the right track. We are making the right sorts of improvements, but it is up to others—ultimately, the candidates—to judge whether those improvements have been successful when it comes to certification in the summer. I am sorry that I can go no further than that.
I am encouraged by that, because I think that we are getting closer to the nub of the matter. However, I am sure that you understand the relationship between the difficulty that you have in saying those things and the press coverage that the SQA is getting. You have found that coverage to be inaccurate and speculative. Mr McAveety has referred to insiders saying this and that—we even get that in political parties, believe it or not—but the reality is that the press wants you to succeed and we want you to succeed. There is a lot riding on this and the SQA will have to understand relations with the press better.
I fully understand that. I would never criticise the press for raising legitimate concerns on behalf of others. I am quite happy to accept that, in attempting to be more open, we are accountable to the public, and that the media are among the vehicles we can use to put across that information. I am concerned that we cannot speak directly to the candidates to reassure them; we are addressing that and trying to find a way of getting information directly to candidates about the SQA's preparedness on their behalf for the summer. We will do our best to ensure that that information is reassuring.
Michael Russell said that the certificate is a mess. Those who have been consulted on that—through the ministerial review group, for example—do not seem to suggest that that is the case.
I am sorry. I meant to finish that point. I have an e-mail in front of me from somebody who is deeply involved in that matter. It states:
"The certificate remains a mess—it is still 5 to 6 pages and we have established that the simple looking certificate—called a commemorative certificate—is essentially there for framing".
I looked at it at the weekend and again today, and I do not think that there has been a significant change or that the certificate meets the requirement of the committee's report. I note from the Executive's response that the Executive also says that there is still progress to be made, so I think that the SQA needs to take the certificate back. The purpose of changing the certificate, according to the committee's recommendations, was to ensure that young people and employers understand the nature of their results. The certificate still does not do that.
You must bear it in mind that the same certificate serves a number of purposes and that it does not merely award the qualifications in the school sector. The inclusion of units, the core skills and the cumulative nature of the certificate are valued in the vocational arena and particularly by employers. We have consulted and we have endeavoured—as far as possible—to revise the certificate to take account of that. Does that mean that simplification of the certificate stops at that point? No. If further improvements are required, we will certainly look at those very positively.
More improvements are required.
You mentioned the committee's inquiry. Indeed, there were three inquiries following last year's difficulties, all of which spent a great deal of time and effort trying to identify the problems and offer solutions. To what extent has the SQA based changes on those recommendations? How do you respond to criticisms that the SQA has been too conservative in the changes that have been made and that it has not embraced enough modifications quickly enough to make a big enough impact on this year's results?
There were four inquiries—including my own, which was first into the frame and the first out in terms of recommendations and implemented improvements and changes. It is a matter of fine judgment. There can be many people outside the organisation making recommendations about what should be done inside the organisation.
However, as chief executive, I have slightly different accountability. I must make judgments about the degree of change that the organisation can take. I must make those judgments in the context of what is absolutely necessary to ensure that we produce the outcomes in the summer, which candidates and their parents and families have a right to expect. I made judgments—as I am sure the committee will understand—that were based on the interests of the candidates, which took account of the whole span of the recommendations and of work with the staff and the board. We have a long way to go yet in terms of communications, but we are making some progress. We are trying to listen as much as possible to the schools, colleges, directors of education and teaching unions.
If the committee surveyed those interests, members would find that they all have a view on how the SQA should be managed. That judgment is mine; I make it and am happy to be accountable for it. The term "conservative" might be used, but I prefer to say that the changes that we have made are right, pragmatic and designed to do what we need to do. They are being implemented on a planned, prepared and risk-assessed basis and we have contingency plans in place. We are accounting for all that through regular reporting. Our board and internal auditors are helping us to ensure that we remain objective and that we do not lose sight of what we are trying to achieve.
Do members have any brief final questions?
On an entirely different tack, the crucial factor this year is getting the 2001 diet completed successfully and on time, but you have said that a five-year plan is crucial to the medium to long-term success of the SQA. Do you accept that there will have to be a fundamental, more philosophical, review of higher still once the 2001 diet is completed successfully and you have learned the lessons of two years' running it and seeing how it is operating and being accepted? The SQA might have a role in deciding whether further changes are made to the qualification to improve its acceptability to pupils and employers.
I will make two quick points and then pass the matter to Dennis Gunning. I can give you only a personal view.
You are quite right that the SQA has a very sharp focus that dictates the amount of change that is necessary and achievable: the successful delivery of certification in 2001. However, that is only part of my responsibility to the SQA. It is clear that dealing with the broader issues will take up to three years or maybe longer—we have said so publicly. The SQA has to reinvent, rebuild and reposition itself. Believe you me, the organisation appreciates that the restoration of trust and confidence has to be earned and cannot be presumed. In parallel with the planning and preparation of the detail necessary for delivery in the summer, work has started on the larger organisational development exercise. Perhaps in the future we will have the opportunity to explain that work in more detail as it evolves and matures.
I will make one comment on higher still. From the pattern of entries this year and the growth in intermediate qualifications in particular, it seems that the higher still programme is tangibly working. We are interested not so much in the political and policy aspects of higher still as in the practicalities.
Higher still was a long time coming. Some of us have been around long enough to remember the setting up of the Howie committee and everything that flowed from that. In a sense, this is an area in which we have to be careful. A huge amount of consultation was undertaken—people joke about it being the greatest consultation in the universe, but it probably was. We have to be careful that the amount of thought, time and patience that went into the building of higher still is not swept away in a reaction that may mix up the first year of implementation of higher still with the problems that the SQA had. I think that Bill Morton is right: there are clear signals in terms of uptake that people are beginning to use the flexibility and breadth of higher still, not just at intermediate 1 and intermediate 2, but at access level, where there is coherent provision for the first time.
On where we go next, reviews have been undertaken by the Executive in conjunction with the Association of Directors of Education in Scotland and the Association of Scottish Colleges. Also, the Educational Institute of Scotland and the Scottish Secondary Teachers Association commissioned a review from MORI. Those reviews are providing evidence that is being considered by the national qualifications steering group, which gathers all the key stakeholders around the table. The steering group is considering, first, the internal assessment issue, which is a big one, but I am sure that it will broaden out its work to include the bigger issues. What has emerged from those reviews is ambivalence about higher still—people see the benefits but they also see, at least in the early years, the down sides. We have to be careful to identify the down sides that may relate to getting used to something new—those of us who have been around long enough can remember the same issues arising when standard grade was introduced, as one has to get accustomed to a new system and then make a cooler judgment.
A comment that has been made consistently by students in surveys is that the use of units and internal assessment encourages them to work consistently through the year instead of doing a cram at the end. That view appears to have been felt particularly strongly by male students. When we do the analysis for last year, it will be interesting to see whether the gender gap in performance levels, which is depressing for us males, might have lessened as male students have been forced to take a more consistent approach to work. We need to be measured and to avoid knee jerks, but I am sure that higher still will not have got it right first time and will have to adjust over the years, just as standard grade did.
I welcome the tone and direction that you are taking. You know that I have reservations about elements of higher still, but I will not go into those too much. I welcome the fact that there is debate and the matter is not fixed.
In your opening statement, Mr Morton, you spoke about an action plan. Will that plan be available so that people can see who is in charge of what and when it is to be delivered? Will that information be made public or will it be available in the consultation groups that you are dealing with?
As we implement our action plan—I think that project plan is the term that the committee used in its recommendation to us—we are reporting on the things that we must to do, the extent to which we rely on others and the targets that we set. Each of those aspects is detailed in the report that is submitted to the minister, the ministerial review group, the Scottish Executive, the stakeholders, the staff and the public via the media. At this point, the report does not go into the detail of individual staff accountabilities, but all the other details are covered.
We are identifying and assessing risks very rigorously. I am sure that the committee will be reassured to hear that and would be very disappointed to hear that we were doing anything other than that. We are identifying risk on the basis of its potential to happen and its likely impact if it does happen. Where there is a risk, we are trying to put in place a contingency plan. The plan is dynamic and fluid, but it is starting to make a difference in the organisation.
We still have some way to go in rolling the plan out and enabling the staff to understand it. Some of the language that we use, such as describing the things that we need to do as critical success factors, is management speak that needs to be simplified. Fundamentally, we need to reach the point where staff believe that it is not just our plan, but their plan, and therefore that it is the SQA's plan. The communication of the plan could be continuously improved, but I do not know whether it would help to go into the specifics of who does what.
Mike Russell has, perhaps, been pushing you in a direction and I respect your decision not to make a full commitment in your reply. No exam system in the world is 100 per cent successful. If you make a guarantee and get it only 99.9 per cent right, someone will say that you did not do it. You would be exposing yourself to an impossible promise. On the other hand, we all hope that this year's diet will be a big step in the rehabilitation of the whole system. If I had £1,000 to place on whether the diet will go well or go badly, on which side should I put my money? What are the odds?
That is the most novel presentation of the guarantee question that I have heard yet. What Mr Jenkins decides to do with his money is entirely his own business.
You probably feel that that is the question Martin O'Neill is asked at every press conference; he does not want to admit it, but it is likely.
From everything I read, including the papers that have been submitted and media coverage, the SQA seems still to have a big communications problem. Instead of commanding the media waves, you are responding to a lot of direct punches. Do you have a communications strategy? I am thinking of a weekly strategy. I have an article, written earlier this month by the Scottish Secondary Teachers Association, which claims that there has been a breakdown in the culture of engagement and the relationship between the teaching profession, trade unions and the SQA. What is the SQA's message? Can you inform us about this massive issue?
Our message is that we are making progress and that we are doing what we need to do. I have tried to convey it to the committee today. I can guarantee that we will do everything possible to ensure that everything is in place for the diet to be certificated. Apart from repeating that message to the committee and the media, I cannot guarantee that that message is the one that will go out as the newsline. It is my hope and aspiration, however, that that will be the case. I would be disappointed if there had been an authentic breakdown in the relationship with the SSTA. We would certainly do all we could to ensure that that was not the case.
In the article, David Eaglesham claims that relationships have reached an all-time low. How can you re-enthuse David Eaglesham and others in the teaching unions?
If that is David Eaglesham's concern, I am happy to go and speak to him to try to resolve that situation. A short while ago, the SSTA executive met the SQA's senior management team, which is present today. We agreed that more information should be shared and we try to do that on the basis that the SSTA helped us. I am happy to repeat that exercise if that is what it takes to restore the relationship, but I am not aware that it has reached the stage that David Eaglesham appears to fear.
I do not want to be seen to be getting solely at you. I know that you are not solely responsible, as a structure of relationships is involved. What worries me, and grieves me as a long-term EIS member to say, is that a significant education stakeholder such as David Eaglesham has put such a strong statement into print. We should be aware of those concerns and see how they should be addressed.
You would have to do what I would have to do and that is to ask David Eaglesham. The only way I can address and improve on the relationship is to deal with his specific concerns. I am happy to do that.
Frank McAveety's point is extremely apposite and it echoes one that I raised earlier. Both of us are saying the same thing: that you have not got your media relations right yet. You may feel that you are being treated unfairly—we always feel that we are—but the reality is that your relationships are not right. It is crucial that you get them right and you have to take action this week to do so. It alarms me that you did not know about that article. If you did not, you should have done.
I read the article in The Scotsman. I knew about it.
But when you responded you said, "If David is saying such and such." Surely, if David Eaglesham says something in an article, the correct media strategy is to ring him up there and then and say, "We need to have a chat about this. Let's have a chat." You need professional media managers in there now. Someone may make money out of you, but it is better that that happens than that this situation continues.
Do not ask us to do it.
That was not a pitch. Frank McAveety and I will do it jointly.
I have been in touch with David Eaglesham and with many other people in similar situations in the past. I am happy to do so again.
I am sure that you appreciate that our main priority is for you to deliver diet 2001 accurately and on time. Our other concern is for improvement in attracting markers to remain under control. Along with ministers, we have a duty to ensure that the SQA is accountable. We need to continue to do that in the best interests of Scottish education.
You have given us a greater insight into the situation. That is helpful. It is important that we see directly what is happening. The committee believes that it would be useful for us to visit Dalkeith and Glasgow to meet staff and to talk to them about some of the issues.
You have been unable to give us the definitive guarantee that every committee member sought in questions of one form or another. It is therefore important that we keep that issue under review. I suggest that we look for a date, perhaps in mid-May or towards the end of June—depending on the exam schedule—for you to return to the committee. I also suggest that you provide us, at some point in the near future, with the definitive guarantee that we seek.
I am happy to return. I thank committee members for the opportunity to appear before you today. I welcome the committee's interest in visiting Dalkeith and Glasgow. We have touched on staff morale. Knowing that members of the Scottish Parliament are interested enough to come and visit them would boost staff morale. I am happy with that suggestion and would actively encourage the committee to do just that.
We will put that in process. I thank Bill Morton and his staff for appearing before the committee today. I am sure that we will see you again soon.
No doubt.
Meeting adjourned.
On resuming—