Section 23 Report
“Managing ICT contracts”
The first substantive item on the agenda is the section 23 report “Managing ICT contracts: An audit of three public sector programmes”.
I welcome to the committee for the first time the new Auditor General for Scotland, Caroline Gardner. She has with her from Audit Scotland Gemma Diamond and Ronnie Nicol.
I hand over to the Auditor General to introduce the report.
Caroline Gardner (Auditor General for Scotland)
Thank you, convener. It is a pleasure and privilege to join you on the committee. We look forward to working with you in future.
The report that members have in front of them this morning reviews the management of three information technology contracts—at the Registers of Scotland, the Crown Office and Procurator Fiscal Service, and Disclosure Scotland—that all experienced delays or cancellation. To put the matter in context, the programmes have cost a combined £133 million so far.
We all know that IT projects are complex. On the one hand, they can make a really significant improvement in public services and contribute towards efficiency savings, which have never been more important. On the other hand, they throw up significant risk, which needs to be managed carefully to avoid the loss of public money and delays in much-needed improvements in public services.
In the three IT programmes that we reviewed, we identified not only some technical challenges, which are common in IT innovations of such scale, but a number of weaknesses in basic project management, which are of particular concern in the ICT programmes involved and which came to light for us in three broad areas.
The first area was the quality of the business cases and the options appraisals that were carried out. They all varied and, in some cases, the expected benefits were not clearly defined. For example, in the Crown Office, the business case lacked detail on the benefits of the new criminal case management system that was being put in place and did not include information about the full costs of the project.
Secondly, we found that governance arrangements were not effective. For example, in relation to the Registers of Scotland ICT programme, there were agreed procedures to raise issues with the partnership board but those procedures were not always followed in practice, which meant that the concerns that have been identified could not be acted on and rectified.
Finally, we found weaknesses in financial control and reporting and inadequate risk management. For example, at Registers of Scotland, the individual projects that made up the programme lacked detailed costs, benefits and milestones. A lack of delegated budgeting meant that the programme managers did not have responsibility for the programme budget.
We found that a failing across the piece was a lack of in-house skills and experience, with the result that the public bodies concerned failed to recognise the complexity of the programmes that they were trying to manage. That manifested itself in a number of ways, such as a failure to adequately manage the interdependencies of the projects that made up the programmes. We saw an example of that at Registers of Scotland, where senior managers failed to recognise the implications of changing the order of two of the projects that made up the programme without understanding the associated impact and risks.
We also found a lack of an intelligent client function in the bodies responsible for the programmes, with the result that both Registers of Scotland and the Crown Office were unable to challenge suppliers as necessary when problems began to emerge. We also found a failure to understand the findings that were identified through the independent assurance procedures that were in place. For example, at Disclosure Scotland, the protecting vulnerable groups programme team did not fully understand the high levels of risk that they were facing with a lack of testing before implementation, even though the issue had been highlighted in the technical assurance reports that were available to them.
It will always be difficult for small organisations to have the in-house specialist skills and expertise that are needed to implement complex information technology programmes. They do not do that work very often, and the scale of the investment is often significant in relation to the size of the organisation.
Our recommendations, therefore, focus on the need for the Scottish Government to strengthen its strategic oversight of investment in ICT to ensure that problems can be identified and rectified early when required. The recommendations include that an assessment should be carried out of the skills that are needed to deliver complex ICT programmes across the public sector and that consideration should be given to whether there would be benefit in establishing a central resource for public bodies to draw on when they need it.
We also recommend a review of the purpose and use of gateway reviews to ensure, first of all, that they are being properly applied and, secondly, that they give integrated assurance about the range of risks, including financial risks, that such investments can involve. We recommend widening the role of the Scottish Government’s information systems investment board, which could perhaps play a greater role in reviewing the skills that are available to individual bodies undertaking significant ICT programmes, monitoring progress and providing access to specialist support and expertise when those are needed.
The issues raised in the report and in other high-profile ICT programmes that have been in the media recently highlight that the area continues to be one of significant risk for public bodies. We will continue to keep it under review through our audit work, which is carried out on my behalf right across the public sector, and in local government on behalf of the Accounts Commission.
The team and I are happy to answer any questions the committee may have.
Thank you. I have a couple of questions about the report and how the issues were pursued.
The report refers to three specific cases in which procurement of ICT has not worked out well. That could be read in two ways. One is that there is an endemic problem, of which these are particularly bad examples. I suppose another is that things have gone wrong in these three cases but that they do not necessarily tell us anything about what is happening more widely. Why was the report developed on the basis of these three specific instances? In Audit Scotland’s view, are they typical of weaknesses right across the public sector?
That is a very good question. The background is that our auditors look at significant investments in ICT programmes in all the 200 or so bodies that we audit across central Government, the health service and local government. The three programmes came to the fore because they were highlighted in last year’s audit reports both as risks and as areas in which significant problems were being encountered.
A year ago, my predecessor, Bob Black, brought to the committee section 22 reports that highlighted the problems, and this report is the result of more detailed investigation into what went wrong. Other recent examples in local government include the problems in Highland Council that have been investigated on behalf of the Accounts Commission. That also happens in other bodies when problems reach a level at which public reporting becomes the right response.
At the same time, we are monitoring other significant investments through our audit work. For example, both the City of Edinburgh Council and NHS 24 are investing very significant amounts of money in transformation programmes that depend on ICT, but I must point out that although those are very challenging investments there is no indication at this stage of any significant problems. The checklist of good practice at the back of the report sets out the sort of questions that we think board members and councillors in the bodies involved should be asking to assure themselves that they will not face such problems in future.
Is it fair to say that there are other examples of emerging problems or, more particularly, of imminent risk in procurement projects that might be informed by the report?
You have already mentioned one or two examples; another recent example that springs to mind is the IT project that is being undertaken by the Association of Chief Police Officers in Scotland. Press coverage has suggested that, because ACPOS is a limited company, Audit Scotland has no role in examining that project. Is that the view of Audit Scotland? Indeed, is it your view as Auditor General?
On your first question, the public sector across government spends about £740 million a year on investment in IT. As it is a complex area with inherent risks, our auditors monitor and review significant investment to ensure that it is being managed well, and the questions in our checklist are intended to help the bodies themselves identify problems early.
As for the investment by ACPOS that has recently been in the media, we have in the past reported on concerns about accountability arrangements surrounding the association, which, although it plays a significant role in policing in Scotland, is not a public body with the usual arrangements in place. We have been investigating where the money that ACPOS has spent on those programmes has come from; indeed, we have been working closely with Her Majesty’s inspector of constabulary, Andrew Laing, on that. At the moment, it looks as though the most significant element of that investment has been central Government funding through the efficient government fund; if that is the case, we can audit it and look at how the money has been spent and what has been achieved.
We are also talking to Her Majesty’s inspectorate of constabulary in Scotland about carrying out joint work on investment in police systems more generally in the run-up to the new Scottish police service, which will depend significantly on good use of IT. In that regard, ensuring that such investment achieves the intended benefits will be even more critical.
That is very helpful. A related point is that the three cases that are examined in the report are Government agencies. In your opening remarks, you made the point that, looking forward, we will probably need greater central support from Scottish Government; after all, because the agencies concerned are small and carry out such procurement very infrequently, they will not have expertise in the area. In the case of Disclosure Scotland, IT procurement was carried out by the Scottish Government because at the time the body had still to be set up. Given its culpability in one of the cases covered in the report, can we be confident in the Scottish Government’s capacity to do this work better?
There were particular challenges around the protecting vulnerable groups system because of the history of the setting up of Disclosure Scotland. The investment in IT began before the organisation was established in legislation; the fact that the Scottish Government kicked it off and then handed it over to Disclosure Scotland led to confusion about who the client was and a lack of ownership over the system’s user needs. Although lessons can be learned about what should happen in such circumstances, I do not think that that particular example can be read as a failing of the expertise that exists in the Scottish Government.
Nonetheless, paragraph 78 of the report points out that the Scottish Government directorate with responsibility—the information services and information systems directorate—has lost 30 per cent of its staff over the past two or three years. Does that pose a significant risk that the Scottish Government will not be able to put in place the support that the report suggests is required?
10:15
We cannot give you an assurance that the Scottish Government can provide that assistance. One of the bodies that is involved in the programmes covered in the report told us that it had requested further support that the Scottish Government was not able to provide. That is the reason for our recommendation that the Scottish Government should review its ability to provide support and assistance to the bodies that need it, and provide strategic oversight of the way in which those bodies are being managed and the general progress that they are making.
I think that the convener has taken all my questions—
That is why I get to be convener.
That is why I am only the deputy.
Government support is a critical issue. Staffing levels have been reduced, and you say on page 6 that there is a “lack of specialist skills” and that
“The ... Government was unable to provide the three ... bodies with all the advice and support they sought”.
Although we can point the finger at the three organisations, I am concerned that, while Scotland is moving towards tax collection and so on, the Scottish Government is, in your words, “unable to provide” what organisations are looking for.
You have asked the Government for assurances, but how does that give us confidence that there is a system in place so that when organisations ask for help, that help will be there?
I felt that your report was extremely critical, not only of the organisations involved but of the Government.
We think that there is a clear role for the Government in maintaining oversight of what is happening, particularly in the agencies and non-departmental public bodies that carry out central Government responsibilities at arm’s length. There would be real merit in the Government looking at establishing some core expertise so that it can provide support and advice and perhaps intervene when problems emerge, as it did with the three organisations.
It is for the Government to decide how it intends to address our recommendation, but we certainly think that there is a role in that regard that is not currently being fulfilled, and that there is an inherent risk with smaller bodies in particular trying to manage very complex IT investments on their own.
The main criticism of the Government was that it was “unable to provide ... support”, and the second criticism was that it could not provide advice and support on time. Why did the Government get rid of 100 staff? Is that what has led to this situation? Is there a problem with expertise or recruitment?
Would you recommend that the Government revisits its staffing reduction from 300 to 200? Is that what has caused the problem, or is it caused by other things? For example, paragraph 30 states that Registers of Scotland
“had agreed ... 400”
changes
“at a total additional cost of £21.7 million.”
We need to find out where the blame lies in order to try to put things right. Were organisations being reckless in their demands for changes? I know that the changes covered a period of eight years, but there were 400 of them. The organisation was obviously not getting what it was looking for, and the Government became involved in that contract only one year ago. There was also an additional cost of £21.7 million. While £9 million has been written off, there are a lot of other millions in the report that have been either wasted or not planned for properly.
Is the fault due to the Government reduction in staff, which meant that it was unable to respond or to do so on time? Could the fault be laid at the door of Registers of Scotland, for example, for not understanding the contract or for requesting 400 changes? That must have been a nightmare to deal with. Can you give some clarity on those issues?
I will ask Gemma Diamond to provide more detail about the Scottish Government’s involvement, but it is important to note that I do not think that it is possible to say that one cause or one body is responsible for the situation. It is clear that small bodies will always struggle to manage investment on the scale that we are talking about, and Registers of Scotland had a very big contract.
It is also clear that, in our system of arm’s-length bodies, those bodies have a responsibility to ensure that their governance arrangements are such that they can spot problems and ensure that they are acted on quickly. It seems that neither side of that equation worked well in this case.
With your permission, convener, I ask Gemma Diamond whether she can add more detail.
Gemma Diamond (Audit Scotland)
The Scottish Government was involved in different ways with each of the three contracts and was asked for different kinds of help at different points. Mary Scanlon is right, in that we have said that it was unable to provide all the assistance that it was asked for.
Disclosure Scotland asked the Government for ICT help, but that help was not available and it went out to a private sector supplier to get the support that it needed. It is not that there was a gap; it is just that someone else had to fill the gap as the Scottish Government was unable to provide help at the time.
With the Registers of Scotland contract, problems came to light only about halfway through the contract, and it was following a gateway review that the Scottish Government became involved with the problems that had arisen.
That was seven years after the contract started.
Yes.
So it took seven years to identify an additional cost of £21.7 million, 400 changes and other problems. That worries me.
At Registers of Scotland, there was a gap of six years in which no gateway review was undertaken. Gateway reviews should be undertaken periodically throughout a long contract, but the person who was in charge of the project at the time postponed the gateway review without letting the accountable officer know. It was only when that came to light that the review was undertaken.
I will make a comment on the back of the convener’s questions about whether the situation is happening elsewhere, on the back of the millions that have been lost and on the back of the other millions that still have to be negotiated.
I would like to think that we could leave the report and say, “Well, lessons have been learned. We can all move on. This is undoubtedly an excellent report.” However, in paragraph 70, you state:
“The findings of each of the ‘lessons learned’ reviews have been provided to some parts of the Scottish Government. But there is currently no mechanism to ensure that the learning is passed to all parts of central government and the rest of the public sector, and that public bodies have access to the appropriate skilled resources to be able to implement the learning.”
We do not even have an assurance that lessons have been learned and that everyone is going to do things differently in future. In the report, you state that there is
“no mechanism to ensure that the learning is passed”
on. That is incredibly damning, given that £740 million is spent on IT.
That is why our recommendations are focused on the Scottish Government and its ability to assure you that it has proper strategic oversight of the money that is invested, that it has the resources to provide support and advice where they are needed and that it can intervene where problems emerge. We share your concern—I think that that is the message of the report.
But it is not even about intervention. I have moved on from that. What I am saying is that lessons have to be learned because we cannot afford to waste money like this.
I repeat:
“there is currently no mechanism to ensure that the learning is passed to all parts of central government”.
How can Government not manage to say, “We’ve had a few problems here. Can we all get together to discuss it? Can we do this differently in future and ensure that taxpayers’ money isn’t wasted and written off?” You have said that there is
“no mechanism to ensure that the learning is passed”
on.
I am not defending that, convener. I am—
Deputy.
Sorry, deputy convener. I am telling you that that is the position as we found it. It might be an area that the committee wants to explore further with the Government.
I am just saying—well, I suppose it is for us to ask the Government to find that mechanism. Thank you.
Is spending on IT something that you will make a particular priority in future performance audits of the bodies that we are discussing in order to ensure that the lessons have been learned and improvements have been implemented?
That is certainly the case. Whenever we publish a performance audit report such as this one, we ask our auditors to use it in their annual audit work, when they look at the risks that apply locally and how they are being managed. That will happen in this case, and it will happen in discussion with the Scottish Government through the audit team.
However, I believe that the committee has a vital role to play in following up the recommendations that we make and seeking assurances from the people to whom they are directed that they are being acted on and that the risks will be better managed in future.
Thanks—we always like to hear how vital we are.
I have considerable experience of managing major IT projects. At times in my previous life, I had responsibility for technology divisions.
In this report, that makes you a guilty man, I think.
I was not working for the Government.
The fact that some of the errors that have been made are so fundamental makes it clear that, at almost every level, there seems to be a lack of understanding of how such projects should be managed. The point has already been made that the organisations that are managing the projects in question are relatively small, so it would be surprising if they had the necessary skills. A major point that must be made is that having those skills or buying them in is a necessity across all areas of government.
I am looking at exhibit 1 on page 4. I do not see any indication that the Disclosure Scotland project is still on budget at £31 million, and I would like to know whether that is the case. There is no mention of a write-off, but given the way in which things have been handled, the situation does not look good. Will the cost still come in at £31 million?
I will ask Gemma Diamond to provide you with more detail, but our current understanding is that Disclosure Scotland expects to deliver the system within the agreed budget of £31 million. However, there have been delays in getting there and a fair amount of workaround has been needed to ensure that the service could be delivered in the meantime.
When the system went live, it did not work as expected. Since then until quite recently, Disclosure Scotland made no payments to the supplier because of the problems that had occurred. Essentially, it withheld the payment that had been due at go-live and did not make another payment to the supplier until the majority of the defects were fixed. It had to put in place quite substantial manual workarounds to ensure that it could still deliver the required disclosures, and it is receiving recompense from BT for that. As the majority of defects are being fixed, the payments are starting to be renegotiated.
The other thing that comes through clearly from looking at the three projects is that there is no common or standardised approach: there is no formula. In every project that I have been involved in, a blueprint has been put in place that people have worked to. There is nothing in the report to indicate that there has been any transfer of knowledge, any sharing of best practice or any adopting of a standardised approach. It seems to have been left up to the individual organisations to decide what to do. I am concerned if that is the approach that is adopted across the public sector.
In some ways, it seems almost unfair to pick bits out of the report, because the picture is so comprehensively bad. However, there is some simple stuff that I would like to highlight.
Paragraph 46 says:
“the PVGs programme team had no experience of managing suppliers in a significant ICT development.”
That is a problem right across the public sector. Previous reports that we have received have highlighted a lack of procurement skills and a lack of experience in dealing with external suppliers. Do you see any general improvement in the public sector on that? I could keep picking bits out of the report, but because the picture that is painted is so comprehensively bad, it is perhaps unfair to do so. Is there any indication of a general improvement?
It is very hard to give you an overall assurance about that. My sense is that the larger bodies are definitely more aware of the risks and are better equipped to manage them properly. As you said, the common factor with the three bodies in question is that they are all relatively small for the scale of investment that they are making, both in relation to financial scale and the scale of the importance of the investment to their core business. Such bodies will always struggle to maintain the people, the skills and the experience to do such projects well. That is why we focus on the Scottish Government’s responsibility for strategic oversight and for having a core of expertise that can be used and drawn on.
I mentioned a couple of the highly significant investments that are under way elsewhere in the public sector. It is clear that they are being managed in a way that recognises the complexity and the risks involved. It is more difficult for smaller bodies to do that properly.
As I said, the picture that is painted is so comprehensively bad that picking bits out of the report is almost unfair, so I will stop there.
There is some truth in that, which may inform how we decide to take forward the report.
10:30
The report rightly highlights the cost to the public purse of mismanagement of programmes. Has any work been done on the human cost—particularly on how members of the public who access the services have been impacted on by the efficiency and performance of the organisations?
We have not considered that directly. We are conscious that each of the bodies has put in a lot of effort to try to minimise the impact on members of the public. For example, Disclosure Scotland staff have worked hard to design and operate manual workarounds to ensure that certificates can still be issued in a timely way, so that people do not lose out on employment opportunities and so that the bodies that they are looking to work with do not suffer. That is at the core of why the investment is being made in the first place. For example, it is intended that the Registers of Scotland will play a major role in the proposed land and buildings transaction tax, and thinking through what are the lessons from those programmes, for future investments that will be critical for public services, is one of the reasons why the report is here for the committee to consider, so that members can decide how best to take action to secure improvements for the future.
Ronnie—do you want to add to that?
Ronnie Nicol (Audit Scotland)
It is important, particularly when major policy developments, new services and new service designs are being planned, that people understand the complexity of putting together the information systems that are needed to support them to make them efficient. There is a sense in a number of the examples that you have been given that more attention needs to be given to having those things in place, and to understanding how long they will take to establish and how much it will cost for them to be established effectively so that, from day 1, the infrastructure is in place to support them when they are delivered.
You mentioned Disclosure Scotland, so I will focus on that. I know of student teachers who have experienced weeks of delays in taking up placements in schools because of delays in issuing their disclosure certificates. Last week, almost every person in a group of volunteers who attended a cross-party group that I was at mentioned problems with Disclosure Scotland and difficulties with getting certificates issued to their volunteers, which meant that their organisations were understaffed and unable to contribute to the community. We might be able to speak to Disclosure Scotland about that.
One of the aims of the system that Disclosure Scotland has been developing over the past two or three years has been to ensure that people can register with it so that their application is handled much more swiftly and they will not need to make repeated applications for new certificates in the future. Obviously, that service, which benefits the individuals and the organisations, is one of the things that has been delayed. That is a good example of why the investment that we are discussing will make a big improvement in public services and will result in efficiency savings. Those improvements will be put at risk if the process is not handled well.
I suppose that no Government is immune to such issues arising from time to time. There are some spectacular examples of information technology failures, including the national health service patient records management system in England, which I believe has just been cancelled by the Government of Mary Scanlon’s party after expenditure of £2.7 billion. Similarly, but even longer ago, the Child Support Agency’s software went live with about 50 known problems, but that was at a time when staff numbers were relatively okay and stable. The issue is not just loss of particular staff; it is about the expertise in the organisations that allows people to interact with the companies that sell the software. Colin Beattie and I have experience of that.
One of the difficulties is that people who buy software solutions often believe what they are told and do not try before they buy. There is a big lesson to be learned from that for any procurement, but especially for software procurement. The vendors say that the product will do everything that the buyer wants and more, but it rarely does, so that lack of thinking at the outset about what the requirements actually are often costs us dearly in the long run. There have been many examples of that over the years, and not just in software procurement but in other procurement, too.
I would like the front end of every procurement process to be strengthened to ensure that the customers—the people who want a product to do something—are absolutely clear about what they want and know that they are going to get it from their investment. Does the Auditor General believe that that is a strong part of what the Scottish Government is planning to do in the digital strategy that it is working on?
Not only public sector bodies get their IT wrong from time to time. In recent months, we have seen problems with the Royal Bank of Scotland and other very big corporate enterprises, which have had a big impact on their customers. That highlights the importance of the investment and why it is critical that it be managed well.
One of the interesting things in the work that the team has done is that they have highlighted that most of the systems and procedures are in place; the question is whether they are being used and whether the expertise exists to understand what those processes are telling us. Some of the gateway reviews that were carried out highlighted problems, but the expertise to understand the implications of lack of testing before implementation was not there. Alternatively—as Gemma Diamond touched on earlier—gateway reviews were delayed but nobody informed the accountable officer of that, so the processes that should have either provided assurance or highlighted the problems were unable to operate properly. Those things do not rely on having great amounts of technical expertise; they rely on having the discipline to use and stay on top of basic project management disciplines that should be in place for any major investment. That matters.
The other dimension is the importance of having the expertise to act as an intelligent client and being able to understand the implications of what comes out of those processes. Neither bit of that equation worked as well as it should have worked in the three cases in the report, although they are in operation across the public sector, and our auditors—as part of their work—routinely examine how they are applied in practice. The checklist at the back of the report is intended to let people who are responsible across the public sector undertake their own health checks on how well things are operating in their organisations.
The report refers to various methodologies that the organisations used along the way. Projects in controlled environments—PRINCE—is mentioned, which is a fabulous methodology, but is perhaps not the methodology that should be applied in procurement of software when it is particularly important to get the definition of requirements correct. I do not think that PRINCE would serve that purpose.
I have mentioned before the idea of adopting formal and recognised methodologies to help. There are lots of them around, but they require appropriate expertise to apply them properly. It is challenging for small organisations to have that expertise to hand, but the methodology should tell them that they need to get that expertise from somewhere to assist them on their journey and to get the product right.
I agree absolutely. That is why we have recommended that the Scottish Government examine from where smaller organisations get that advice and support, when it is required.
May I follow that up, convener? I will be brief.
Yes.
I did not see this in the report, but were the three organisations dealing with legacy software that the new systems had to integrate with, or were they building completely new software? I am not clear about that. The former can be almost impossible to deliver. The committee has seen examples of new software having to integrate with something that is fairly out of date.
The picture is mixed. Gemma Diamond will give details.
There was a mixed picture across the three organisations. As far as I am aware, Disclosure Scotland had a legacy system in place and was adapting it and building new requirements for it. Certainly, there was use of the legacy system to support it when it had to put in manual workarounds.
The Crown Office project was to build a new system, although it had in place a system that partially did what it needed. Many of the Crown Office’s difficulties were to do with the fact that the software was out of date and was no longer supported, so it had to find ways of managing that.
In Registers of Scotland, because the IT programme was a large one that took place over an extended time, developments took place one after the other. There were legacy systems and all the systems had to fit together. Therefore, the order in which things were done was important because one system had to be built to interact with another, which caused complexities. There was a mixed bag within the projects.
In one instance that the report considered, there was a partnership contract over a long period that covered on-going ICT and the development of new requirements. In the end, Registers of Scotland concluded that that was not an appropriate way for it to procure ICT. Would you support that conclusion and, if so, would you support it only in that specific instance, or is that model of contract fundamentally flawed?
That is difficult to answer, because it is the view of Registers of Scotland’s management that that was not the right type of contract for it to enter into. A lot of problems were experienced right from the start of the contract and all the way through, so it is hard to be clear whether it was the wrong type of contract or whether, if the problems had not happened, it would have been the right type. We cannot draw any conclusions about the type of contract. Registers of Scotland’s management can give the committee its view on the problems with that type of contract.
The report is interesting. The Scottish Parliament is fortunate to have a body such as Audit Scotland that looks into such issues so deeply. I have a number of questions; Willie Coffey touched on one of them, but I might come back to it later.
The Auditor General mentioned that the report deals with relatively small organisations, although they are important ones that have many powers. You also said that the bodies had problems because they are so small, and that larger organisations do not necessarily have similar problems. Can you give a couple of examples of the larger organisations to which you were referring? Why do they not have the same problems as the three organisations in the report appear to have?
I do not want to give an assurance that larger organisations do not have problems, because they clearly do. As I said, we recently saw that with the Royal Bank Of Scotland. As Sandra White said, a number of small bodies in Scotland provide important services to the public and are central to how Government as a whole runs. The conclusion that we drew from our work on the report was that, by definition, it is harder for those bodies to have the technical expertise that they need in order for them to consider what is possible—and to be the “intelligent client” to which Mr Beattie and Mr Coffey referred—that can test and challenge what suppliers tell them and maintain that involvement in managing what can be fairly complex projects.
For a body the size of, for example, the City of Edinburgh Council, it is easier to build a team with the necessary expertise and experience to understand the requirements. I am absolutely not saying that things will not go wrong in such bodies, but they have a head start in understanding what is required and in being able to bring together people who have the appropriate skills and expertise. There is a challenge for the Scottish Government in getting the benefits of arm’s-length bodies that are fleet of foot and can understand their businesses well, while ensuring that such bodies can tap into the expertise, experience and support of a body the size of the Scottish Government. I guess that the question that we are asking is whether that balance is right yet.
The key phrase for me in that answer was “arm’s-length bodies”. We might consider having an umbrella body from which small organisations can get expertise, but obviously those organisations will want to continue to be arm’s-length ones. How do we square that circle and continue to have arm’s-length bodies over which the Scottish Government has more control?
In paragraph 77, on page 19, you mention the major projects authority, which has been established by the Cabinet Office in England. However, that deals with major projects, whereas we are talking about small ones. What are your views on that?
10:45
It is a fine balance to get right and the committee might want to explore that with the Scottish Government. It is very clear that it cannot manage the IT investment of all the bodies for which it is responsible. Equally, it is clear to me that the accountability lines run back into the Scottish Government and that there is a responsibility, as well as an interest, in making sure that the money is spent well and that the benefits are achieved.
The major projects authority exists in England for major projects. We in Scotland now have an infrastructure investment board that sits within the Scottish Government and looks at significant investment projects such as the new Forth crossing. One of the questions that we are asking is whether that board might play more of a role in oversight of significant investments by getting assurance about the way in which procedures are being applied for maintaining quality and getting intended benefits, and being able to access the resources that are needed to step in if a body is not able to do that or if problems emerge.
That is like saying that the bodies would not be arm’s-length bodies, if they are being controlled. Perhaps the committee should consider that.
You mentioned gateway reviews, particularly of Registers of Scotland and the Crown Office. You said that procedures were put in place to deal with difficulties, but were not followed up. You also said that those two bodies were not able to challenge their contractors. Could you expand on those two issues?
I will ask Gemma Diamond to do that.
Certainly.
All three bodies had gateway reviews of the programmes. The Crown Office had three gateway reviews, and each subsequent review picked up the fact that earlier recommendations had not been implemented, so the same issues were arising at each review.
As was mentioned previously, in Registers of Scotland there was a large gap of six years between gateway reviews, which did not allow the problems to be picked up at an early stage.
On challenging suppliers and taking on board findings from independent assurance, it all links back to the intelligent client function that we were talking about earlier and having the skills to understand what independent assurance is saying. It might raise issues around testing the type of contract, the areas that the body understands, and the levels of risk that are associated with that, and a body can have a conversation about whether it accepts that level of risk, or can decide what it can do to mitigate those risks. Independent assurance is essentially about helping a body to understand what it is being told in the gateway review reports so that it can act appropriately.
There is a particular issue with the strategic partnership agreement into which Registers of Scotland entered. In 2004, it made a decision to outsource most of its IT provision to a private sector provider. Registers of Scotland would agree now that it did not recognise at that stage the need to continue to invest in its own client function centrally so that it could interact with the provider, and understand what problems might be due to needing better management of the required changes and developments.
There is probably a wider lesson there. If bodies are considering significant outsourcing, they still need to invest in the ability to manage the contract well over a long period of time.
This is where the difficulty lies when we are talking about arm’s-length bodies. You said that Registers of Scotland decided to go for private provision, but there were no checks and balances. Also, when Registers of Scotland had its management review, it thought that it was doing all right. Where do we cross the line between keeping bodies at arm’s length and giving the Government absolute oversight? That is the difficulty for me. The section 23 report is very good, but there is still difficulty with whether a body is arm’s length and independent or whether it should be brought into the Scottish Government. That is something that the committee will have to talk about.
Our starting point would be that when taxpayers’ money is involved, proper arrangements should be made to keep people accountable for how that money is spent and what we get for it.
I think that my colleagues have asked most of the questions that I was going to ask, so I will just ask for a bit of clarification. Perhaps you have answered this and I did not pick it up. Ms Diamond made a comment about a six-year gap. Who made the decision not to carry out the gateway review?
That was decided by the senior responsible officer of the ICT programme at the time.
I take it that that person would have known that gateway reviews are supposed to happen at a particular interval—whatever happens to be agreed. Does an officer in that position have the authority to not instigate such a review?
That process has been changed by the Scottish Government. Now, if a gateway review is postponed, the accountable officer is informed and if it is postponed a number of times, it will go up the chain within the Scottish Government to see whether anything else is required. Therefore such situations will not be allowed to recur.
So, there was nothing in the rules to say that the review had to happen. The officer had every right to postpone it and had that authority.
Essentially, the officer should take in a series of gateway reviews. The initial series looked at the procurement process. Once the contract is up and running, they can have a regular check on how things are going at programme level. There is no set timeframe for when that should be undertaken. It is generally accepted that it should be undertaken every few years—it depends on how the contract is running.
Did that person have the relevant skills to identify that a gateway review would have been helpful and that postponing it has perhaps made things worse?
That could be the case. It is hard to know, but certainly a gateway review might well have brought some of the problems to light slightly earlier, which would have allowed them to be addressed slightly earlier.
Paragraph 30 of your report points out that there were 400 change requests. Were those requests over six years?
Yes, that is correct. In six years there were 400 change requests.
It is important to note that the way that the contract was set up with BT meant that anything at all that Registers of Scotland wanted to change—any request—went through as a formal change request. Minor things such as asking for extra equipment had to go through as formal change requests, which is why the numbers look high. Obviously, the sheer mountain of 400 change requests indicates that there were some difficulties.
I own a business: 400 change requests in a six-year period for my IT system would not sound like too great a number. I assume that we could be talking about additions, not changes—additions to make the system more user friendly, for instance. Is that the case?
That is the case. The change requests would encompass additions and changes.
Would upgrades also be included in the 400 changes?
I am not sure whether upgrades would be included. We would have to check that with Registers of Scotland.
We are not necessarily seeking to make a judgment about the number. We are simply saying that the scale of the contract that was in place was not matched by the expertise and the client-side function within Registers of Scotland to ensure that those changes were necessary and were the best way, in value-for-money terms, of achieving what it needed.
I understand that the figure was in the Registers of Scotland internal audit report, so it was its internal audit that identified that figure as noteworthy.
Yes.
That is fine, but the figure is in the Audit Scotland report and to me it looks as if the 400 figure means wrongdoing. I am not an IT expert, but in my experience, I know how much I need to pay and how many changes I make. For a contract that size, 400 changes does not seem to be a lot, to be frank—especially if Registers of Scotland is buying in upgrades and it is not a question of changes to the contract but upgrades to it. It costs money to do that. I think that it would be worth the committee’s while to drill down into the issue and examine what a change would imply.
We can certainly consider asking Registers of Scotland that question.
That would be the right direction to go in.
Perhaps this is not the right place to pose this question, but I am not clear in my mind how the Scottish Government could provide the expertise. In various sectors, what typically happens is that you buy in the services of an expert in that sector, so I am concerned about the suggestion that the Government set up a service that all the different quangos could go to for solutions to their individual problems. If you could tell me what is in the back of your mind and explain how that might work, it would help me a lot. How could we set up a single authority to deal with a multiplicity of demands about specific systems? After all, as my colleague Sandra White has suggested, the bodies are separate entities. How could we impose our powers—something, I have to say, that I would love to do—on those quangos?
We do not have a specific model in mind. At the moment, a Scottish Government unit plays a role in ICT investment by some—but not all—public bodies and, as the report notes, a couple of those bodies asked for assistance that, for reasons that are not entirely clear, was not available. At least one of those bodies purchased the assistance privately, which is—of course—another answer. The Government might want to weigh up the costs and benefits of different ways of providing such help.
Our starting point is that, given that there are so many bodies whose functions, size and expertise range so widely and given that they spend taxpayers’ money, the Government has a responsibility for ensuring that it oversees management of that investment; that, in its view, the right skills and expertise are locally present to manage it well; and that support is available.
We have deliberately stopped short of suggesting there be a new quango or that the Scottish Government expand its own team to do the work—I am not sure that that is the answer—but the evidence in the report suggests that there is a gap and that people cannot necessarily get the expertise they need when they need it for investments that would have real benefits for public service users and which could generate real efficiency savings.
Ms Scanlon wants to ask a short supplementary question.
It will be very short, convener. My question follows on well from Gil Paterson’s.
We are aware of the problems and I am sure that my colleagues and I want to find a solution. However, I wonder whether that solution can be found on paragraph 76 and 77 of the report. I am certainly not looking for another quango or more central control, but the fact is that we already have a strategic corporate services board and information systems investment board and in paragraph 77 you say that the information systems investment board
“has the potential to provide increased scrutiny”.
My Government in England—which Willie Coffey kindly referred to—faced problems and came up with the major projects authority, which you highlight in case study 1. If I have read the report correctly, you seem to be saying that the strategic corporate services board and information systems investment board could be brought together along the lines of the English major projects authority to provide more coherent management of projects and, where appropriate, to intervene directly in failing projects. That is very different to taking control centrally.
Is the framework already in place for a body that could be there for people who are seeking advice or—as Colin Keir and Willie Coffey have suggested—to go through the project management checklist and intervene when serious problems emerge in the project?
11:00
That is definitely an option that is worth considering. We have not suggested any structural arrangements for addressing the issue; instead, we have suggested what we think needs to happen, including better strategic oversight and the ability to monitor and provide support where needed. However, Mary Scanlon is right: what you have highlighted provides a starting point to work from, if the Government decided that that was the best way of addressing the matter.
So, there is already a foundation that can be built on.
Absolutely.
Willie Coffey has a very, very short question.
I am sure that if you scratched the surface of the MPA and found out what it was doing, you would find that it was not a heck of a lot different to what the Scottish Government is planning to do. The issues are common and are all about getting right procurement of certain services. Loads of methodologies are available for deployment by Governments and companies to get it right and I think that, if you really looked at the matter, you would find pretty common solutions emerging.
That was more of a comment than a question.
If you do not mind, convener, I actually wanted to ask the Auditor General about her recommendation on having a central pool of expertise. What does that really mean?
Some comments were made about outsourcing, but I must point out that outsourcing does not mean outsourcing responsibility and management, both of which you need to keep close to your chest. That should not really be an issue. If you need to buy expertise, you should do so, but you must retain control and responsibility. When you talk about having a centralised pool of expertise, do you mean outsourcing to third parties or having intelligent-client type services?
I do not mean that directly. You are absolutely right that, no matter how much of the IT service is outsourced, it is necessary to retain the capacity to be an intelligent client or the organisation will either not get what is wanted or will have to pay much more than is necessary.
With due respect to certain services, if a body does not have the capacity to be its own intelligent client, should it buy that service from a pool of resources?
The three case studies show that some bodies do not have access to the expertise that they need. In a couple of cases, they asked for help, but it was not available. How it is provided is, again, a policy choice; it is not a matter for us; however, there is already a pool of expertise in the Scottish Government and there might well be scope to expand it—perhaps in a strategic way that looks ahead at planned IT investment over a period of years and matches it to peaks and troughs, the type of work and so on.
Another alternative might be to buy in expertise as required but, again, you need to start with an understanding of what you are investing in and your needs with regard to particular skills and volumes of expertise if you are going to manage it well.
That ends this particular item. I thank the Auditor General and her colleagues for attending.
As members have been extremely good about my new coffee rule, I suggest that we have a five-minute comfort and coffee break and reconvene just before 10 past 11.
11:02
Meeting suspended.
11:08
On resuming—