Public Audit and Post-legislative Scrutiny Committee

Meeting date: Thursday, April 20, 2017

Official Report 603KB pdf

---

Section 23 Report

---

“i6: a review”

We move to agenda item 5, under which the committee will take evidence on the Auditor General for Scotland’s report, which is entitled “i6: a review”. I welcome back Mr Foley and Mr Flanagan. Thank you very much for your written responses to our numerous questions. As there is no opening statement, we will move straight to questions.

I am looking at page 5 of the SPA letter of 13 April 2017 that was sent to the convener, and in particular at the second question and answer. During our previous session, a question was asked about whether any cost had actually been incurred by the cancellation of the system. An assurance was given that it had not, although there was the question of the notional loss of potential savings that would have arisen from that system.

However, the letter says:

“The decision to cancel i6 has resulted in a need to reinvest in maintaining ... 125 legacy systems that would otherwise have been decommissioned.”

How much money is being put into maintaining 125 legacy systems and were we given incorrect information last time?

10:30  

No, you were not given incorrect information.

Incomplete information?

No. I think that I was trying to convey that some of those systems would have carried on for a period of time anyway, while they were being decommissioned. Not all those systems incur a cost and some of the cost is in house; we have programmers and developers and such like—

But an in-house cost is a real cost.

It is a real cost but we have a no compulsory redundancy policy as well, so it is very difficult—

I am sorry—how does that feed into this?

Some of the people who would be involved in that work would be involved in this—

Some of the people involved in this work would have been—

In-house people—

Would they be made redundant?

Potentially, if i6 had been successful, on the basis that it would have been much more efficient than some of our current systems, we might have expected some of those posts to be made redundant, but—

But that comes under what we spoke about a moment ago, which is the notional loss of savings, which we understood. However, the letter says specifically that there is

“a need to reinvest in maintaining ... 125 legacy systems”.

How much are we spending on that?

I would have to come back to you on that. That work is carried out within Police Scotland, so the information is held there—

But it is important information that is referred to in your letter.

I can certainly get the information for you.

I would appreciate it if you could.

I was also told that there had been no integration of systems across the eight legacy police forces. When i6 was cancelled, I asked a specific question about that and I was told that there was no integration and that no system had been successfully integrated. Now, in the second part of that answer in your letter, you say that

“30 new national applications have been implemented.”

That was prior to i6—

Prior to i6?

Prior to i6 concluding.

But my question did not relate specifically to i6. It was in the wider context of Police Scotland and it was about whether any single system had been successfully integrated. Now you are saying that there are 30 new national applications. Does that mean that there has been successful integration at any point?

Those would have been new systems that were put in, which are in use across the national force—across the legacy areas. However, they were not integration of existing systems, if you see what I mean.

But you say in the letter that

“a significant number of legacy systems have been closed down”.

Yes, because they were redundant—they were no longer required.

They were redundant and no longer required—were they replaced by national applications?

They might have been replaced in some cases, but not in others. When the forces came together, we uncovered some systems that had not been used, had been used very rarely or were not required, so those systems were closed down as unnecessary. However, there was no impact on policing as a result of that.

The impression that I got at our previous meeting was that there had been no successful integration of any systems within Police Scotland in those three years. Is that correct?

No, and if that was the impression, that was not the intention.

It was what was said.

There are the systems that were put in over that period of time and there is also the custody system, which was rolled out nationally after the failure of i6. That system was previously held in Dumfries.

It would be helpful to understand a little bit more about which systems have been integrated across the legacy police forces because it is of great concern if, in three years, we have not succeeded in doing any of that.

Work has been done, Mr Beattie, and I can certainly furnish you with information on that.

Can we get that information? The information that we received previously does not appear to be correct.

In the letter, at the bottom of page 3, there is an answer about efficiency savings. I take the point about that. You talk about

“a robust ICT strategy which is nearing completion”.

When will that strategy be available?

It should coincide with the launch of the policing 2026 strategy. As we mentioned earlier, that is out for consultation and the information and communications technology strategy must support that. They should come together at the same time.

That is critical if this committee looks at the potential savings for the future.

Yes, it is vital. We need to have the strategy in order to deliver the savings and efficiencies in the 2026 strategy. That is well recognised.

When will the ICT strategy be available?

It should be available around about June or July to coincide with the policing 2026 strategy. As we said earlier, the consultation period for local authorities will finish at the end of May, which is when we will start to gather up all the information and comments from the public and stakeholders. That will inform the final version of the policing 2026 strategy and the ICT strategy will absolutely underpin it.

To return to i6, I accept what the Auditor General stated about Police Scotland and the SPA doing all the right things and ticking all the right boxes. However, it still went wrong.

Yes, it still failed.

What lessons have been learned from that? There is now a digital Government body—I cannot remember its name—that co-ordinates all that. How do you feed back into that?

There are aspects about the lessons learned in the Auditor General’s report. Looking back, there were lessons learned internally. For example, we would never go into another project using the waterfall approach and we would have more technical expertise on board if we were going into significant IT projects in the future. In my view, the i6 project did not have enough ICT experience. In the future, we will engage more with the Scottish Government’s chief technology officer on such projects. Those are all important lessons that we have learned. We intend to participate more with a number of public sector bodies to gain the benefit of their knowledge and lessons learned, and to share our experiences.

I am looking at the first question and answer on page 6 of your letter, on the three external expert advisers and how much they were paid. I realise that you got the money back, but, including VAT, they were paid more than £4.6 million.

Yes.

I do not know about anyone else, but I find that eye watering. How much was the project?

Forty-five million pounds.

Forty-five million pounds, and more than 10 per cent of that was for three external advisers. I find that astonishing.

If, for example, you take the amount that was paid to Exception—that was the highest figure—that was because that level of expertise did not exist in Police Scotland and it had to be bought in. That is why the amount is so high. In most organisations, you might expect to have a level of expertise that could deal with technical aspects on behalf of the client, but it did not exist in Police Scotland.

Were there no resources in the Scottish Government that you could tap into to get that expertise?

We tapped into some resource from the Scottish Government; we had the assistant chief technology officer on the programme board. However, the project was so specialist and detailed that it was felt that we really needed to have external advice prior to and in the course of the contract.

Basically, the internal resources did not have the expertise to handle a project of that scale.

That is my understanding of what happened at the outset. I was not there, so I am answering based on knowledge that I have picked up since being in the job. My understanding is that that was why the decision was taken at the beginning. I also understand that a significant amount of the Deloitte cost was in relation to managing the procurement process, because—again—it was so specialist.

Managing the procurement process?

Yes. It was involved throughout the duration of the project, but it was heavily involved in the procurement of i6, too.

Who wrote up the specifications for the project? Was that Deloitte?

My understanding is that the specs for the project were written up by Exception with the involvement of Deloitte. Exception was the technical lead.

I return to the £4.6 million figure. That still seems like an awful lot of money for the size of the project and for what was achieved. We got the money back, but I am looking to the future and to other projects that might attract that level of external costs.

Looking to the future, we would not do it the same way. A clear lesson has been learned from the experience. We would do a project like i6 on a modular basis, and slice it into small pieces. We would seek wherever possible to manage the technical aspects in house, under the ICT director in Police Scotland, and we would draw in specialist expertise as and when it was required. I genuinely do not believe that it would be undertaken at the same level.

It feels like groundhog day. I have served on the Parliament’s public audit committees for a number of years and looked at many information technology projects, and our discussions today sound very much like the conversations that we had five, six or seven years ago. There is a lack of expertise in the organisation to define the requirements for a piece of software, leading to inevitable delays, problems and issues further on in the development programme.

I want to pick up on the point about the requirements that we wished the software to deliver for us. You said that Deloitte and Exception were involved in that, but—surely to goodness—you were the client, or the customer, who was commissioning the system and you knew what you wanted. Why was your organisation not at the forefront in defining the requirements, rather than relying on an external body to do that for you?

Deloitte and Exception brought the particular expertise. My understanding is that, prior to the contract being awarded, Police Scotland was heavily involved all through the process, with Deloitte and Exception, in defining the requirements, but it did not deal with the technical aspects, which were something else. With regard to the actual operation and use of the software, my understanding is that Police Scotland was definitely involved in defining the requirements.

The project was under the control of a former deputy chief constable, who was the senior reporting officer to whom all the consultants and the i6 programme team reported. The policing aspect was very much at the forefront. That is why people were able to say throughout the duration of the project that, if it was successful, it would free up 23 per cent of additional police time—only the police would have an idea of how that would come about. Exception would not be able to do that, because it dealt with the technical aspects, and Deloitte is a firm of consultants, so it would not have had the knowledge to do that.

If you are going to design a piece of software that is based on the requirements that you want it to deliver, you need a detailed requirements specification. I cannot see from the Auditor General’s report or from your answers to the questions in the letters that a detailed requirements specification was in place. It looks to me as though the software was based on the Spanish system, which was thought to be what was required in Scotland. The report says that quite clearly. I do not see where effort and time went into defining in detail the requirements for what the software had to carry out. Is that inaccurate?

I was not there personally, but my understanding is that that is what happened when the requirements were defined. From my attendance at meetings on i6 over the past three years or so, I have the impression that the senior officers in Police Scotland knew exactly what they wanted. They had gone out to Seville to see the system there, but they did not go out with the view that that was what they would bring back. My understanding is that they went out to look at the system to see whether Accenture had delivered successfully for another police organisation in a different country, and a view was then taken on whether some of that could be replicated for Police Scotland. It was not intended at any point in time that the system in Seville would be used by Police Scotland.

I can only go by what I have in front of me. The timeline at the back of the Auditor General’s report shows quite clearly that the full business case was approved and the contract awarded to Accenture in June 2013, and that the high-level design started the following month. I have never seen anything like that in my life, and I am a software engineer with 25 years’ experience. There is a bit missing—the detailed requirements specification, which can take months and months, or even a year, to get right.

I suggest to committee members and to you that that is why the project went wrong. You thought that you were getting a bus but the contractor delivered a truck because you did not define what the requirements were. You got what the contractor thought that you wanted. If you set out the detailed requirements to be followed, you usually get what you want. From what I can see, that is what was missing in the project.

10:45  

Again, I do not want to sound as if I am not taking any responsibility for that, but I was not in post at that stage. I have to accept that your technical knowledge is greater than mine. If that is how you see that type of project being designed, I have to accept that that is a valid assumption.

Okay. There was a discussion about the waterfall approach versus the agile approach, which are development methodologies, and it was suggested that the waterfall approach would never be used again and that the agile approach would now be used. However, that hides the fact that the key component at the beginning of any project, software or otherwise, is the defining of requirements. If we get that right, we usually get what we want.

What happened with the i6 project is another example of something that I have seen in the past. Public bodies seem not to have learned the lessons from a number of previous Audit Scotland reports relating to IT projects. That kind of thing does not happen only in Scotland, but it seems to happen at the heart of public bodies such as yours, Mr Foley. You said a moment ago that you do not have enough technical expertise or IT expertise within the organisation. That seriously has to be addressed so that we do not keep having to give the same message year after year when looking at IT projects that just do not work. They can work if they are done correctly and proper quality management and development methodologies are applied and embraced at the outset in the organisation. In such cases, the things that happened with the i6 project will not happen.

I agree. However, having had discussions with the information and communications technology director of Police Scotland, I can assure the committee that that is how he approaches matters. That might give some assurance that any issues will be addressed as we move forward.

What methodologies do your development projects use?

The director has a quality management system, but I do not know what it is called.

Okay.

I want to stay on that theme, if I may. The letter that we received has a section entitled “Contract failure”, in which the SPA notes at least eight learning points from the failure of i6. However, the SPA also states that there was no failure on the part of Police Scotland or SPA staff to carry out their responsibilities effectively. Given the project’s failure, is it credible to suggest that there were no failings on the part of SPA or Police Scotland staff?

The project was managed within Police Scotland. As I said earlier, heavy reliance was placed on the contractors, particularly from the technical point of view. Exception was the technical lead on the project, and clearly we would expect Accenture, as the contractor or deliverer of the software, to be extremely technically capable as well.

In your view, were Police Scotland and the SPA not responsible at all for what happened with the i6 programme?

There has to be some responsibility, but whether that is a failing of the staff who were involved is a different matter. Clearly, there is responsibility, because the project failed.

Whose responsibility is that in Police Scotland, then? Do you take the responsibility?

The senior officer who was responsible for it in Police Scotland was a deputy chief constable who is now retired. He was the responsible officer for the whole project.

Okay, but that is different from what you have said in your responses to the committee’s questions, which is that Police Scotland and SPA staff are completely in the clear. Is that your position?

In terms of the SPA staff, they had little involvement in the delivery of the project.

Is that your position, Mr Foley?

My position is that the majority of the failings that occurred in the project were centred around its technical aspects.

If the majority were centred around the technical aspects, the minority were somewhere else. Was that within the police?

I am not aware of anybody within Police Scotland who failed in their duty in relation to the project. From a contract point of view, the project was very tightly managed, which is why we managed to get the money back.

I will come back to the contract, because that is interesting, but first I want to follow on from the questions that Willie Coffey and Colin Beattie asked. Am I correct to say that you have been the CEO since autumn 2013?

Yes.

You are a chartered accountant to trade.

Yes.

You are a former president of the Chartered Institute of Management Accountants.

Yes.

You are a fellow of the Institute of Directors.

Yes.

You have been the director of over 40 companies, at one time or another.

In the past—yes.

Yet at no stage from 2013 did you pick up that something was going massively wrong. Is that correct?

We picked up that things were going wrong at the user acceptance testing stage.

I am not asking about “we”; I am asking about you.

That is when I picked it up as well.

When was that?

That was in autumn 2015.

So, between 2013 and 2015—through a contract variation, mind—you did not pick up anything at any point. Despite all your experience, at no point did you say, “Hang on—I think we’ve got a problem.”

There was no indication that we had a problem until the user acceptance testing stage. That is where you normally pick up these things.

There was enough of an issue to do a contract variation.

The contract variation was put in place as a result of difficulties between Accenture and Police Scotland that surfaced in the summer of 2013. The main purpose of the contract variation was to provide more flexibility for both parties to deliver the product. As the Auditor General refers to in the report, there had been tensions between the parties, and there was an attempt to alleviate those tensions and improve working relationships between the parties. That was a positive and proactive measure that was instigated by the SPA at that point. That was delivered, and then we carried on, with support from the consultants.

As I said, we did not have the technical experience, which is why we used Exception—it was acting on our behalf and working with Accenture.

Let us look at the consultants. When the committee last discussed the issue, on 16 March, I said:

“When I was in professional legal practice, I was paid to make sure that there was no ambiguity in my clients’ contracts. Does the ambiguity in this contract suggest that the failing—or a failing—lay with the professional advisers who had been engaged?”—[Official Report, Public Audit and Post-legislative Scrutiny Committee, 16 March 2017; c 8-9.]

You say in your letter that you were “well advised” by the adviser—which, as Colin Beattie rightly pointed out, was at a cost of over £4 million. How is that position sustainable when it appears that there are fundamental flaws in the contractual nexus, as Willie Coffey has highlighted?

The issues with i6 came to light during the user acceptance testing, which was part of the programme. They were uncovered by Exception. Exception noticed that something was awry and it challenged Accenture on that and advised Police Scotland. That is when I became aware of the issue as well. Work then started, first by Accenture, to establish exactly what the problem was, because it was not fully aware of the problem at that point. Accenture brought in additional expertise to its part of the programme to consider what the problem looked like. In early 2016, it was obvious that delivery was not possible in the way that had originally been set out, so an options appraisal had to be carried out to consider what we could do. I took charge of that project and led it through to 1 July 2016.

It was the advice from Exception that highlighted that there was an issue. That was its job.

Good on Exception, but what about back at the start? As Willie Coffey pointed out, you cannot go into something of such a scale with any ambiguity or uncertainty in what you are signing up to. Very quickly, the programme started to unravel. To my mind, that is a flaw in the contractual nexus. People were going into things not knowing what they were doing. The authority seems to have paid nearly £5 million to external advisers to avoid that situation. In your letter, you have lauded those external advisers and said that they are wonderful, but that is clearly not true, is it?

They did not fail.

Who did fail? Was it Accenture?

Accenture was delivering the system.

Is Accenture the only party that is responsible for the failure of the i6 programme?

The relationships could have been better on the way, and if they had been, it might have helped in identifying issues earlier. However, that is just potentially the case; I do not know whether it would definitely have been the case.

Mr Foley, I asked you a closed question. In your view, is Accenture the only party that is responsible for the failure of the i6 programme?

No.

Who else is responsible?

Police Scotland and the Scottish Police Authority also have to take some responsibility.

That is interesting, because that is not what you say in your letter. Who in Police Scotland and the SPA should take responsibility?

We should take collective responsibility for the fact that a contract that we signed up to back in 2013 did not work out.

Did someone misadvise you? As an authority, you would surely not have signed that—collectively—if you had been properly advised.

The contract was signed in June 2013, but a lot of work was carried out before that. My understanding is that a lot of the detailed work was carried out—principally by Police Scotland, with the knowledge of the authority—at that time. However, I was not there, so I cannot tell you whether that was of sufficient quality or not. I am sorry, but I was not there. I have no evidence to suggest that.

You have evidence, because I am putting it to you that it has all gone wrong. The problem is that you talk about lessons having been learned and you say that this will not happen again, but the reality for the committee is that we see it happening time and again. With respect, I have absolutely no confidence that it will not happen again.

What I hear from the responses to our questions is a suggestion that Accenture is to blame for everything. You now accept that Police Scotland had some role, but you will not accept that the external advisers had any role. I foresee this happening over and over again. Is that a fair summary?

No, I do not think that that is fair. I do not think that it would happen again in that fashion. We have learned enough to know that we would never attempt to have what we might call a big bang approach to such projects—it just would not happen. We have learned from that. We would have more in-house involvement than we had before, and if that meant going into smaller installations or implementations and employing expertise rather than contracting it, that is how we would approach it.

I am interested in that point because you are currently adopting a new in-house modular build. You are in that process at present, are you not?

Yes. Some of that is going on.

Have you taken the steps that you have just talked about?

In part, yes, but it depends on the project. Doing something that is highly specialised and involves a significant contractual arrangement is different from carrying out a piece of work largely in house but with input from experts for a week or so at a time. We would use a different arrangement going forward. We would never have the previous type of arrangement again—absolutely not.

I want to pick up on that and take it further. In August 2014, the Scottish Government conducted a gateway review of the project and designated it “amber/green”—it was called a delivery confidence assessment. In effect, it said that it was all systems go, yet in the same month, according to the Auditor General’s report, milestone 5, which was the functional design, was behind schedule and you withheld the payment of £2.6 million.

Given the exact overlap of time, I am curious to know whether the Scottish Government was told about that. Was the information withheld? Did the Government understand what was going on with the delays that were already being experienced, which the Auditor General identifies?

Yes. Audit Scotland was aware of that. There was a programme board, which was chaired by the DCC who was responsible for the project within Police Scotland, and the Scottish Government had representation on that board.

If I am right, the Scottish Government will go through a separate assessment process to assure itself that a project is running according to specification and time. The Scottish Government gave the project an “amber/green” designation in the delivery confidence assessment at the same time as delays were being experienced. How do you explain that?

11:00  

I can explain it only by saying that there was perhaps a timing difference. It is more than likely that, although the gateway review was published in August, the work had been carried out before that. Because it was a fairly fast-moving contract, if people did not hit a milestone, payment could be withheld. This is only a possible explanation, convener, but it could be that there was a time lag between the publication of the gateway review and this fast-moving project not hitting a milestone later in the same month.

I would be grateful if you could reflect on that and come back to the committee on the timing, because it seems extraordinary that we are pointing in two different directions in the same month.

I will come back to the committee on that.

Thank you. I have a final question, and then I will see whether other committee members have any more questions. A recent newspaper report, in the Daily Mail of 18 April, quotes an SPA director as saying that the i6 failure has

“left policing five years at least behind where it should be.”

The article goes on to quote the general secretary of the Scottish Police Federation as saying:

“Five years seems a very generous underestimate”.

What would you put it at?

Personally, I do not believe that we are a full five years behind. We are years behind, and when the ICT strategy is finalised and made public in a few months’ time, it will set out the timeframe to get to delivery of what we lost in i6. In conversations that I have had with the ICT director in Police Scotland, he has suggested to me that the time to do that is closer to four years, but that includes some other installations that he has planned. We are a number of years behind.

We can debate the number of years, but there has clearly been an opportunity cost.

Yes, there has indeed.

As there are no further questions from other members. I thank both witnesses for attending. Mr Flanagan did not get many questions in this session, but I am sure that he is grateful for that. The committee will write to you with our requests for additional information, and I hope that they can all be met.

We will now move into private session.

11:02 Meeting continued in private until 11:20.