Contents
- Attendance
- Decision on Taking Business in Private
- “i6: a review”
“i6: a review”
Agenda item 2 is an evidence session on Audit Scotland’s review of Police Scotland’s i6 programme. I welcome Caroline Gardner, the Auditor General for Scotland, and, from Audit Scotland, Catherine Young, audit manager, and Mark Roberts, senior manager.
I thank the Auditor General for accommodating the change of date for this meeting and for coming along today. Auditor General, I am aware that you have a commitment directly after this meeting, so I intend to finish the session by quarter to 2 to allow you to leave. I invite you to make a brief opening statement.
Thank you, convener—it will be brief. In June 2013, the newly established Scottish Police Authority signed a contract for £46.1 million with Accenture to deliver a national information technology system for Police Scotland, known as i6. That followed an 18-month procurement process, which we found followed good practice and included intensive engagement with potential bidders.
The i6 system was intended to replace more than 130 IT and paper-based systems inherited from Police Scotland’s predecessor forces and to transform how Police Scotland records, manages and analyses information. It was intended to be a key component of police reform. Three years later, in July 2016, the Scottish Police Authority and Accenture mutually agreed to terminate the contract. They agreed a settlement of £24.65 million, which included a full refund of all the money paid to Accenture—a total of £11.09 million—plus an additional payment of £13.56 million.
At the heart of the i6 programme’s failure was a disagreement about its scope, the interpretation of the contract and the extent to which Police Scotland’s requirements were met by Accenture’s solution. The disagreement surfaced almost immediately after the signing of the contract in June 2013, and permanently damaged trust, relationships and confidence between Police Scotland and Accenture.
Other factors also contributed to the failure, including the method adopted for the system’s development, overreliance on an existing system that had been delivered by Accenture for the Guardia Civil in Spain and a misplaced optimism about the prospects of delivering the system, which might have led to reluctance to consider terminating the programme earlier.
The failure of the i6 programme means that some of the benefits of police reform have been, at best, delayed. Police officers and staff continue to struggle with out-of-date, inefficient and poorly integrated systems. There are also wider implications for the modernisation of the justice system. It is critical that the Scottish Police Authority and Police Scotland put in place a plan that sets out how the benefits that i6 was supposed to deliver will be secured. That is particularly important given the emphasis on the use of technology in the recent “Policing 2026” draft strategy.
The team and I will be happy to answer the committee’s questions.
Thank you very much for that statement. I refer members to paper 1, which is a note by the clerk, and paper 2, which is a private paper for members.
I want a bit of background information. Your report acknowledges that good practice was followed during a fairly lengthy procurement and discussion process but, within three months of the contract being awarded, the project was at red status and only one milestone was being achieved. Could anything have been done differently throughout the procurement and the discussion process to avoid the difficulties that emerged? If Police Scotland and the SPA were to embark on a similar process, what recommendations would you make to them?
That is a big question, convener. I will kick off and then ask Mark Roberts to come in. As you say, we found that the process that was followed fitted with the good practice that we and others have recommended. The underlying problem was a mismatch between what Police Scotland specified in the contract and how Accenture interpreted that, particularly its understanding that the system that had been delivered for the Guardia Civil would supply most of what was required for the Police Scotland system. That was the gap. I ask Mark Roberts to talk you through that in more detail.
As we say in the report, the programme for procuring the system followed the good practice that we would expect to see and that we have recommended in previous reports. For example, a programme board was established, a single individual was identified as the senior responsible owner and a dedicated programme manager managed the day-to-day operation of the programme as a whole. The procurement process followed what we and others recommend as good practice.
To respond to the convener’s question about what could be done differently, it is unlikely that the organisations would again go down the route of pursuing such a large-scale IT programme, and the SPA’s chair has said as much. They are far more likely to break things up into more discreet bits and have a much more modular programme rather than one giant one. That might assist considerably in trying to avoid some of the problems that i6 ran into.
I struggle to understand how, throughout the process, nothing was picked out or identified to suggest that the problems were insurmountable.
Despite an intensive process—the report refers to the 160 dialogue sessions that the police had with potential bidders, which were focused on the technical specifications required of the system and how it was to be delivered in the detailed contract—the differences in interpretation of what was going to be delivered emerged very quickly and within a short period after the contract was signed. That is a challenging part of the history.
It almost seems as though the discussions that were taking place were at a high level rather than a practical or operational level. Is that a fair comment?
We cannot comment on what was involved in those detailed dialogue sessions. My understanding is that the discussions on how the system was going to operate and what was expected of it were at a fairly detailed level. Clearly, despite all that, for some reason a gap was left once they started to get going on developing the programme, and there was a difference in interpretation about what was expected and what would be delivered.
I spent several years lecturing on computer project management systems to postgraduates. I have gone back to my lecture notes and one in particular, which is from 2002, is relevant. It says:
“An off-the-shelf solution solves someone else’s problems, not yours.”
That sought to identify for my students that, in looking at ways forward, we should be wary when looking at off-the-shelf solutions, as they would be solving someone else’s problems. If somebody thought that an off-the-shelf solution was a good solution to the generality of the class of their problem, they could, of course, take that option and change the way that they did things to adapt to it, if it was an industry leader. What has happened here has borne out my point. Is there a general point that this situation illustrates that one must exercise particular caution when buying somebody else’s off-the-shelf solution and imagining that it can be adapted while not compromising what you want to do to adapt to the system that you buy?
That is a very well made point, Mr Stevenson. In the report, we say that there was overconfidence on the side of Police Scotland and of Accenture about the extent to which the system that had already been developed for the Spanish policing service would provide the basis for what Police Scotland needed. In the event, it became clear over time that much more bespoke development would be required.
We say a couple of things in the report that are relevant to that. First, once it got closer to it, one of the central concerns that Police Scotland had about the Spanish system was about the capabilities of the search function, its different elements and how far it was genuinely able to integrate the information. The second comment is about concerns that were raised by the programme board about the extent to which the Accenture team understood policing in Scotland. Those points suggest that there is something important in what you said, Mr Stevenson.
Mark Roberts might want to add to that.
It is fair to put on the record that i6 was not about applying the Spanish system directly to the Scottish policing environment. i6 was built on the Spanish system but it was recognised at an early stage that the product would be much more complex. i6 was always designed to be founded on the Spanish system rather than being a like-for-like replacement. As we say in the report, it gradually became clear that significantly more was going to have to be bespoke rather than an extension of what had been delivered in the Spanish context.
In his book “The Mythical Man-Month”, Professor Fred P Brooks of the university of southern somewhere—I am sorry, I cannot quite remember where—says that you should never seek to modify more than 10 per cent of a system. Would this project break Fred P Brooks’s law of 10 per cent?
The estimate that we have seen and heard about was that two-thirds of the system could have been based on the existing system. In that context, that rule would have been broken.
So it is well adrift. It is worth making the point that Fred P Brooks wrote his book in 1974.
I want to ask about the genesis of the project, which happened over a number of years. I want to know about the extent to which your report comments on or you feel able to comment on what someone who gave information to me referred to as the missing years. I understand that the project actually started in 2004 and incurred significant staffing and consultancy costs. Can you comment on that? For example, can you say whether those costs were part of the settlement?
The report that is in front of you looks at what happened from 2010 onwards, which is when people started talking specifically about i6 as one large programme. Mark Roberts has had more history with this, so I ask him to comment on the longer-term timescale.
I recognise that this is the latest episode in a long history of police information technology developments during the past couple of decades. We took it from the 2010 date that the Auditor General mentioned because that was when the outline business case was agreed by the then Association of Chief Police Officers in Scotland council. That was the starting point for us. We did not look at costs or any activity that had gone on prior to that date while doing this report, although there was obviously contextual information.
I cannot fully answer the question about whether those costs were recognised in the longer term. It might be better to pose that question to the SPA. Our understanding is that the settlement agreement that was reached with Accenture recognised staff costs during the course of the i6 programme, not in the longer-term period, which includes its genesis.
13:15
Does the business case for i6 allude to any of those previous projects? There was one significant and extremely costly failure. Does it allude to issues around the collaborative working between the eight forces and two other organisations that existed at the time?
In the full business case, there is reference to previous attempts to develop different systems. Off the top of my head, I do not recall whether there were any specifics about the costs that had been incurred prior to that date in those other programmes. As I said, we were focused on i6 as i6 from that 2010 date.
The business case, which you mentioned, talks about potential savings of £200 million. Are they likely to be realised, or are they written out of the equation?
The i6 system has not come into existence, so the benefits that were supposed to start accruing from it are not accruing. Until we see the plan that will follow on from the current situation, which will, we hope, implement the vision that is outlined in “Policing 2026”, we will not be able to tell what benefits will accrue.
What part was i6 going to play in overall police reform?
It is fair to say that it was intended to be a central part of police reform, both in terms of the £200 million savings, which were a contribution to closing the funding gaps that we are now aware exist, and, just as importantly, with regard to making police officers and police staff more effective, joined-up and fleet of foot because of the good use of IT.
Andrew Flanagan, the SPA chairman, indicated that there will be no “son of i6”. Will there be any non-gender-specific siblings, cousins or anything like that?
That comes back to the point that Mark Roberts made that the intention is not to have a big-bang approach but to look at ways of achieving lots of better and more efficient ways of working, in the context of the draft “Policing 2026” strategy, through the use of technology in smaller and more modular ways. We have not yet seen a plan for that, but I think that that is the approach that is intended.
The Auditor General has asked the auditors of the Scottish Police Authority to monitor the development of future plans for information and communications technology as part of the annual audit process. That is the form that our on-going monitoring of progress will take.
For the benefit of a non-technical person such as me, could you say whether replacing one big project with, for example, 10 smaller projects would spread the risk or increase the risk?
Generally, it is perhaps a good idea not to put all your eggs in one basket. Taking an approach involving smaller projects would follow the general practice in the IT world. Practitioners who we have interviewed on this subject have talked about a more modular and agile approach to software development being much more likely to be adopted these days, rather than one giant programme.
I wanted to touch on the balance between a waterfall method and an agile method, but I think that that has been quite comprehensively covered.
The Auditor General and Mr Roberts have spoken about the fact that the 18-month procurement process followed good practice—perhaps the term “best practice” was also used. Can any lessons be learned in order to enhance the process even further? For example, could anything be improved around the invitation-to-tender arrangements?
In broad terms, we think that that process followed good practice. Indeed, the fact that Police Scotland and the SPA not only managed to recover in full the payments that they made to Accenture but secured an additional payment in compensation for staff time and other costs that were incurred demonstrates the strengths of the approach that was taken.
We are in the process of pulling together the lessons from all the work that we have done on IT system failures and problems. You will be aware that the i6 issue is one of a series of IT issues that I have reported on. The results of that work are due to be published in May. I am also reflecting on the remarks that Mr Stevenson made about the risks of relying on existing systems that appear to do what we want them to do, because we think that overconfidence was one of the contributing factors in the issue that we are discussing.
Do you want to add anything to that, Mark?
No, except to say that we are planning to publish work that pulls together a lot of the themes in our various IT reports.
I look forward to reading that with interest.
Mr Stevenson highlighted the off-the-shelf nature of the system, although it was a hybrid of off-the-shelf and development material. Some of the issues that were discovered later were down to the fact that the initial implementation plan and all the resources were based on that assumption about the system. The lesson learned from that is that, even though there was a very detailed tendering bid and the business scenarios that were outlined in it looked at all the functionality of the system, assumptions were made that led to difficulties later with the resources and implementation plan. Some phases had to be overlapped, which resulted in issues. So one lesson learned is, I guess, to make no assumptions around such a hybrid solution.
One of the things that indicates that a project has died is when change stops. As projects move forward, the deliverer and the customer better understand their needs and what requires to be done, because people do not know how their own lives work until they have to dissect them in detail. Do you have evidence from what you have seen that there was that learning process where the customer—in this case, Police Scotland—was continuing to learn and understand? As we know, we cannot write a spec at the beginning of a computer project because the final 5 per cent of the work delivers the project and the first 95 per cent is just working out what we need to deliver. Was that learning process still going on? Or was there a failure for that to progress because there was not a good structure in which it could happen for both Accenture and Police Scotland?
One of the positives that has come out of what happened is a much clearer understanding and documentation of Police Scotland’s business processes and how things are done. That was going on fairly early in the lifetime of Police Scotland. The business process mapping revealed some of the divergences in how things were operated in the various legacy forces. Work on understanding that has now been done, but it was an evolving process. I hope that Police Scotland will now have a clearer understanding in the future of whatever process it chooses to develop. With reference to what Mr Stevenson said about people knowing about their lives, I think that Police Scotland now understands a lot better how its processes work and will have a plan for how things should be done in the future.
So that will be a useful foundation for the next stage of learning as Police Scotland moves forward with developing systems that it needs in a different model.
Absolutely.
Auditor General, you said in your opening statement that the failure of the IT programme had wider implications for the modernisation of the justice system. Can you elaborate on that a wee bit and maybe comment on what you think needs to be done to put police and justice reform back on track?
Again, I will ask Mark Roberts to provide you with the detail. Exhibit 1 in the report shows some of the key areas in which i6 was intended to provide wider benefits over and above efficiency savings, and one of those was the criminal justice system. The aim was to make it easier to do things such as full case reporting and handling warrants and police citations, so it was not just about what the police do but passing that on through the court service. Mark can give you a picture of the impact that that is having or not having.
The full business case for i6 was strong on the importance of the police as a part of the justice system as a whole. A group was set up to link in organisations such as the Crown Office and Procurator Fiscal Service, the Scottish Prison Service, the Scottish Children’s Reporter Administration and the Scottish Courts and Tribunals Service, and that was a key component in the making justice work strategy and the Scottish Government’s justice digital strategy.
To give a bit more colour to that, in recent weeks, the Scottish Courts and Tribunals Service has published the latest phase of its evidence and procedure review, which illustrates that, if police provide photographic evidence to the courts, it has to be printed out in hard copy and handed over as photo books rather than being exchanged electronically. That is just one example where there is no connectivity between the police IT system and the wider justice systems. Again, that is in the plan for how Police Scotland’s IT should develop in the future. There also has to be the connection to all the other elements of the justice system as a whole.
Thank you, that is helpful. Can you estimate the approximately length of delay that you think that has caused?
I would be speculating wildly. I am sorry but we did not try to do that and it would be wrong for me to try to estimate it now.
To follow up on that, the Auditor General talked about the IT project being central to police reform and to the policing 2026 strategy. As Mr Roberts has rightly pointed out and as the Justice Committee has seen in our Crown Office and Procurator Fiscal Service inquiry, the digital strategy is very much central to the changes and reforms that that service is looking to introduce.
Are you confident that the learning in relation to i6 is being shared not just within Police Scotland and the SPA but with partners in the COPFS? Through the COPFS inquiry, we got the impression that some individuals were heavily involved in the digital strategy—they understood it and were all over the detail—but the vast majority of people were assured that the strategy was going to unlock all the benefits but they did not really understand how that would come about. That also appears to have happened with the i6 project in that Police Scotland and the SPA have not necessarily properly understood the detail or people have had different interpretations of what it would deliver. Is that learning being shared across public agencies?
My starting point is that it is hard for us to say. As Mark Roberts said, the original business case included a wider justice system perspective and had mechanisms for linking to the key organisations that have roles to play, so the starting point was good. As the contract got up and running, the difficulties arose so quickly after the contract was signed in June 2013 that people’s attention narrowed down to getting the system itself to hit the milestones and deliver as planned. We should not forget that the original planned go-live date was August 2015. Very soon after the problems arose, we were butting up against delays to that date.
If we step back and look at the new policing 2026 draft strategy, it is looking across the criminal justice system and the interchange of information about criminals, suspects, witnesses and all the different parts of the system that need to work together. However, we do not have any sort of plan for how that strategy will be delivered with IT as a central part of it. It is hard to be sure that the lessons have been learned more widely and that would be an important action to take forward, whatever the plan is for delivering the 2026 strategy.
Very briefly, we did not speak to external partners during the course of this audit; it is important to put that on the record. However, from our previous work in other areas, we can say that the justice sector is quite good and well co-ordinated as a portfolio. The work of the justice board provides a useful forum for bringing together all the various bodies. That does not guarantee that lessons will be learned and shared but certainly, from we have seen, it is quite an effective grouping and, because the bodies are operating as a system, the potential for learning is probably quite good.
In our previous session, we talked about the deficit, which is estimated at around £180 million, but it is climbing. The report reasonably points to the political context within which the reforms are taking place. We all acknowledge that the relationship between Police Scotland and the SPA is in a far better place than it was when this project was getting under way.
Nevertheless, we have been asked to take on trust some fairly heroic assumptions about the pace at which Police Scotland feels that it will be able to turn the deficit around. Given that that is combined with, if not a big-bang IT project, IT being central to the strategy going forward, does that not create some concerns? In trying to hit timeframes that are perhaps unrealistic, do we run the risk of, if not necessarily repeating the same mistakes, making a bunch of different mistakes that will end up with us being in the same place that we are in at present?
13:30
There is no doubt that the vision for the use of ICT in the 2026 strategy is ambitious, but that is not a bad thing. We should be ambitious about the way in which digital can transform public services, and policing is ripe for that because of the slow progress that has been made to date.
Would you not caution that we should have a strategy of underpromising and overdelivering rather than one of overpromising and keeping our fingers crossed that we can hit the targets?
As I say in the report, I would like to see a detailed strategy and plan for how ICT will support the change to policing across the piece, as well as how it will contribute to making the savings that were initially an integral part of the police reform agenda. I think that that will be a challenge. That is not to say that it is not achievable but, in the absence of a strategy and a plan, it is hard to be clear about whether it is challenging but realistic or challenging and unlikely to be achieved in the required timescale.
Are there lessons that can be learned about the process? There is a question about whether the waterfall approach was the right one or whether an agile gateway process would have picked up on the issues earlier and allowed either an agreement to be reached for the project to be abandoned or fundamental changes to be made to it. Do you get the impression that a gateway process is likely to be implemented this time?
It is very unlikely that Police Scotland and the SPA would adopt a similar waterfall approach. As we have said, the agile methodology and a much more compartmentalised, bit-by-bit approach to software development will be the way forward.
The Government’s gateway process, which provides assurances at certain points and on certain elements of programme management, is still in place and in operation. In our wider work on the ICT programme, we have seen the Government putting in place an independent assurance framework that allows it to become involved in projects at earlier points if there are any concerns or difficulties where an external partner might be able to help and smooth the process or provide a check. That additional layer of external assurance and governance has been put in place.
However, if we go back to the financial pressures that Police Scotland is under and, I suppose, the political pressures that come on the back of that, do you believe that the SPA and Police Scotland are fully cognisant of the risks of responding to that pressure in a way that stores up problems further down the line because they are being overambitious in either what they are trying to achieve overall or the timeframe within which they can achieve those milestones along the way?
That question about the extent of the awareness of the risk is perhaps one for the SPA and Police Scotland. The only point that I would make is that, relative to some of the timescales for development, 2026 is not far away. Given the scale of the financial pressure that you mentioned and the fact that we have previously seen underspends on capital budgets, all that ratchets up the pressure. I am sure that they are aware and cognisant of the risks, but they could probably describe better than we could how they are mitigating those risks.
What has been the impact of the financial pressure that the SPA and Police Scotland are under to make efficiency savings? How much pressure did that bring to bear on the fact that the contract was not or could not be terminated any earlier?
The report mentions our slightly different concern that this mattered to the police because the previous IT system failures that Mr Finnie asked about meant that there was real determination to demonstrate that they could get it right with this one. The project was important to Accenture globally because it expected to be able to deliver it and then win more business elsewhere on the back of that, starting with the work in Spain and then looking to Scotland and further afield, so it had a stake in making the system work.
Beyond that, the political interest in the reform agenda and the operational problems that policing was facing at that point with issues such as armed officers and stop and search meant that the optimism bias was ratcheted up. People were so committed to demonstrating that they could get this right that the option of saying, “Should we pull the plug at this point?” might not have been considered as seriously or as early as it might otherwise have been.
In saying that, I am applying hindsight. It is difficult to know whether there would have been a better time to do that. However, I think that our view as a team, looking across the available evidence, is that those things might have got in the way of people deciding to terminate the contract as soon as they might have.
Will that steep learning curve help the SPA and Police Scotland with whatever system is put in place to replace this one? They will be under far more pressure, and a far bigger spotlight will be shone on whatever they do next. Will that learning curve help them to be far more aware that things might not work and that they might need to step back and review what they are doing?
Again, that is an issue to explore with Police Scotland and the SPA. I entirely agree with a point that came up in the exchange between Mr Stevenson and Mark Roberts—the process has helped the police to understand their needs better. The experience has been salutary for them and they now absolutely understand the risks that are involved. The challenge is to turn that into a greater capability to deliver systems that will do what is required.
We also say in the report that it is clear that relationships between the SPA and Police Scotland are better than they were at the time and the roles and responsibilities are clearer in formal terms, and those are positive things. However, the programme of work that will be required to deliver the 2026 strategy is still challenging and ambitious.
I have a number of relatively short questions, which I hope will lead to relatively short answers. Was it understood what the whole-life costs of the system would be? It might last a couple of decades.
I do not know the answer to that question. I am sorry.
Right. That is fine. Do we know who owned the intellectual property that came from the development? Was it Accenture or Police Scotland or was it shared? As the Auditor General said, the supplier was looking to gain more business.
We will have to go back and look at the contract again and come back to the committee in writing on that.
That would be helpful.
Finally, on the future maintenance of the system, was there provision that a different supplier could take over the maintenance? In particular, was the source code—the necessary makings of the system—available by escrow or other means so that another supplier could pick it up and maintain it?
The contract was for on-going maintenance by Accenture.
Okay.
As there are no further questions from committee members, I thank our panel for attending today and for the evidence that they have given. We have covered quite a lot of ground in a fairly short time. Thank you very much for your co-operation in that.
The next meeting of the Justice Sub-Committee on Policing will be on 20 April.
13:38 Meeting continued in private until 13:43.