Public Audit and Post-legislative Scrutiny Committee

Meeting date: Thursday, March 12, 2020

Official Report 624KB pdf

---

Section 22 Report

---

“The 2018/19 audit of Disclosure Scotland”

Agenda item 3 is the section 22 report “The 2018/19 audit of Disclosure Scotland”. I welcome our witnesses, Lorna Gibbs, former chief executive of Disclosure Scotland; Gerard Hart, director of protection services and policy with Disclosure Scotland; Joy Bramfitt-Wanless, former programme director for Disclosure Scotland; Paul Johnston, director general for education, communities and justice with the Scottish Government; and Michael Chalmers, director for children and families with the Scottish Government.

I invite Lorna Gibbs to make a brief opening statement.

Good morning, and thank you for the opportunity to provide evidence to the committee in response to the Auditor General for Scotland’s section 22 report “The 2018/19 audit of Disclosure Scotland”.

I was Disclosure Scotland’s chief executive and accountable officer from September 2016 to February 2020, so I was in charge through the period that is covered by the Auditor General’s report. Joy Bramfitt-Wanless was the programme director for Disclosure Scotland from November 2016 through to September 2019. Gerard Hart is currently acting as the accountable officer for Disclosure Scotland while a new chief executive is being appointed.

I welcome the report, which demonstrates that Disclosure Scotland reached a significant milestone in the transformation of the disclosure system by completing the transfer of all its activities on to a new system in September 2019. The new PASS—protecting and safeguarding Scotland—system will save £8 million a year in running costs compared with the previous system. The Auditor General’s report highlights the challenges that were faced during the transition period and gives clear indications of lessons that must be learned going forward. It is important to note that, throughout the process, we ensured that Disclosure Scotland was able to carry out its vital safeguarding functions.

We accept that the transformation programme has taken longer than originally planned. That is due to the complexity of the programme, which is acknowledged in the Auditor General’s report, and the challenges of delivering a system that allows Disclosure Scotland to continually monitor and match 1.3 million PVG—protecting vulnerable groups—scheme members with tens of millions of pieces of new information that arrive in DS every day.

We accept that lessons must be learned, and we have already started to act on them. For example, we have done so by establishing a change delivery advisory panel as a sub-committee of the board to improve independent scrutiny and challenge.

The audit has given us the opportunity to reflect on the areas where we could have improved. We could have done better in relation to some of the areas that are outlined in the section 22 report, but I believe that the hardest part is now behind us and that the new system leaves Disclosure Scotland in a good place to face the future.

We also have an opening statement from Michael Chalmers.

Thank you for the opportunity to provide evidence to the committee following the Auditor General for Scotland’s section 22 report. In my role as director for children and families, I line manage Disclosure Scotland’s chief executive and I am Disclosure Scotland’s Fraser figure. I report to Paul Johnston, who is the portfolio accountable officer for this area of the Scottish Government’s budget.

We believe that Disclosure Scotland’s digital transformation programme is delivering benefits for the agency’s customers and will prepare the agency for the reforms that will be brought about by the Disclosure (Scotland) Bill, which is currently before Parliament. However, we acknowledge that there are lessons to be learned. The section 22 report raised valid concerns about the reporting and governance of the transformation programme in Disclosure Scotland and said that those should have been better.

Disclosure Scotland has already acted on the section 22 report by reforming its governance structures, and the Scottish Government is also acting on the findings in the report. Lorna Gibbs and I have provided training and reflections to agency chief executives. The Scottish Government is reviewing its processes and guidance for the proper application of optimism bias in significant projects for agencies and sponsored bodies. We are strengthening our support and guidance for agencies that are undertaking digital transformation to ensure that proper account is taken of the impact on the capacity of an organisation’s leadership to run the day-to-day business while undertaking major transformation.

We are happy to assist the committee in any way that we can.

I will open up the questioning with a question for Paul Johnston. This is just the latest in a long line of information technology projects that have come before the committee. Why are we so useless at delivering IT projects on time, on budget and to spec?

As my colleagues have already recognised, we have lessons to learn in this particular case. It is critical to recognise that the programme is delivering benefits to the public at this point in time. As the Scottish Government, it is vital that we are constantly learning lessons and improving.

I know that the committee has looked at a lot of IT projects. I have in front of me Audit Scotland’s June 2019 report “Enabling digital government”, which the committee looked at. It gives a very fair assessment of some of the areas of real strength and some of the specific things that the Scottish Government has put in place to ensure that we can build on the learning that the committee and Audit Scotland have highlighted. There are successes that we can describe, particularly where agile methodology has been used and is delivering benefits, but I absolutely recognise that we can still do more. The section 22 report on Disclosure Scotland helps us to further strengthen the provision of support, advice, good practice and challenge to others in the public service.

Is the project a success, or is it a failure to be learned from?

The project has delivered. As of today, it is delivering a better service to the public.

I was very interested to go on to the Disclosure Scotland website as part of my preparation for today’s meeting. I am sure that members will also be interested in doing so and seeing the simple interface that now exists for members of the public who wish to get a basic disclosure. That, for me, is an indicator of success in the programme.

You regard the project as a success, rather than a failure to learn lessons from.

I regard it as a success, in that it is delivering better services to the public. However, I also recognise that there are real lessons to learn, particularly around good governance and optimism bias. We have to learn lessons from that.

Lorna Gibbs, so far, the project has cost £78.5 million. The initial business case was £77.2 million, and, in the end, we went for a business case of £34.1 million. For the £78.5 million, are we getting the £77 million package or the £34 million package?

The package for the outline business cases that were done in 2015 aimed to deliver the same outcome. The business case that was originally done in June 2015 was set at the higher level and ministers were understandably concerned about its affordability.

Disclosure Scotland went back and considered a number of other options. It went out to the market with requests for information. It invited 10 organisations to comment on what would need to be done, the associated risks and the likely costs. Disclosure Scotland took those requests for information and what the organisations said they thought would be the likely outcomes, reduced the optimism bias—to an extent—reduced a number of other costs, and came up with the £35 million package. We used that process so that we could be sure that the later 2015 business case was clearer about the likely costs.

Both the 2015 business cases were based on a traditional waterfall delivery. Early in 2016, it became clear to us that that would not deliver a workable, usable system. We therefore commissioned an additional report from PA Consulting, which recommended moving to an agile delivery programme. The business case that was done in 2017 was based on agile delivery, which was a better way of exploring what we were actually looking to achieve.

I do not think that anyone would believe that the £34 million package would be the same as the £77 million package. No one from any walk of life would believe that they could be the same package.

They were aimed at the same outcomes.

Okay, but they were not the same.

The packages were aimed at delivering the same system changes. As I said, we went out to the market and got additional information, so costs were cut in a number of areas but nothing was taken out of what we intended to deliver in 2015. What we intended to deliver was the same in the outline business case that was done in June 2015 as it was in the outline business case that came out later in the year. We were just more confident that the costs would be somewhere in the region of £35 million because we had been out to the market.

Is it correct to say that, as it stands, we are not getting the full functionality that we would have expected from the business case and the £34 million package?

We are not at full functionality at the moment. The priority had to be to get us off the BT system, which was expensive, inefficient and not user friendly and would not have allowed us to make the changes that we are looking to implement from the Disclosure (Scotland) Bill.

There are two elements to the build. There is the back-end functionality, which allows us to do the processing, and there is the front-end functionality, which is how users engage with the system. There is a lot that we still need to do on the front-end functionality. Customers for a basic disclosure can apply online and use a good-quality online system, but we have not yet built the functionality to allow customers who want standard, enhanced or PVG disclosures to engage digitally, so more needs to be done.

As far as the public is concerned, you had a £77 million functionality package, which was knocked back, and then a £34 million functionality package was accepted. However, in the end, we spend £78 million, we do not even get the functionality that we should have got from the £34 million package, and there is still more spending to be done.

The £78 million includes the additional spending that we expect to spend over the next couple of years. That money should allow us to get all the systems online.

The £78 million will get us what should have been provided by the £34 million package, not the £77 million package.

No—the functionality provided in the £77 million package is the same as that provided in the £34 million package. It was only the cost, and not what we were planning on delivering, that changed in 2015.

However, as it stands, we do not have the functionality from the £34 million package.

No.

I will continue the acting convener’s line of questioning. I am looking at paragraph 8 of the Auditor General’s report, which says that the original outline business case from June 2015 projected that the cost would be £77.2 million. Given what Lorna Gibbs said, am I correct that the Scottish Government’s objection to that package was based purely on cost?

Ministers accepted that the system needed to be replaced. In 2015, we commissioned research that identified that the BT system was never going to be capable of delivering a properly functioning online digital system. Ministers understood that we needed to make changes, but they were concerned about spending £77 million on them, so they asked us to go back and do some more research in the business community to see whether the cost of £77 million was valid or whether the business community would say that we could make the changes for less money. We did that work between the June business case and the October business case.

You engaged with the business community in the first place to reach your original figure.

I ask Gerard Hart to pick that up, because I was not in the organisation in 2015 when the original business case was done.

The estimates in the original business case were reached in conjunction with knowledge from the existing BT contract. The request for information exercise to which Lorna Gibbs referred had not been done—it came after that. Ministers’ challenge related to affordability. They said that the estimates looked as though they were based on information that was perhaps too elderly, and that we needed to go back and look again at the way in which the market operated at the time. We should remember that that was the beginning of a period of innovation in IT in the public sector more generally. There were lots of new corporate entities in that marketplace, so there was a testing opportunity through the RFI exercise. It is now common for such exercises to be done.

Not all the businesses involved in the RFI exercise gave detailed costs, but three or four of the 10 did. Those detailed costs confirmed that ministers were right to ask us to look again at affordability, which allowed us to reduce the overall estimation of costs. Optimism bias accounted for only one third of the reduction in costs between the two business cases. The rest of the reductions related to savings that could be made; the information that we got from the market was about what was possible—the art of the possible. We probably had not fully appreciated that before we did that exercise.

Did you have a proper project management team in place?

There was a project management team, but, at the time, our structures were very much geared up to managing what was a long-standing contract with BT and the iteration of the existing legacy system that it had built in 2002. That system required to be replaced, and was replaced, in 2011 with what we now know as the PVG system, which BT provided. The team was set up to manage that contract and to procure the successor to the original legacy system. It was perhaps not the team that we would have had after that contract ended.

09:15  

I am looking at the timings. In June 2015, the Scottish Government rejected the outline business case. In October, it accepted a revised projected cost of £34 million. That is extraordinarily fast for an RFI exercise in the market. The process for getting Government approval is not very quick, so it all seems very compressed. How did it work?

Shall I answer that?

Yes, and I can add to your answer.

There was an appreciation that this was urgent. Even then, the BT system was understood to be ageing and approaching obsolescence. We knew from the technical intelligence that we had that that situation could not continue without significant, multimillion pound investment in the BT platform. Therefore, there was an urgency to do the RFI work and to accelerate the process so that we could come back with a clear plan for ministers as to how we could deliver the replacement.

Let us leave that aside. In June 2015, the Government rejected the business case. You would then have had to put together specifications on the project and what you expected from it. Fairly detailed information would have to out to get effective RFI proposals back from the market. That is not a quick process. How did you manage to do it in that time?

The specifications that we had were the ones that we had used to underpin the existing June 2015 business case. We used those specifications to go out to the market and do the RFI.

Those specifications were based on the BT offering.

Yes.

Surely you were trying to move on from the BT offering and on to something better. Surely you did not just take your existing specifications, go out to the market and say, “This is what we need”.

That is what we did back in 2015, and it became very clear that that approach was not going to get us the new digital system that we needed. That is why we switched our approach in 2016 and moved to a more agile delivery in which we were much more engaged in designing the system.

The RFI process in fact did not work.

The RFI process gave us an estimate of how we could go about delivering a waterfall system.

After that, when we went back out to the market to write a detailed specification for a waterfall delivery, it became clear that we were not going to be able to set out a system in as much detail as was needed to guarantee success for a waterfall delivery, because of the complexity of the system. Therefore, we brought in a different organisation, PA Consulting.

We spoke to the delivery directorate, and we discussed whether a waterfall approach was the right way to go, given the problems that we were having in specifying what we wanted the new system to do. That led us to take the agile approach, which the Auditor General agreed was the right approach for an innovative, complex and challenging IT programme.

It seems to me that there is clear evidence of poor preparation in going out to the market and expecting people to be able to give you a quote for a system that you had not properly defined.

The system that you have now is classified as a minimum viable service. A minimum viable service, plus all the manual workarounds that you are doing—at a cost of £2.7 million in the past year—is only going to give you the same service that the BT system gave.

I refer members to exhibit 2 in the Auditor General’s report, where she has clearly indicated that the minimum viable service plus the manual workarounds bring Disclosure Scotland up to the level of the existing baseline of the BT solution. The ambition is higher—

We have an online application process that meets the digital standards, which we did not have before, and, importantly, we have the ability to flex and develop the system.

At the moment, we are looking to make changes to the existing system that will allow us to implement our elements of the Management of Offenders (Scotland) Act 2019 and the Age of Criminal Responsibility (Scotland) Act 2019. We could not have done that with the BT system. Although we are some way from having all the digital front-line enhancements, we can make changes to the current system that we could not have made to the BT system. Although we do not have the full system that we wanted, we have a range of flexibility and the ability to change, which we would never have had if we had stuck with the BT system.

Is the depiction in exhibit 2 on page 7 of the Auditor General’s report incorrect?

It is not incorrect in terms of the services that we provide; however, it does not take account of the flexibilities and abilities that we now have to change the system to adapt to changes in legislation. That is one of the key benefits of the system.

We could not change the BT system. If we had stuck with that system, we would not have been able to bring to the Parliament the Disclosure (Scotland) Bill, which is going through at the moment, because we could not have made changes to the system. We have the ability to own the system—to change it and to bring in innovative new policies, which we would never have been able to do with the BT system.

My final question is about optimism bias. Why is the level of optimism bias throughout the lifetime of the project lower than that recommended by HM Treasury’s green book for such an innovative and complex project?

It was within the recommended levels, although at the lower end. Clearly, one of the lessons for us is that the level of optimism bias was set too low, both in 2015 and in 2017. We, and the Scottish Government more widely, were wrong about that. We were overly optimistic, particularly in 2017, about how quickly we would make progress on bringing products online. That led us to add in a low level of optimism bias, particularly around timing. In hindsight, that was incorrect.

Just to be clear, are you saying that you were within the recommendations of the Treasury green book?

We were very much at their lower level.

Okay.

Whose decision was it to go to that level?

The original business cases were signed off at Scottish Government level. It had gone through a process of detailed discussion with Scottish Government finance and with the digital directorate, and eventually was signed off by ministers.

So, to be clear, the decision to go to the lower level of the range was proposed by Disclosure Scotland, but signed off by the Scottish Government.

The overall business case, which contained the optimism bias, was explained and signed off by the Scottish Government in 2015. The level of optimism bias was part of that.

I ask Paul Johnston whether that is correct.

I accept that, and I accept that it is an area of learning for us. We are already ensuring that the lessons from this case are shared with others who are at the point of developing business cases now. We need to ensure that optimism bias is not put in at a level that is too low.

Coming to this from the outside, I am not quite sure that lessons are being learned. People keep saying that they are learning lessons, but there is not much evidence of that.

An outline business case was made in June 2015. As somebody who does not pretend to be an expert in this, I would like to know how long it would have taken to draw up that case.

I ask Gerard Hart to answer.

I was peripherally involved at that point. I was specialising in managing the PVG barring service, so I was not part of the transformation or IT programmes at that time. However, I was on the senior management team, and I remember that it took several months to draw up that business case.

Just months?

Yes.

So, from the beginning of the project, with presumably some rigour somewhere, a figure of £77 million is produced. Within two months, it has gone down to £34 million.

We had had significant external advice, not just on the matter of the replacement of the BT platform. It was not just about the functionality of the BT platform. Even then, it was clear that a significant degree of obsolescence was going to be found within the BT platform. The exam question was not about rebuilding that system exactly as it was—

With respect, that is not the question that I am asking.

I understand that there is complexity, and I presume that the people who devise such a business case know that there is complexity. I hear everything that you are saying—that it is all very difficult—however, we have ended up with a system that is marginally better than the previous system but has cost £78 million.

Somebody somewhere produces an outline business case. Within two months, the projected cost can go down to £34 million. Who, in the system, looks at those two figures and decides that either somebody does not know what they are doing in producing the figure of £77 million or they do not know what they are doing in producing the figure of £34 million? It is simply not credible to think that the same project could cost either one or the other. Is it the Scottish Government’s job to look at those two figures and think that somebody somewhere does not know what they are doing?

My colleagues have tried to set out the work that led us to go from £77 million to £34 million. It is right that the Scottish Government did not accept the figure of £77.2 million, because, when the request for information went out, what came back from that exercise was material that allowed Disclosure Scotland to put together a business case at a reduced cost. However, in 2020, with the benefit of hindsight, we recognise that the figure of £34.1 million was too low.

With respect, the only benefit is in hindsight, because the project has now cost £78 million. I assume that it is a technical job that needs to be done by somebody with technical expertise. You are not saying that you are offering a poorer service for £34 million. You must have looked at that figure and said that somebody had got their sums wrong in arriving at one or other of the figures. Who looks at those two sets of figures and says that they are not credible? You need to look at the people who came up with the figure of £77 million or the people who came up with the figure of £34 million.

I have sought to explain that, in 2015, the figure of £77 million was viewed by the Government as not affordable and therefore requiring further work.

Therefore, the Scottish Government asked for a poorer service.

No. The Scottish Government asked for further work to be done on the business case.

The Government said that it was not affordable, so somebody came back and said that they could make it affordable, with no detriment to the service. Why do we have people in the system who produced the original figure? You have two things that will deliver the same service. One person says that it will cost £77 million and, when the minister says that that is a bit expensive, two months later, another person says, “It’s all right, you can get it for £34 million.” Whose job is it to look at those two figures and bridge that massive credibility gap? Technically, there is a credibility gap.

The Scottish Government said that the £77.2 million business case required to be revisited. That is what led to the RFI exercise, which gave Disclosure Scotland the basis for revising the business case and coming up with the figure of £34.1 million.

That reduction was later established to have been a fiction, because the cost is now back up to even higher than it was before. I hear what you say about complexity, but you knew on day 1 that the project was complex and how important the scoping process was. We now hear that the BT project was hugely expensive and not fit for purpose, and we still had to extend the contract. Everything about the project was known on day 1, when that 2015 figure was produced. Am I right?

We knew that the project was complex. It was not until we started to explore exactly what we wanted it to do that we realised how complex it was. The BT system had built up over time, and we discovered subsequently that there was nobody who could sit down and write a detailed specification. That is why we—

Sorry—what do you mean by “we discovered subsequently”? Subsequent to what?

Once we had had approval for the original business case, we needed to write a specification to go out to the market for a procurement for waterfall delivery. We started to set out in detail what the system does and what we want it to do. At that point, it became clear to us that we could not specify in sufficient detail what we needed the system to do for waterfall delivery to work. That was when we brought in experts, who said that waterfall delivery will never do it for such a complex system, in which we manage 1.3 million members and compare them to tens of millions of pieces of information.

That is when we took the decision to move to agile delivery, which allows us to do a lot of user research, build the system slowly and get what we and our customers need over a period of time. Agile was how the project was going to deliver what we have now—a system that the customers who use it online are positive about—and it gives us the ability to change things as the legislation changes.

However, it is a minimum viable service; it is not what was anticipated in 2015.

It is not yet the full range of digital front end.

But it has cost more than was set out in the business case in 2015, which was thought to be unaffordable.

Yes.

09:30  

I have a question for Paul Johnston that follows on from Johann Lamont’s questions. From a real-world perspective, if a business was looking to have some renovation work done and got one quote for £77,000 and another for £34,000, or if someone was redecorating their house and got one quote for £7,700 and another for £3,400, they would ask themselves, “Who is at it—is it the person who gave me the first quote, because they are bumping me, or is it the person who gave me the second quote, because they will add on lots of costs once the project starts?” With IT projects such as the one we are talking about, who in the Scottish Government decides who is at it? Do you think about that?

That is into the territory of the valuable learning that we have from the exercise that Audit Scotland has done and from the Disclosure Scotland work, and—

I am sorry to interrupt, but, to put it more directly, in this case, when you got the quotes, did anyone in the Government think about who was at it?

In this situation, the £34.1 million revised business case was accepted and the money started to be put in to enable the work to be done, based on the RFI exercise that had taken place.

Using the passive voice and saying that the business case “was accepted” does not explain who accepted it. Who had the responsibility for looking at the business case with a sceptical eye, as the convener suggested should happen?

We have a number of functions that have responsibility for that. In particular, those are the finance and digital functions and our colleagues in children and families. Michael Chalmers, as the current Fraser figure, might be able to say a bit about the scrutiny of the area that goes on in the Scottish Government.

Absolutely. As the committee has heard, the business case was based on the RFI exercise. That exercise involved 10 suppliers from the market providing information, so the figure was not plucked out of the air. We absolutely accept that, with the benefit of hindsight, we would not do some things in the same way again—in particular, I am thinking about how the optimism bias was set—and we are seeking to learn lessons. In developing the business case, our finance and digital colleagues engaged directly with Disclosure Scotland’s team, and the business case was based on information that was provided by 10 contractors in the market; it was not plucked out of the air. The earlier outline business case in June 2015 was not such a detailed exercise.

Mr Kerr has a follow-up question on that.

It is very brief. Audit Scotland concluded in its report that the governance arrangements at the time were not clear and that several organisations and bodies were feeding into the process. I presume that you would say that that is partly the reason for what happened. If so, why was there such ambiguity and for so long about the role of the various groups that were involved?

My understanding is that the reference to ambiguity relates primarily to 2018, which was when key decisions were being made on the extension of the BT contract. At that point, Disclosure Scotland had a number of governance groups. We had the Disclosure Scotland board, the transformation programme board, the leadership team and the audit and risk committee. As a relatively small organisation, we had a relatively small leadership team—it was me along with four directors. They were on the board and, obviously, part of the leadership team. They were also on the transformation programme board, and they attended the audit and risk committee. The lack of clarity and certainty in 2018 came from the fact that five people were wearing a number of different hats at various points.

In 2018, our focus was on what we needed to do to get off the BT contract. Our operational governance—the way in which we managed the agile delivery—was strong and was commented on positively by the independent assessor that Michael Chalmers subsequently brought in to give him extra assurance. The strategic governance was not as clear, because we were focused just on what we needed to do to get off the BT contract. There was not enough clarity about what hats we were wearing at any particular time because we were on so many different groups.

Forgive me, Ms Gibbs. You say that the ambiguity arose in 2018, but all the things that the convener, Colin Beattie, and Johann Lamont have just been talking about happened in 2015 or thereabouts. In fact, what led to this problem was not some ambiguity around internal governance; it was something far more fundamental involving all the agencies that are before the committee today. Is that what I am hearing?

In 2015, because we were looking at waterfall delivery, the roles that the different governance structures within Disclosure Scotland were playing were clearer. The transmission programme board and the Disclosure Scotland board looked at the draft business case, which then went up to be considered by Scottish Government finance, Scottish Government digital and, ultimately, ministers.

In 2015, when the business cases were being made, there was an understanding of the roles of the different bodies within Disclosure Scotland, whereas in 2018, when we were in the thick of actually delivering, the situation became much more fluid.

In the Auditor General’s evidence, she mentioned the importance of governance being in the right place at the outset of a project like this, and, again, that is something we will reflect on. If we had our time again, we would be looking at the governance changes that Lorna Gibbs has described, which were made in response to the section 22 report at the outset of the project. We accept that, and it will be part of the learning that we undertake.

When we recognised that the project was going to take longer than we had originally planned, and therefore the expense kicked in, during the autumn of 2018, we spoke to BJSS, as the contractor, and the Disclosure Scotland leadership team, and we sought outside and third party assurance from Capgemini, the private sector firm that undertook an assurance process for us. We also sought all the other assurance that was the norm for a project like this. Nothing that we got from any of them suggested that there would have been a way to do it more quickly. That is quite important. We accept that the governance was not right at the outset, but whether it would have led directly to a quicker completion of this transformation, I am not so sure. The third-party expertise that we brought in did not tell us that.

I would like to ask some questions about the IT skills side of this particular story. Does your organisation have sufficient IT software development skills in-house? Were they involved in the design and delivery of this package in any way, shape or form?

They were. In 2016, when we moved over to agile delivery, we looked at the skill set that we had and we created, in effect, a composite team. We brought in BJSS, which had a proven track record in agile delivery, and it brought in a number of skills that we did not have at the time. We brought in some independent contractors and we had our own people. At our office in Glasgow, we created a composite team with a range of different experience and expertise so that everybody was working together to deliver the software that we needed.

As we have moved more towards a steady state, we have trained up our own people to run and maintain the new system. That is challenging. As I think we discussed the last time, software development is a very competitive market, so we are using contractors in some areas when, ideally, we would like to be using our own people. It is a work in progress to ensure that we have embedded in the organisation the skills that we need for the future. The digital directorate is looking more generally at the skills that are required for the public sector.

Lorna Gibbs, I refer you once again to the chart and the timeline in exhibit 1. Members have talked about the business case dropping from £77 million to £34 million and so on. Was your software design development team at the heart of the specifications that led to the £34 million estimate?

At that period, we were using waterfall delivery, so we were not looking at doing our own software development; we were looking at outsourcing.

Who was doing the looking? Was it your software team or the management team?

We did not really have a software team at that point, because we had a managed service from BT. BT was, in effect, doing all the software analysis, all the builds and all the looking after. We were working with BT in 2015 to get its input into the design of the system. The approach that we took subsequently was quite different.

In October 2015, you arrived at a proposal with a cost estimate of £34 million, but BJSS was not appointed until August 2016. How on earth could you have thought that what was proposed in October 2015 was what was going to be delivered, when the contractor had not even been appointed at that point?

That was when we changed the approach. In 2015, we were still aiming to go out to the market and commission somebody to draft everything and install the system for us. We then realised that that was not the right way to go, so we changed our approach in 2016. Once we had done that, we went out to the market to get a business partner to come in and do the initial stage of agile delivery, which is when BJSS came in.

That is the starting point for me: that is where it started to go wrong. The contractor that would ultimately deliver the project for you was not on the scene until 2016. Prior to that, work and estimating were being done—the initial estimate was £77 million, which went down to £34 million. The contractor was not involved; it was not at the table at that point. How on earth could the project have been delivered when the contractor had not been sitting at the table with you?

We took a very different approach. It was almost as though we stopped the clock and stopped our thinking and started using a very different approach.

You started again, basically.

Yes. With hindsight, we should have asked ourselves whether we were sure that in September 2016, when BJSS started, we would be able to deliver by March 2018. That is not a particularly long time. Following the principles of agile management—which Joy Bramfitt-Wanless can talk you through, if you wish—we started with BJSS. There was not just BJSS; BJSS as the contractor, our people and interim people were a team. We were starting to do the work to determine what we wanted the system to do, and we were following proper agile principles to deliver that. The approaches that were taken in 2015, 2016 and 2017 were very different beasts.

Aye, I know what agile methodology is and what waterfall methodology is; I know all that stuff. We have heard it all before here, and are quite familiar with it.

According to the permanent secretary, who wrote to us yesterday, the digital first assessment of the programme took place in November 2016. I presume that that was the first time. However, much work had been done prior to that. My question is probably for Paul Johnston. Why was the digital first assessment done only in November 2016, given that all that work had been done before then? It seems that there was no assurance of the design proposal or of the cost estimate until that point.

Michael Chalmers might want to say more about this in a minute, but I would say that the Scottish Government was involved from the outset. The Scottish Government has refined our processes for oversight of such projects, which is what is being referred to in that letter.

I would add only that the Scottish Government digital directorate was engaged with the team as the business cases were developed.

What happened in November 2016? I am referring to the permanent secretary’s letter, which says that there was a digital first assessment in November 2017 and that the first assessment was in November 2016, which was after BJSS arrived on the scene. What was the Government’s involvement before that?

I could check the chronology, but I can be very clear that there was Scottish Government involvement from the outset. However, the types of assessment that the Scottish Government uses have developed over the years. Therefore, what is described to you in the letter are the ways in which we have assessed the programme over recent years, as we have learned and developed and matured our processes for oversight of such programmes.

The digital first assessment looks specifically at what we are doing, our use of research and our engagement. It could not have looked at that any earlier than 2016 and 2017, because we had not started doing any of it. It came in at the appropriate point, once we were starting to do the agile delivery work. One we started the agile delivery and to actively do the work, the processes around that, including health checks and digital first assessments, started to kick in. That was in 2016, when we were really starting to get into delivering the agile process.

09:45  

I accept that, but a heck of a lot of work went on before that point in August 2016, and it seems almost to have become redundant, because you basically started again in August 2016 with BJSS. From a software development perspective and a cost-estimation perspective, should not some kind of assessment of the project have been done much earlier? It is as though you threw away the work that you had done up to August 2016 and started again.

When the business case for £34 million was agreed, the team did a lot of work on trying to develop the requirements to build the service. It took the members of the team until the following year to realise that they could not do that. It was not possible for them to stipulate up front what was required, so we had to take a different route. We could not use the work that had been done, because we simply could not define the requirements up front and would have got a system that did not work if we had. We had to discount the work that had been done and start using the agile approach. We accepted that—

Surely you would have thought of that before you came back with a business case for £34 million. You are saying that you did not know, but you would have thought of that before the business case was agreed, not afterwards.

I understand that. Gerry Hart might want to come in. When the business case for £34 million or £35 million was agreed, the business still thought that it could stipulate the requirements to deliver the system. It thought that it could get the money, write all the requirements for the system and have somebody build it. That is what the business case was for.

Two business cases did that.

Yes.

Until that point, the team thought that the waterfall method could deliver the project, but it was decided at that point that that was no longer possible and that it would be better to use the agile approach. The costs then shot back up to £78 million. What on earth do you think the public would think of that? You chose a development methodology that you thought would work, but which eventually doubled the price of the methodology that you rejected.

When we put together the business case for £34 million, we could not stipulate the requirements up front. By using the waterfall approach we would, ideally, have drawn down money at certain points as we learned more. Unfortunately, we are required to develop a business case up front. We have accepted that we did not know the requirements to build the system. We were required to come up with a cost for building it, although we did not know how long it would take or how to build it. We understood that the system that we had to build would be complex and unpredictable, and that we would learn as we went along. That was why we decided to use the agile approach. The intention was to keep revisiting the costs: that is what we did, when we found out that something would cost more.

We looked at costs year to year. We had conversations with the Scottish Government’s finance team and the Fraser figure about our budgets for every year. We did not go back and fully redo the business case, but every year we told the Scottish Government what we thought we would need for the next stage of the process. Of course, it has cost more than anyone would have wanted, but we have a system that works, that can flex and change, and is delivering a service that our customers want. I think that we could never have delivered such a system through a waterfall delivery system. The agile approach was the right one to take.

I am sorry to keep pressing you, but the agile methodology has been around for a while; it did not just arrive in 2016. It was around when you embarked on the first business case in 2015. Where were the people with expertise in your organisation, or in BT, who could—and perhaps should—have said that the waterfall methodology would not deliver what was required? Why did it take you so much time to discover that you had adopted the wrong methodology? Where were the people who should have told you to deploy agile methodology at the outset, rather than halfway through the project?

We did not have that expertise in the organisation at the time. We called in PA Consulting in 2016 to ask it what would be the best way for us to go ahead. We did not have people who had used the agile approach, which is why, when we subsequently made the decision, we brought in Joy Bramfitt-Wanless. Things have moved on significantly. If we were starting the programme again, we would automatically start with agile because of the complexity of the work that we had to do.

The same question goes to Mr Johnston. You must surely have been aware that such development methodologies were available to be deployed in IT projects. Why did you not ask at the outset of the programme whether the right methodology was in place? Why did it take so long to come to the conclusion that agile was a better delivery methodology for the programme?

That is a good question. I can speak only for where we are now: it seems that all those who have examined the programme are saying that agile is the best methodology. We need to ensure that the proper methodology is deployed in the range of other current IT development programmes, and that when agile methodology should be used, it is used.

Does the organisation currently have the IT and software development skills to enable the project and others to be developed in a consistent, cost-effective and timely manner? Do you have the skills in-house to continue to deliver the requirements of this and future projects?

We do, at the moment. Some of those people are contractors, as we are trying to upskill our own staff. At the moment, I am confident that we can run and develop the system with the people that we have.

I want to go back to the governance issue. Lorna Gibbs said that, in 2018, you had many boards. At our meeting on 23 January, I noted that your website showed

“only minutes for September 2019, audit committee minutes for November 2018 and the 2018 annual report and accounts.”—[Official Report, Public Audit and Post-legislative Scrutiny Committee, 23 January 2020; c 12.]

We were also told by your auditor that he had raised that with you. Perhaps this question should be for those who are still in post. What approach does Disclosure Scotland now take to publication of board papers and minutes?

That has been discussed. We are working to identify the correct platform on which to publish papers as soon as possible. There is a commitment to that. Our current website situation is that we have some pages in a corporate section of what is primarily intended to be a service-oriented website—the mygov.scot website. We are currently working with Scottish Government colleagues to find a way to host the board minutes, which will happen in the immediate future.

What is on the website now?

There are some minutes on the website, but we have not yet been able to upload all the minutes that we intend to upload. Members will see that happening over the next couple of months.

I understand that the most recent minutes are on the website: whenever there are new minutes, they replace the ones that have been on the website.

I looked at the website again yesterday—I do not know whether you did so in your preparation for the meeting—and it seemed to be that the same minutes were there as had been there previously.

We are having discussions with—

How much discussion does it take to put a set of minutes on a website? There are minutes on there at the moment that could be replaced.

There are complexities around that because of how the website is set up and its purpose. However, we are finding solutions. A project has been set up to ensure that not only board minutes but other valuable corporate documents are hosted. Members will see that happening in the near future.

Why will that take two months?

It is not a technical challenge. It is a challenge to do with what the mygov.scot website was set up to do and its set-up making it not the most appropriate place to host a large number of corporate documents. Physically, we could put up more minutes on the website, but it is not what mygov.scot was set up to do.

But there are minutes on the website.

There is currently a set of minutes on the website. Audit Scotland and our internal audits have encouraged us to put a suite of minutes online. That is not what mygov.scot website was set up for: it was set up to be much more service based, so it is—

Even though the website is not set up to do that, if you currently have minutes on the website, why can you not add additional minutes? Why would that take two months?

We do not own the website, so it is not entirely up to me, or Gerard Hart, to say that we will put X on it. We had a Disclosure Scotland website that was not fit for any digital standards, so we moved on to mygov.scot, in order to be consistent with that user focus. There are quite strict rules about what organisations are allowed to put on the website; the Government is not comfortable about having lots of sets of minutes on it.

At the moment, it is not in our gift to say that we will upload six months’ worth of minutes to the website. There is a negotiation—

Will you be carrying on that line of questioning, Bill?

You have anticipated most of my questions.

Your auditor raised the issue, as an overseer for the Scottish Government. Is there no danger signal? Do you not red flag an organisation that is not up to date with its minutes and disclosures?

We have accepted that the overall governance arrangements were not good enough, and certainly—

But you did not do anything about it.

I have engaged with boards and the transformation boards during the time when I have been the Fraser figure. I certainly think that, if we were looking at the issue again, we would focus more on the governance arrangements.

At the time when we were engaging with Disclosure Scotland, it seemed to me that the biggest issues were to do with, and the focus was on, the digital transformation programme. On reflection, the governance should have had more attention.

Does the situation apply to other organisations that are hosted on the mygov.scot website?

I do not think that other agencies are using mygov.scot in the way that we use it. Many have their own websites; therefore, they have more freedom to put up as many minutes as they, and the auditor, agree is appropriate. I would need to double check, but I think that we are the only agency that uses that website.

How do you know that? Are you speaking from your knowledge of Disclosure Scotland, or do you have broader knowledge?

I have broader knowledge. I was part of a group of chief executives who would get together regularly for conversations about issues including websites and how we use mygov.scot. I know, for example, that the Student Awards Agency for Scotland has its own website that has minutes on it. However, I cannot speak for every single agency. We can check and confirm the position, but as I have said, I think that we are the only one that uses mygov.scot in that way.

We have a project running right now, managed by a director, to solve the issue. The correct solution will be identified and minutes will be uploaded in the near future.

Surely Paul Johnston has the solution to the problem.

The solution is that we need to get the minutes online. I undertake to take the issue away, ensure that that happens quickly and ensure that the committee is kept up to date when that is done.

I make it clear that doing it in two months is not doing it quickly.

I agree that we need to do all that we can to get the issue sorted immediately, given that the issue has been raised by—

So, it will be sorted in weeks, not months.

Let me take the matter away and get back to the committee. I want to understand what the options are for sorting it immediately.

I agree that it seems to be overly complicated to get minutes on a website so that they are publicly available. We need to get that sorted, so I commit to getting back to the committee with an update.

You are talking about it taking two months. When you respond, will you also tell us how long you have known and been thinking about the issue?

Okay.

I am less surprised that there are problems with the whole project if people are incapable of getting minutes on the mygov.scot website, which I presume is run by the Scottish Government. Is that right?

Yes.

It is not a cupboard or a filing cabinet. I understand that the joy of the internet is that astonishing amounts of information can be stored online. Are we seriously saying that a senior official must go away and work out whether it is possible to find a way to get past a process in order to put minutes up on the mygov.scot website?

Joy Bramfitt-Wanless is now in a senior role in the digital directorate, and might wish to pick that up.

No. I am asking you, Mr Johnston. What on earth is such a problem that we cannot simply agree that minutes from the organisation should go up on the mygov.scot website? I presume that it is not a question of one set in, one set out, and that you can have an archive, too.

I have no idea how the technology works, but even I know that we are not limited in capacity as though the website were a filing cabinet.

I have said that we need to get it sorted, and we will get it sorted.

I look to Joy Bramfitt-Wanless to provide any additional explanation about why we find ourselves in this situation.

Mr Johnston, the Scottish Government is responsible for mygov.scot—is that right?

Yes.

The Scottish Government will agree to be the host of the minutes for Disclosure Scotland.

Yes, that seems reasonable to me.

10:00  

I am sad to say that IT projects come to the committee with monotonous regularity. Looking at the permanent secretary’s letter to the committee, I see that more than 400 projects are registered with the digital audit office. Are you satisfied that all the organisations that procure, design and develop IT software-related projects have enough skills to deliver the projects on our behalf? Have they got the right standards in place? Do they know about things such as the agile methodology and so on? Can you assure the committee that that is all in place?

We know about Audit Scotland’s report “Principles for a digital future”, and we expect everybody to embrace it whole-heartedly, but can you assure the committee that that is actually happening and that we are not going to see another trail of IT projects coming before us?

The permanent secretary’s letter has set out the measures that are in place that seek to ensure that we develop greater capacity, competence and oversight of what I recognise is a vast number of IT projects. We want and need to be making swift progress across those projects. I want us to be an organisation that captures, deploys and spreads the learning. The permanent secretary’s letter sets out the way in which we are seeking to do that.

My hope—and my expectation—is that the committee will see an increasing number of examples of IT projects being delivered and delivering benefits to the public. I come back to the fact that, for all the issues that are being discussed this morning, this particular project is delivering a service to the public that is safe and which, crucially—this is something that we have not focused on yet—is estimated to deliver year-on-year savings of £8 million. I know that the committee will be keen to scrutinise the extent to which those savings are delivered on a year-on-year basis. It is important that we recognise the need for the project, the benefits that it is delivering, the underpinnings that we now have that will enable change, reform and improvement in the future, particularly through the Disclosure (Scotland) Bill, and the ability that we now have to deliver year-on-year savings.

I quite accept that there is more to do here, and that we need to make sure that the learning is spread across a range of other projects. We have built the infrastructure through the digital assurance office, and the training and upskilling that we are doing, to ensure that those lessons are learned and that practice is increasingly building in that learning from the outset.

That was a yes, then.

I cannot assure the committee that it will not be looking at IT projects in future. I am quite sure that there will be continued learning, but I want us to be taking the learning that we have now and applying it right across the system.

Liam Kerr has a specific supplementary question to Mr Coffey’s questions.

Mr Coffey made a good point. The Scottish Government has put in place an assurance framework for managing IT projects. Would that have captured the difficulties experienced by the past project?

That is a good question. We can look at the various gateway reviews and assurance measures that were put in place. As Michael Chalmers said, those that have been done have given us a degree of assurance that the Disclosure Scotland programme was on the right track.

In my opening remarks, I mentioned that the review is looking at how we apply optimism bias, for example. We would not necessarily have done things any more quickly but, at the outset, we might have taken a better approach to the planning for the governance and the issues that the committee has raised about the business case.

I want to press you on that. I was trying to follow what you said, but I am genuinely not clear what you were saying. Has someone reviewed the new framework that has been put in place, I presume, specifically to capture that sort of issue and said that the framework is fit for purpose and is a great process, and that if it had been in place, it would have picked up the problems with the project? If no one has already done that exercise, is it being done now?

That is what I am saying that we are doing. We are reviewing how we look at optimism bias and support the preparation of business cases for agile projects, along with all the other issues I mentioned in my opening statement.

A further question is begged. In January, we heard that 300 projects were listed on the assurance framework, and I think that Mr Coffey is saying that there are now 400 projects on it. We also heard that it had fairly limited resource. Do you have any plans in place to increase that level of resource?

I cannot speak to the exact level of resource. I am happy to take that away and provide the committee with an update on what the current resources are in the digital assurance office, and whether there are plans to increase that further.

That will be worth doing. The Official Report will show that Audit Scotland told us that there is quite a limited resource available. Clearly, if there are 100 more projects on the books than we thought, it would be worth having that information.

I will come back to the committee on that point.

Mr Kerr asked whether the new framework would have picked up the problems, and Michael Chalmers seemed to suggest that there is currently a review of whether that would have happened. Surely it makes sense to do the review first and then agree the framework rather than set a framework and then decide whether the framework would have worked.

On that point, part of what I am reflecting is that there is learning for us in the section 22 report and in the Auditor General’s evidence. It is proper that we consider the issues that have been thrown up by that report and that evidence and see whether the framework that we have up and running takes account of those.

Are you talking about a new framework?

No, I am talking about a review of the learning from this—

Would the new framework that Mr Kerr asked about have helped to highlight the problems at Disclosure Scotland?

That is what we are looking at.

My point is that, if a new framework is supposed to reduce the problems, it would make sense to conduct a review first and then decide the framework instead of setting a new framework and then doing a review. That seems like a rather confused approach.

I want to come in on one specific point. The letter that the committee received yesterday described, for example, the stop-go mechanisms that are now in place as part of that framework. My expectation is that, in the light of the learning from the report, the colleagues who are operationalising that framework will be examining factors such as optimism bias to ensure that the approach is pitched at the right level. I would be happy to get back to the committee on that point, because I quite appreciate that we need to ensure that the learning is being applied now in the cases that are still being developed.

I think that what the committee would like, in response to the question, “Will the new framework help to reduce the risk of the problems that are happening in Disclosure Scotland, so that such problems do not arise in future?”, is for you to be able to say, “Yes.”

That is exactly what it is intended to do.

Can you say that that is the case?

Yes.

What is the lifespan of the new IT system that we have been talking about, which costs £78 million?

Because of the way in which it has been built and maintained, it is capable of changing and flexing. It is not like the BT system, which had built-in obsolescence. There should not be an end date. We can replace different parts of the technology and we can change the way that we do things. It is not something that has a particular lifespan.

But doing those things would require an additional spend beyond the £78 million.

For continuous improvement, yes, it would.

What kind of figure is budgeted for that?

We do not know yet. We know that the Disclosure (Scotland) Bill that is going through Parliament at the moment has some high-level digital ambitions in it. We need to do a lot of user research to take those digital ambitions and turn them into something that our users want. Once we have done that, we can go through the agile process and cost them.

What I do not want to do with the bill is what we almost did in 2015 and say at this point, before we have done any of the user research or planning, how long any of that will take and how much it will cost. I think that that would just be replicating the things that we did wrong in 2015.

The bill completed stage 2 yesterday, and it should complete stage 3 and get royal assent within the next couple of months or so. Are you already scoping out the additional work that will be required in order to implement the provisions of the bill?

We are starting the very early stages of that. We know that there are digital ambitions such as the goal of enabling a PVG scheme member to share their record digitally. However, until we do detailed user research, we will not know what our users want in that regard. It is an on-going process that we will use agile technology to do.

How long will that take?

Until we start the process, we do not know. We need to do the user research; once we have done that, we will have a small business case for a discrete piece of work, which we will need to get approval for. If we get the approval for it, once we know what we are going to do, we will be able to say how long it will take. At the moment, I cannot say how long it will take, because we do not know what we are trying to do.

I realise that you cannot give me a precise date, but I am trying to get a feel for the order of magnitude. Will it take a year, two years or five years?

Although I am speculating and would not be want to be tied to this, I suspect that we are talking about a couple of years. However, until we do the detailed user research, nobody will be able to give you an exact timescale, because that is what tripped us up before. We need to do the detailed user research and then get approval to spend the money. Therefore, it will take a while.

I will add to Lorna Gibbs’s remarks. Not all the content in the bill requires a digital solution; some aspects can be delivered without a corresponding IT build. The approach must be iterative—we must look to deliver the value of the bill as soon as we can and build out the functionality that is needed to deliver the larger pieces. For example, the largest piece that we have to deliver in the bill will be managing a mandatory scheme of 1.3 million members, perhaps with different start-and-stop points in a time-limited membership period. That all requires IT computing power. Other aspects, such as the 12 to 17 stuff that is in the bill around younger people with offence exclusions, can perhaps be considered in a different way.

As you said, the scheme for 1.3 million members is a key provision in the bill. How long will it take to implement that part of the bill?

It is about stages. Our ambition is to deliver a way to apply for PVG online, as a stepping stone on the way to the wider functionality that will be delivered with the bill. Therefore, it is not about black and white; it is about improving the service successively over time. We have extremely robust ambitions about delivering the online application process under the existing law as soon as possible. That will take us a long way towards meeting some of the 2015-16 onward aspirations that our customers told us about. We need to take it in that stepped way and deliver the value iteratively.

I hear what you say about each stage and the fact that you need to do the consumer research before you can identify what needs to be done, and that you cannot finalise the timetable and the budget until you have done that. However, have you agreed the methodology of how you will go about implementing the new provisions in this bill?

We will use agile methodology.

Right. If you were doing something new in the private sector, you would have an adjunct to your business plan that said, “We have this additional remit and here is how we are going to go about implementing it.” I accept that, at this stage, the dates will be unspecified for achieving some of that, but do you have such a plan?

We have a high-level plan. Embedded in the business, we have the transformation team, which will be taking it forward. Within its business plan, that is the main focus of its work. As I said, one of the key lessons from what we have been through is not to tell you how long something will take and how much it will cost before we have done the user research.

We have had that point. I want to know whether we can get a copy of that to see what stage it is at, what has been done, what the sequence is and what the eventual aim is. Can we get a copy of your methodology for implementing the bill’s provisions?

Forgive me; I am not sure what I would give you, apart from to say that we are operating an agile methodology. I cannot give you a plan until we have gone through the early stages of agile.

Therefore, at this stage, you do not have any methodology, even in a draft form, for implementing the provisions of the bill?

Yes, we will use agile methodology to do it. We will start by doing the discovery phase, then we will start doing some build and then we will start doing some testing. That is the agile methodology that we will use.

You have a plan for a plan, rather than an actual plan.

Yes.

Could we see the plan for the plan?

All it would be is to say how agile delivery works. I am sorry—I am not sure what else I would give you.

Okay. Right. I am not filled with confidence.

The team will be able to share its discovery plan, which will say what will come out at the end of the discovery. At the end of that process, we will have the information that shows whether we will proceed to an alpha build. As Lorna Gibbs said, we will be able to say what we are doing in the first stages. At the end of discovery, we will move into alpha if we have enough information and that is the right thing to do. The discovery plan will be more detailed and will set out what chunks of work will happen. We will not be able to tell you anything beyond that, because we will not know what will happen in alpha until we know what comes out of discovery.

10:15  

We have not heard a cheep this morning about the role of your board in all this, including how you go forward with the bill. How many non-executive directors do you have?

We have four.

How did they monitor everything about the project that we have been discussing?

It is important to remember that, as we are an agency, the board is advisory rather than statutory and it does not make decisions. It is there to provide challenge and support for me. The board is made up of the five members of the executive team and four non-executives. They receive regular updates on what has been happening with transformation. At every meeting, they receive a paper on financial—

I am sorry to interrupt, but I am not particularly interested in what they receive. I want to know what value they have added to the process, if any.

They have added a considerable amount of value. For example, we have a non-executive member who is experienced in corporate governance and risk management. Another member of the non-executive team is an experienced agile delivery practitioner—

I am sorry to interrupt again—

They bring their expertise and challenge us.

You are giving me their CVs, but I want you to give me practical examples of where your non-executive directors have challenged you and said, “You’re making a fist of this. You need to change direction because you’re not doing it right.” Give us examples of where those non-executive members have added value. It seems to me that they are part of the failure.

None of our non-executive team nor any of the independent assurance that we have had has suggested that we should take a different approach.

In November 2018, when the leadership team had done some contingency planning and considered the options for getting off the BT contract, the result of that was taken to a full board meeting, involving the leadership team and the non-executives. We were challenged robustly by the non-executives about our confidence that we could deliver to different timescales, whether we had taken all the possible approaches on board and whether we were sure that BT was not a long-term option. In that instance, when we were getting additional assurances from elsewhere, the non-executives pushed us hard about whether we had the money to do it, what the timescales were and whether we were certain that we could protect safeguarding throughout. That is one example of when we had a significant amount of challenge from the non-executives about what we were doing.

We have had a lot of questions about the figures of £77 million, £34 million and now £78 million. Did the non-executives not ask what on earth was going on? There is a big gap between £34 million and £77 million. Did they not question that?

The non-executives received a presentation that talked them through the request for information responses that we had received and the areas where we planned to make reductions in the costing. They challenged us and had a conversation about that before the proposals went up to finance for a further conversation.

Did you change course as a result?

No.

Did the non-executives say that you should change course in any way?

I was not at that meeting, but my understanding is that they did not. Gerard Hart was at it.

My recollection is that they did not. The evidence base at the time of deciding between the two business cases was illuminated by the RFI, which was a strong piece of evidence. We should remember that two independent reviews had been done of the BT system, which had indicated that it would never be a suitable platform to move forward. There was an acceptance by the board, including the non-executives, that that was the reality. Therefore, it was about how we replaced BT, and the RFI responses, which were from credible companies, suggested that the costs could be lower. That was the appetite of the board at the time. There was no discussion from the non-executives about changing tack.

Can you give me any examples of non-executive directors changing the course of your decision making?

I can give you an example of when they have had a profound impact on decision making.

There was a contract with another supplier that did not complete. The board had to make a decision to end that contract and move on to another platform or progress in another way. The board’s discussions with and the scrutiny of the IT team and the chief executive—it was not Lorna Gibbs at that time—were extremely challenging and were in depth and detailed. I distinctly remember there being heated and animated debates about that decision and what the right thing to do was—

Did it change—

It did—it certainly refined and altered our awareness of all the risks and challenges. It is important to get across the point that the board works as a consensus group—it works, generally, to reach consensus. There have not been many times when there has been a stand-off between the executive and non-executive members. How decisions are discussed and made tends to evolve from that approach.

Is it aware of the issue to do with the minutes?

Yes.

You said that you had a director looking into the issue. Is someone working full-time on the minutes?

No. I apologise for the misunderstanding.

That would be wasted years.

No, no at all—there is not a director working full-time on the issue. However, I have asked one of our senior team to focus on the matter, because we are aware how much importance should be attached to it.

This is something that should be solved in a week—at the most. Mountains and molehills come to mind.

I am now speaking from the point of view of the digital director who owned the mygov.scot and the corporate websites.

We could solve the issue really quickly and put the minutes on the corporate website. However, we want to do so in a way that enables people to find what they are looking for. That is the work that Gerard Hart is talking about. If we need to look at that and make it an urgent priority, we can do that.

I think that you do. We are talking about uploading the information—I imagine that the minutes have already been written.

We are just trying to make the site usable for people—it has to be accessible to everybody.

I emphasise Johann Lamont’s earlier point: it is not a good sign for our IT projects if we cannot even get minutes online. Let us get it done.

I am troubled about the fact that something that was deemed to be unaffordable in the first business plan in 2015 is now seen as a great success.

I have a question about the Disclosure (Scotland) Bill. I hear what you say about bits of it being done—that is not an issue. The other bits sounded quite complicated. What consultation did the Scottish Government, and the bill team specifically, have with Disclosure Scotland on the deliverability of some of the proposals?

The bill team is embedded in Disclosure Scotland.

Sorry?

We own our own policy, so the bill team comprises Disclosure Scotland staff.

You are saying that you have sanctioned a bill, which will be discussed by and passed very soon by Parliament, but that there are bits of it that you are not confident you will be able to deliver.

No, that is not what we said. We are confident that we will be able to deliver the bill. We need to say exactly what that delivery will look like. We are confident that we understand and can use—

You should have started with that before you drafted the bill.

We could not do the detailed work on user engagement until we had authority from Parliament that that is what it wanted us to do.

I am confused now. You have complete ownership of the bill, including all the proposals in it, but some of the proposals will take a long time to work. That is my understanding of what you said.

Yes.

You said that some of the proposals are quite complicated, that you were not sure when they could be done and you could not put a timescale on it. It now transpires that you have created the proposals. It is the Scottish Government that has sanctioned the bill—it has to be on the authority of the Scottish Government rather than of the Parliament. Ultimately, it is a Government bill that is being presented to Parliament.

You agree with everything that is in the bill. We—the Parliament—will, I think, pass it. The bill is being scrutinised and it has a fair wind behind it. You have highlighted that there are proposals in the bill that you consider will be challenging, but it transpires that those are your proposals.

The proposals in the bill are on the basis of extensive consultation with users. We know that the principles in it are what our stakeholders and users want us to do. That is where the policies in the bill come from.

As you would expect from a digital transformation, the next stage is to take the high-level principles and ask users whether, when they get access to the data, they want it to be in a PDF file or they want it to be available via an online portal. We need to go away and do that piece of work with users following the way in which the agile methodology should work. You take the high-level principle, and then you speak to the users and engage them with the specifics of how that is going to be implemented.

With respect, I do not think that we should put things into legislation if we are not confident that we can deliver them.

We are confident that we can deliver them. We have experience of delivering.

You are confident that you can deliver them, but do you know when you will deliver them completely and when you can deliver them by?

We do not know when we will deliver them by because we need to develop the detail of that with our users.

Could it be five years or 10 years?

I would not expect it to be that long.

Months or years—what are we talking about?

I would say a couple of years, but that is an estimate—please do not hold the future chief executive of Disclosure Scotland to account on that, because we still need to go through the details.

When are we going to get a new chief executive?

The recruitment process is under way.

Surely we can expect it to take more than a couple of years. I sat through some of the evidence on the disclosure legislation. I assume that you could nail down, to within a couple of years, what is reasonable. It does not sound very certain to me.

I am concerned that we end up passing legislation that is a million miles away from what is happening on the ground. The purpose of bodies such as Disclosure Scotland is to bring these two things together and to make the legislation realistic to discourage politicians from just passing laws and hoping that they might work at some point in the future.

Before you respond to that, Lorna Gibbs, Liam Kerr has a specific supplementary question, and then I will go to Bill Bowman.

Johann Lamont’s point is absolutely spot on. The logical progression from that is that, at stage 3, we will be asked to pass a bill that will have a set of associated costs that are, I presume, based on a financial memorandum. Parliament will be asked to look at those costs and decide what looks like a reasonable cost and whether the bill is clear to go. From what I am hearing, however, it sounds as though the financial memorandum, by definition, will be fairly inaccurate because, for the reasons you have given, you do not know what the system will need to look like, how it will need to scale up, how many users there will be, and whatever it might be. How accurate can that financial memorandum therefore be?

I will pass that to Gerard Hart because the bill team reports to his area.

The agile principles that will underpin the development of the bill work to the idea that you build out to what your budget permits you to build out to. There is no blank cheque.

We need to make sure that, in our user research and engagement on what we build, we build to the available budget and we deliver the functionality that is needed within that envelope. The financial memorandum is based on a reasonable estimate of the cost of implementing the legislation, allowing for the fact that agile methodology will be used and that it could deliver different flavours of functionality within a predictable envelope of finances. We cannot simply overspend just because we want to; there has to be some governance and control over the spend. The figure that is in the financial memorandum has come from a detailed scrutiny of all the variables that—

What is that figure?

I do not have it to hand. I will provide it to the committee.

Forgive my naivety on this, but what happens if we pass the bill, the financial memorandum says how much money you have to develop the programme, and Lorna Gibbs then goes away and says, “Actually, to deliver the functionality that Parliament has just told us to deliver will cost twice that.” I presume then that either you do not deliver, and we find ourselves back in this situation, or you do but you come back to Parliament for more money.

This remark might be helpful to the committee. The hard part about what we have done was building out the complex infrastructure that is needed to store police data in the cloud and make the complex connections to the police national computer, and the English and Welsh systems that we rely on. That was the difficult stuff and it meant that we struggled with new challenges that have not been met previously. Nobody had put police data into the cloud; nobody had made those kinds of connections into a cloud-based platform. Those were novel challenges.

The challenges that remain are around well-understood principles of how to build and operate web-based services for customers to engage with a product or a range of products. The big stuff around the bill that will require further development work on PASS will be building out largely in the digital domain. My view, as the accountable officer, is that that is a much more palatable challenge than the one that we have just eaten, in terms of building out the infrastructure piece around PASS. It is important that the committee understands that dichotomy and that what we have done was perhaps much more difficult than what is to come. I am not trivialising what is yet to come, but the order of magnitude is not the same.

There are five of you here and you have a bill going through the Parliament. Surely one of you knows how much it is going to cost.

There is a financial memorandum to the bill. I do not have it with me but, obviously, I can make that information available to the committee.

I presume that you know, Mr Johnston.

I do not have it in front of me. Clearly, we are here to—

To be fair, if you have a bill going through Parliament, I would expect at least one of you to know how much it is going to cost.

We will take that away and get back to the committee with those figures right away, without any delay.

10:30  

I am perhaps a little confused. On our agenda, Lorna Gibbs and Joy Bramfitt-Wanless are listed as former postholders at Disclosure Scotland, but you are talking as if you are still deeply involved in running it and developing a system. Can you explain what is going on?

I was deeply involved until a month ago. I am still very close, although I am not taking the system forward.

Are you part of Disclosure Scotland?

Not any more. I am back in a core Scottish Government role.

I was there until the system was delivered. I was loaned to Disclosure Scotland from the digital directorate and then, when the system was delivered, I went back to the digital directorate.

It is perhaps important to say that the people who work in Disclosure Scotland are Scottish Government civil servants, which is the reason why we have a bill team that is integrated in the organisation. It is perfectly appropriate for an agency to have that arrangement.

Although Lorna and Joy have moved on, they are still part of the Scottish Government—as am I, albeit that I work in the context of Disclosure Scotland at the moment.

We are trying to find out about Disclosure Scotland and what has gone on. It is interesting to hear about the development, but I was confused that Lorna Gibbs was speaking as though she was still running the organisation.

I am happy to come in on that. The period of the audit report relates to the period when Lorna Gibbs was chief executive. We took the view that, in order to assist the committee, the best panel would include Lorna and Joy, given that they were leading this work.

We accept that.

Lorna Gibbs made a comment about the role of non-executives on the board, which Alex Neil was asking about. I think that you mentioned that one of them had experience in agile. Was that person on the board before you chose to adopt agile?

A different board member with agile delivery experience was on the board when we decided to move to agile. Her term with us came to an end towards the end of last year, so we recruited a different non-executive, who also had very strong experience in agile delivery. At the time when the move was made to adopt an agile approach, there was a non-executive member of Disclosure Scotland’s board who had a background in agile delivery .

So they were also on the board at the time when you did not think that agile was appropriate.

Was Stephanie Kerr on the board at that point?

It was around that time.

I think that we brought her in around about that time.

We are where we are, but why, at that time, was the advice not given to embrace and adopt agile earlier, if that person had the expertise and the vision?

I would need to double-check the date on which she was brought on to the board. I am looking at Gerard Hart here. It may be that she was brought on to the board because we were making that move into agile and therefore we needed that level of experience. I am sorry, but I do not have her start date in my head.

What is the full-time equivalent head count for Disclosure Scotland?

It about 400 at the moment.

It is just north of that. There are 300-odd civil servants working there, and the rest are agency staff.

Will you require additional staff to implement the new build?

It is a complex picture. When we deliver the application process of PVG online—I hope to deliver that in the near future—that will allow us to reduce some of our agency staff complement, because we will no longer be going through certain manual processes that we currently have to go through to process PVG applications.

The bill also brings new functions to Disclosure Scotland. For example, there is a function of allowing people to appeal the inclusion of convictions on their disclosures, which will be administered by us in the first instance. There is a functionality about police information finding its way on to disclosures, for which we will have to be the arbiter. There is a whole range of functions around referrals and barring that are changing, for example, the ability to put conditions on people who are being considered for barring to control what they can do with vulnerable groups in that consideration period.

All that will change the Disclosure Scotland workforce quite profoundly.

What assumptions have you made about what additional staff you will need to implement the bill’s provisions?

Those figures are in the financial memorandum.

The figures say that no net increase in staff is needed, but the configuration of staff will change quite markedly.

So it should stay around the same.

That is my recollection. I will double-check that and inform the committee of the accurate picture in the FM.

Thank you.

We have had a quite robust and good exchange on that.

Paul Johnston, given everything that you have heard this morning, do you stand by the fact that you regard this as a successful project?

I will stand by the facts that this project is delivering a service that the public is finding to be better than its predecessor, that we now have the underpinnings for further reform and transformation, and that the agile methodology was the correct methodology to take us forward.

I will also reiterate that there are lessons that are already informing the approach that we are taking to other IT development programmes across public services. For that reason, the scrutiny of the auditors and of this committee is important and valuable.

Thank you. I hope that, in the future, we will not be discussing IT projects and reflecting on some of those lessons, but I suspect that we may well be in the very near future.

I thank all the witnesses for their evidence this morning.

10:36 Meeting continued in private until 11:23.