Public Audit and Post-legislative Scrutiny Committee

Meeting date: Thursday, October 26, 2017

Official Report 512KB pdf

---

“Principles for a digital future: Lessons learned from public sector ICT projects”

Under item 2, we will take evidence on the Audit Scotland report “Principles for a digital future: Lessons learned from public sector ICT projects”.

I welcome the following officials from the Scottish Government: Colin Cook, director of digital; Anne Moises, chief information officer; Lisa Baron-Broadhurst, programme director for social security; and Andy McClintock, chief digital officer for social security.

Before I invite an opening statement from the Scottish Government, I will put the matter in context. This is the third such report that we have received from Audit Scotland in five years. The committee has also considered critical reports about common agricultural policy futures and i6, both of which were major and expensive public sector information technology projects that, frankly, did not perform satisfactorily at all.

We do not know the exact cost to the public purse of information and communication technology projects that have not fully delivered, but we know that it is likely to have been very substantial. It also appears that the public sector faces greater challenges in delivering ICT programmes than the private sector does.

There have without doubt been successes, but we want to be reassured that the Scottish Government and other public bodies have tried to understand fully why previous failures occurred. We also want to be assured that the Scottish Government’s new suite of initiatives will make a difference this time.

On that basis, I invite Colin Cook to give his opening comments.

Thank you. I welcome the opportunity to have this discussion and to talk about the lessons learned from previous IT programmes.

I was appointed to the role of digital director on a permanent basis back in June, having had the role on a temporary basis for a few months. The report has provided a useful and constructive input into our thinking on how we shape up the directorate that I am now leading—the ways of working, how we measure success, our structures and the approach that we need in developing our staff.

The report has also helped to inform the Scottish Government’s new digital strategy, which we published back in March 2017. I believe that the strategy sets an important context for today’s discussion and for the efforts that this country is making to ensure that it is a successful digital country in the modern world.

In our letter to the committee of 25 August, we set out some of the changes that we are making. We are changing the structure of the directorate so that we can focus the resources that we need on assurance, transformation, service design and the opportunities of new and emerging technologies. We are introducing a new tiered approach to assurance, based on challenging standards. We are improving skills through our new digital skills academy and the digital champions programme, which works with leaders in the public sector. We have new approaches to procurement, including the further development of CivTech. The office of the chief designer has been created to drive design thinking into the heart of Government. I also highlight the way in which we work on digital transformation and lead projects, with multidisciplinary teams that can reflect on and learn the lessons that come through the Audit Scotland report.

I believe that a digital country requires a digital Government, and I am determined to build a team that is truly excellent and which gives that leadership. I welcome the opportunity to discuss that with you today.

I am joined by Anne Moises, who is the Scottish Government’s chief information officer, as many of you know. She is leading our work on assurance and the new assurance framework, which came out of the findings of the Audit Scotland report. I am also joined by two senior colleagues from social security: Lisa Baron-Broadhurst, who has overall leadership of the programme; and Andy McClintock, a former colleague of ours in the digital directorate, who is acting as chief digital officer of the social security programme. If there are technical questions on how social security is developing, he will be in a great position to answer them.

Thank you. The first question is from Bill Bowman.

Good morning. I would like to focus on the Scottish Government’s submission and on annexes A, B and D, which you are familiar with. Annex D, which is on social security, is short and to the point, and I understood that. Annexes A and B are full of impressive language but they are a bit short on names, dates and numbers. Are you responsible for there being no further IT failures and can you explain briefly how you will ensure that that is the case?

For my part—

If you are responsible, that is.

Thank you for the clarification.

I am responsible for making sure that we implement a robust process of assurance and for ensuring that we work with projects wherever they occur across the Scottish Government to pick up problems early if they are going to occur. I am responsible for ensuring that we have clear standards and guidance and that training courses are available for staff across the Scottish Government to help them to lead programmes in a modern and effective way. In that sense, I am here to improve the way in which the Scottish Government and the public sector beyond that—we work in partnership with local government, health and others—deliver IT programmes.

Where have you reached in that process?

I am sure that you do not expect me to say that everything is sorted, as this is a long-term project. We are making good progress and we now have a good approach to assurance in place through the work of Anne Moises and her team. We have built up that team and increased its resources, so we are in a good place on that. We have the groundwork of good training and support programmes in place, and we have a number of examples of good practice where we have co-located multidisciplinary teams that bring civil servants—policy people—together with delivery people and blend internal and external experience in the right way. We are getting there, and we are certainly improving the way in which we manage IT programmes across the Government.

Could you give some examples of the number of people involved and of teams that you have co-located? From the submission, I did not get a feel for where you are and how it is working in practice. How many people do you have, and how many locations are you in?

The digital directorate has about 450 people, but they cover a range of things; it is not all about the management of IT programmes. For example, we are responsible for the roll-out of superfast broadband, and about 250 of those people are working in the Scottish Government’s IT function, so they are working to maintain the systems and give civil servants, ministers and others the IT service that they need.

Could you come down to examples?

I can give you some examples of where we have co-located teams for projects. In our office in Victoria quay, we have teams working on transformation activities. We have a team working on the development of a common approach to licensing. Sorry—those people are actually co-located with the Scottish Environment Protection Agency in Glasgow. We are developing a new approach to licensing in SEPA on the rather glamorous subject of licensing of septic tanks. That is allowing us to test a common process for licensing for the future. We have co-located teams with social security, which we may go on to explore. My team is a partner in the social security programme. We bring technical and delivery expertise to that programme, and we are co-located there.

We have a number of examples of co-located teams, and I am happy to take you through them.

How many people are focused on, shall we say, preventing future problems?

The office of the chief information officer, which Anne Moises can talk about, has around seven staff focused on the assurance process at the moment, and we are building that. We have ramped it up consistently over the year. However, that is not the only way in which we look to ensure that there are no IT failures. That is an assurance process, which involves going in and assessing. We of course draw on external expertise for that. We have a network of people across Government who carry out digital first assessments and who do not all work in my team—in fact, the majority of them do not. We also draw on external experts to do that.

The digital transformation team as such—the digital business models team—has about 100 people in it, and they are working on various programmes including our common approach to the provision of information, the metrics and some of the projects that I talked about earlier. They are working with SEPA, and they are working on licensing and on social security. We have about 100 people on what I might call the delivery side, seven and growing on the assurance side and more people working on projects around data, for example—there are about 40 people in that team. There is a significant investment in staff and resource to get this right.

Have you carried out some assurance functions and found something?

I might pass that question on to Anne Moises, but the short answer is yes. We have been embedding a two-tier assurance process for about a year now. We have a set of digital first standards that set out how projects should be delivered and our expectations about how users are engaged, how teams are constructed and how projects are run. We have conducted a number of those exercises, and then we have a specific assurance process for what we regard as projects with high value or high reputational risk. Anne Moises’s team is responsible for leading those, and I am sure that she can tell you about some of the specific examples.

I can indeed. We have carried out a number of stop-go assessments of major projects. Those are the ones that either have a value over £5 million or have a significant risk or reputational value associated with them. We have also carried out a fair number of digital first assessments.

I can give you the exact numbers. From the launch, which was in August last year, we have carried out 12 digital first assessments. There were three in the pilot stage, which was important, because we wanted to ensure that the process not only added value for us from an assurance perspective but added value to the projects. It was important that they understood what we were looking for and that we had a good view of where they were going to need support should the assessments come up with recommendations.

In the area of digital first, we work closely with the digital transformation service that Colin Cook mentioned. Should we identify areas where more support is needed—for example, user research or an understanding of web or quality—we can offer support to go into the teams and work with them to bring them up to the required standard. Since then, we have done 12 digital first assessments.

We have also carried out four major stop-go assessments on large projects. Social security is one, and the others are in Revenue Scotland, the National Records of Scotland and Transport Scotland. We have quite a number planned between now and the end of the financial year.

James Kelly has a supplementary question.

On the point about assurance, is there a dedicated computer audit team?

Not within my team in the office of the chief information officer. There is an internal audit team within the Scottish Government.

Are computer audits carried out of the systems that the Scottish Government is responsible for?

What I can tell you about is the systems that I have personally been responsible for in running the IT for the core Scottish Government, and yes, we are subject to regular audits.

Yes, but do dedicated computer audits take place?

I am not 100 per cent—

My point is that there is a discussion about assurance and, in order to provide assurance about the quality of the IT systems that are under development and those that have been implemented, I would have expected, having looked at this thing in businesses, that they would normally be subject to regular computer audit to ensure that the processes and procedures that you have in place are top quality and are delivering to the standard that we expect. There does not seem to be a computer audit function, so how is that fulfilled?

09:15  

Supplemented by external experts who are brought in for particular projects, my team carries out what we call technical assurance, which is in effect an audit at a point in time in a project. To be clear, the projects are in flight or in the process of developing or delivering. For example, we piloted the stop-go process on the e-counting project that did the counting of votes for the local authority elections. We created a mixed team for that. We did not just look at the computer elements; we brought in recording officers from local authorities and people who understood how the algorithms behind the counting worked.

We determine what the project is and then create a team that can explore the technical bit of it and how that technical bit works in context. I am not 100 per cent sure that is exactly the same as a traditional computer audit.

It sounds to me as if that will drill into a particular part of the process rather than taking a complete overview. From a computer audit point of view, we would expect a level of independence, and I am not convinced that the example that you have described had that.

The independence is definitely there. My team is now entirely separate from any delivery mechanisms across central Government. As I said, my team is often supplemented by experts who work at the United Kingdom level on specific topics. We try very hard to make sure—well, we do not try; we succeed in making sure that the team is independent and that it has the requisite expertise to look at particular project that we are involved with.

Is there segregation between your team and the teams to whom you provide assurance? How is that independence assured?

Anne Moises has referred to the fact that, until about six months ago, that independence was not as clear as we wanted it to be. As part of the changes that we are introducing, we split off responsibility for assurance into a separate function under the OCIO, which now has responsibility for assurance and the development of staff and capabilities in the professions across the piece. The operational running of the Scottish Government’s computer systems is now the responsibility of the chief operations officer, which is a new post and is a better way and more in line with the principles of Audit Scotland.

I want to ask a wee bit more about this stop-go process. That is in effect an assurance process, but I would like to understand a wee bit more about what it is exactly. What causes something to stop? What happens between stop and go, and what constitutes permission to go again? It sounds as if it is a quality check but could you explain a bit more about that? Is it event driven? Is it failure driven? What causes something to enter that process?

We intervene in projects at key points in their development cycle, and pick up a lot of the key indicators that are in the Audit Scotland report, some of which were previously in our checklist, which has been supplemented to make sure that we have them all. Our intervention is now on a mandatory basis; it is no longer a matter of choice whether projects engage with us. We come along and say that we are going to do a review at a key point. For example, a key point in a project might be before it goes out to tender or puts out a requirement to contractors. At that point, we will have convened an assurance team that understands the subject matter. It will look at pretty much all the indicators that are in the Audit Scotland report of what good looks like, and at all the checklist material that we have from previous experience of the Scottish Government, the UK and other countries about the indicators of potential failure.

The project will go through either a three-day or four-day process with a number of people on the review team. The project provides us with all its background material and we have in-depth interviews with the people who are delivering the project. At the end of that, we produce a report that looks at all the key indicators and the expert assurance team gives a view on whether there are areas of risk, and if so, whether that risk is such that the project should halt until the risk has been mitigated or issues rectified.

We have not put a complete stop on any project while this process has been running, but we have come up with a number of recommendations that the projects have implemented quite quickly before they have got the momentum back up again.

If, for example, we came up with a really big issue around a document that was about to go out to tender, our expectation is that the document would not be issued until such time as the problems that we identified had been rectified.

I will just add to that because we also talked about our digital first standards and the fact that all significant projects are subject to such reviews. That would involve the review of the progress of the project at discovery level alpha and beta, in our terms, in the life cycle.

The first review of whether a project is following standards is in the exploratory phase—the phase when you are finding out about user needs and you are starting to design the process. One of the criticisms that Audit Scotland has made of us in the past is that we did not engage early enough in checking that a project was being effectively scoped and that user needs were being effectively understood.

One of the important things about the standards process is that it gets in early and it asks questions about the design of a project, not just how it is being delivered.

These are basically system reviews—it is about review points—that are an audit process in itself.

Yes.

Who would carry that out? Is it people who are slightly outwith the project scope or—

It is always people who are outwith the project scope.

Who are they? Are they software engineers? Are they auditors?

It depends on the project. We always make sure that there is significant technical expertise within that team but, as I said about digital first standards, for example, what is critical is that there are people who are expert in understanding customers or the way in which user research or service design is carried out. They are multidisciplinary teams and the balance of that expertise will be skewed depending on the type of project and where it is in the life cycle. Where we need to—and we have done this on occasion—we will bring in external expertise if we think that there is a particular issue. That may be around cyber or some very detailed technical issue.

I am getting increasingly confused as to layers of accountability here so let me just be practical. Let us say that there is a problem ultimately—which none of us would wish—with the social security system that is being designed. Is it Lisa Baron-Broadhurst I would go to? Is she the one who is accountable, or is it you, Colin?

Lisa is accountable for the delivery of the social security programme—

But who—

—in which IT has a critical part to play. I am a member of the programme board of social security, so I share responsibility for the delivery of social security. My specific role is to make sure that, as social security IT systems and digital systems are developed, they are developed as far as possible in line with the Scottish Government’s digital strategy. I also have people who work for me who are part of that team. Ultimately, the accountable officer for social security is in the social security line.

So it is not you—the buck does not stop with you; it stops with Lisa. Is that what you are telling me?

If it is a specific issue around the delivery of social security, the buck stops with Lisa. If it is about how the programme is developed, we share accountability through the programme board structure for the delivery of that programme.

So there is not one person who is responsible for the design and delivery of the project in its entirety.

I am responsible. That is my role. As Colin Cook said, his role is to ensure that we have the materials, the tools, the techniques and the people to support us in that delivery. My ambition would be that we do not hit a problem—that we work together and work with Audit Scotland and others and learn lessons so that we do not get to a point where we have a problem.

Just to support what these guys were saying, we have just been through a pre-procurement gate, so we have just awarded a contract. It will go live on October 30—I think that we notified it yesterday. We went through quite a robust review—and yes, Mr Coffey, it is a review, but it is a really robust review. On the panel we had technical IT people who knew the digital first standards. It was quite a rigorous and challenging review that looked at what we were doing and how we were doing it. As well as looking at what IT or technical measures we had put in place, it considered what governance we had around that, what people we had in place and what capability we had, so it was a robust assessment. I do not know whether that helps.

I am very simple—I want one person to be accountable for all IT projects in the Scottish Government, and I am not hearing that that is the case. I am hearing that there are individual departmental responsibilities.

The nearest thing to a single accountable person would be Anne Moises. If a stop is missed in the stop-go system, that comes to Anne Moises, does it not?

Yes. Basically, if the assurance process recommends a stop—

Or misses a stop.

Yes.

It is absolutely the case that the quality of the IT assurance process is the responsibility of the digital directorate and, within that, the office of the chief information officer. The quality of the delivery of a particular IT programme of the size of the social security programme is a matter for the relevant team—which, in the case of the social security programme, is the social security team—but we do everything that we can to challenge that team and to make sure that it follows the best practice and that all the assurance gates are followed and acted on.

We have the ability to stop a project—that is why they are called stop-go assessments—if we do not believe that the best practice is being followed.

That is interesting.

Good morning. I want to go back to the fact that, as the convener told us at the outset, there have been three reports by Audit Scotland on the matter. Why have previous Audit Scotland reports and the actions that the Scottish Government has taken in response not prevented further ICT issues? Do you have oversight of that?

My job is to make sure that we learn from those processes and that we are thoroughly acquainted with all the messages that Audit Scotland sends. We work closely with Audit Scotland to learn from what it says. I have described some of the things that I believe best reflect its principles. I know that the committee will hear from Audit Scotland later, but I think that we have had a very positive working engagement with that organisation to develop effective approaches.

As I said, I came into the job on a permanent basis in June, so I cannot talk with confidence about actions that were taken two or three years ago, but I can talk with confidence about what we are doing now and how we are learning from what happened previously.

Ought not you to be able—I put this to you on the basis of the answer that you gave; I am not challenging you—to say why things have gone wrong, notwithstanding the previous reports? One would have thought that one of the first things that you would have done would have been to look at the situation and ask why, when previous recommendations were made, they were not taken on board and/or not actioned sufficiently well to prevent the same thing from happening in the future.

I very much take that as input to my thinking on how I need to organise this function. As I said, we have made sure that there is a clear separation of responsibilities between delivery and assessment, which is important. We are increasing the training that is available and we are making sure that we can bring in expert staff from outside the organisation when we need to. We are making resources available to people who are involved in projects and programmes right across the Scottish Government so that they can act as intelligent clients and can find the right staff to work in their programmes.

My team has been involved in the assessment process for a lot of the recruitment that has been done in social security on the digital side, because we have—I hope—the expertise to identify the kind of talent that we need to deliver. For example, we were involved in the appointment of Andy McClintock. In the past, there was a lot of criticism that we had the wrong people in the wrong places, and we are trying to deal with that.

All of Audit Scotland’s reports—particularly the report that we are talking about today, which is great in looking at how IT projects are managed internationally—are heavily influencing our thinking and our approach.

I might come back to the staffing and talent side of things later.

As the convener said, the committee is looking for reassurance that this time things will be different. You might have answered this, but what specific actions are proposed in the new Scottish Government response that have not been previously tried or considered, such that this time will be different?

09:30  

I have explained that we have a new structure and have increased our resources. We are taking a much firmer role on assurance, as Anne Moises said; there was a time when engagement with the chief information officer on assurance was voluntary, but now it is mandatory.

Scotland’s digital strategy talks about digital first standards that allow us to stop projects and the assurance process that gives stop-go powers to this team. We have considerably strengthened the bite of the audit process, which is a really important difference from the past. When I took up this job, I said that a major objective was to have an assurance process with teeth, and that is what we are building. We are making the resources available to do it. That is not always easy given the constraints, as you can imagine, but we have made that call and commitment because we think that, as a digital directorate, we are in a unique position to do so.

Scottish Government officials have told members of our predecessor committees that they have highlighted an Audit Scotland checklist on ICT to the chief executives of the relevant public bodies. From your review of what has gone before, do you know whether that action has resulted in any significant and measurable improvements?

Can I produce a list saying which projects would have failed otherwise? No. Do I think that the approach has resulted in measurable improvements? The answer is definitely yes.

I will go back to actions that were taken after the 2012 Audit Scotland report. The checklist was initially designed to help senior responsible owners ask sensible questions of the projects that they were guiding or directing. As our processes evolved, we changed that and now ask for copies of the responses to the checklists, so that the office of the CIO can run a quality check over them and pick up any danger signals. The checklist questions have been highlighted in the latest Audit Scotland report as key indicators of potential problems later on. They include questions such as, “Do you have the right skills in the team that is delivering your project? If you do not, do you know where to go?” We can pick up the responses and direct people to the right source of advice and guidance or to bodies on the ground. They also ask about the funding and governance arrangements and signpost good practice.

Quite a number of the failures that were high profile—the i6s, the NHS24s and the CAP futures programmes—were all in train before those assurance processes came into place. Do I think that we could have stopped some of the problems if we had had those processes then? The answer is potentially yes. I will not be able to prove that the process works, because if it does we will not have the problems. I am proving a negative.

You have developed the process and the assurance, but how will you ensure that public bodies also learn those lessons from the previous programmes?

We make guidance widely available, but the most effective way to do that is to go out and talk to public bodies, which we are doing. As the OCIO has staffed up, we have sent colleagues out, starting with the major projects. We are not spending a lot of time on projects under £100,000; we are concentrating on the major projects. We are engaging with the chief executives and heads of corporate services to explain not just that the guidance exists but why it should matter deeply to them. We are creating a rapport, so that they feel comfortable picking up the phone and asking for advice, help or our assistance in signposting resources.

What specific steps has the social security programme taken to learn lessons from previous ICT projects?

I have been in post for 15 months, and I am really big on lessons learned. One of the things that I have done is speak to our Audit Scotland colleagues about what they have done and the previous reports that they have produced. My team calls meetings every three months and invites people in from other major projects in the public and private sectors and across Government, not just locally. I think that we have had 25 different projects in so far.

My view is that you cannot just learn lessons and put that learning on a shelf. Our lessons-learned document is a living document—we catalogue the lessons learned, give them an action owner, say whether they are a good lesson or a bad lesson and set out what we are going to do about them. Certainly, the Audit Scotland report was good for us because we were able to put in an action plan on the back of that to ensure that we do not fall into any of those traps and that we do not do a big bang sort of thing.

We have done a tremendous amount of work on lessons learned, and I stress that that involves not only a look at the lessons learned, but work on what we are going to do about those lessons and how we are going to implement them in relation to social security.

I will just say at the outset that this document from the Scottish Government is probably one of the more obscure ones that have come forward in this session of Parliament. I shudder to think what a constituent of mine would say about it if I happened to hand it to him in the street. However, I will pluck a few bits from it and try to expand our knowledge. Of course, it would have been helpful if the pages had been numbered.

Annex A talks about

“an IT Assurance Framework which supports Senior Responsible Owners”.

Can you give me an understanding of the role of the IT assurance framework in relation to the support role of the senior responsible owners? How do they work together? What do they do? How do they come together? Basically, what can you tell me?

The IT assurance framework is the approach that we have been describing, in terms of the assessments that are made, the various stages of the operation, the things that we look at, the fact that we share those results and discuss them with the senior responsible owners and the fact that we have the ability to stop programmes.

One of the key recommendations from Audit Scotland was that all projects should be seen within an overarching assurance framework, which is effectively what we have established under the OCIO. We have worked in what I hope is an intelligent way over the first few months of its operation. When Anne Moises, the chief information officer, gives the findings from an assurance, it is not just a tick or no tick; it is a set of guidance points, lessons that can be learned and things that we expect to improve. Of course, as I said earlier, in my team we have resources that senior responsible officers across the Scottish Government and beyond can buy into, which means that we have somewhere for people to go to get access to the expertise that they need. It is not just a question of criticising something and leaving it there; there is a way in which people can access skills to help them address some of the problems.

The reason why the assurance framework is intended to support senior responsible owners and accountable officers is that it is about not just technical information technology but the entire programme or project. One of the lessons that we have learned from our previous experience is that we have to ensure that the entire range of the team understands what is in the report. It is critically important that the person who is leading the programme or project understands what is in the report, what the consequences of issues are and what he or she should be expecting the team to do by way of remedial action. It is not a report that just goes to the IT people; it is intended for the programme, and it is specifically intended to ensure that the senior responsible owner—the person who is ultimately responsible for the programme—knows what issues there are and what is expected.

In respect of the social security system, is the senior responsible owner Lisa Baron-Broadhurst?

Stephen Kerr is the senior responsible officer for social security.

Stephen Kerr has overall responsibility for the delivery of the social security programme per se, and Lisa Baron-Broadhurst is his programme director. He is in charge of the directorate within which social security will sit, and that includes the programme to develop the new agency and, after that, the running of that agency.

Does that make him the accountable officer as well?

Ultimately, he will be the accountable officer for social security. However, you have to remember that social security is not an IT project; social security is about putting an agency in place for the people of Scotland, with the right processes and procedures, which we will IT enable. The IT is part of that. It is obviously a big, important part, but—you will probably hear me say this a few times—it is not an IT project; it is a social security agency for Scotland.

For social security, who is the accountable officer?

Stephen Kerr is the senior responsible officer for social security.

He is the responsible officer.

Yes. His head will be first on the chopping block, followed by mine.

He is the responsible officer, but he is also the accountable officer.

Yes.

So the two positions could actually be the same.

They could be the same. In some organisations, they are not. In an executive agency, for example, the senior responsible owner might be the head of corporate services for a project, whereas the accountable officer would be the chief executive.

Is the structure going to work?

The structure is working. The reason why those two particular terms are included is that, to return to Lisa Baron-Broadhurst’s point, we are keen that the measures do not just involve looking at IT as an island. There is no such thing as an IT project. The point of doing something that is IT enabled is to create a business change or a transformation. That matters to the person running the entire programme and to the accountable officer, if it is an agency. IT contributes and can cause massive problems, but it is only part of the bigger picture for an agency chief executive, for example.

The structures for social security, for governance and for accountability will absolutely work. Part of the Audit Scotland report is about ensuring that that accountability is there and that there is one route up to the top, which in our case is the programme board. With reference to my activities in social security, all the boards feed through to the one programme board, and that programme board includes Colin Cook as the representative from the central IT/digital function. It will also include lead commercial colleagues and lead finance colleagues. For social security, it all leads to the programme board.

There have been big lessons learned from the previous programmes that you will know about, such as when groups of people in different areas are involved. I think that that might be what you are getting at, Mr Beattie, in relation to when accountability sits in different areas. For social security, accountability lies in one pillar or one tier. It all feeds up to one place.

We will get independent assurance through the digital centre but, ultimately, the accountability leads up to one programme board. For social security, I am very confident that we have the right governance structures, with the right levels of accountability in order to deliver.

I am glad that it is so clear.

I come back to stop-go gates. I am looking at paragraph 5(ii) of annex B in the paper that we have before us. Are the stop-go gates not a bit of a cop-out? Are they not just a way of pushing decisions up into the lap of the lead minister and, ultimately, the Cabinet Secretary for Finance and the Constitution?

According to that paragraph,

“Where a ‘Stop-Go’ assessment is made an Accountable Officer could only proceed following a transparent process, requiring the lead Minister to agree this arrangement with the Cabinet Secretary for Finance and the Constitution.”

They are clearly not IT experts. Whom will they be advised by? They will be advised by the people who are running the project. Those people will presumably go to the minister and say, “Yes, it’s okay to continue. That’s the expert advice,” and the minister will say, “Oh, all right, then.” What sort of judgment are you expecting ministers to make for an IT project? Surely it is the responsibility of the civil service to deliver that.

The civil service will make a recommendation to the minister. This is one of the ways in which—

But it will be the same people doing the assessments.

09:45  

The minister will be given the assessment that has been made and the recommendations that have come from Anne Moises’s team around that assessment. The judgment on whether to proceed will be taken in a broader context, and it is right and proper that it should be taken at the highest possible level in cases of huge public interest.

What is important—this is a reflection of some of the letters from Audit Scotland—is that going through that process and ensuring that the lessons from an audit are seen at the highest level means there can be no suggestion that decisions are not being made in an open and transparent way. We learned from some of the Audit Scotland reports that decisions needed to be escalated and that the right level of focus needed to be brought to them at the right time, and this is very much part of that process.

As I read it, every stop-go will end up with the accountable officer going to the lead minister, who will then have to go to the cabinet secretary.

No, only if there is a stop.

Only if there is a stop.

Only if there is a stop that the programme team disagrees. The expectation is that, if the assurance team comes up with a stop, it will be evidenced and, under most circumstances, the programme team will recognise why there is a major issue and what they should do before they restart the programme.

We recognise that, for a variety of reasons, including payment deadlines and all sorts of things, there may be circumstances in which the programme team does not agree with the assessment by the independent review team and—despite the fact that we have recommended that it stop—decides that it needs to continue. That is the only circumstance in which the matter would be escalated.

That is not what the Government’s submission says. It says:

“Where a ‘Stop-Go’ assessment is made an Accountable Officer could only proceed following a transparent process, requiring the lead Minister to agree this arrangement with the Cabinet Secretary for Finance and the Constitution.”

In that case, I apologise for the language. It is when a stop-go assessment has decided that there should be a stop. We will clarify that. I have already taken the lesson from today’s discussions that we need to look at the internal marketing of the assurance process to make sure that everybody is clear about what it constitutes, how it operates and whether there is any obfuscation around that. I apologise if we have created a difficult way of understanding it in our submission. We will make sure that, as we communicate it across the Scottish Government, that is done in a very straightforward way so that everybody—be they IT experts or non-IT experts—can understand the implications for themselves.

If it is easier, I will put it into basic language. If, as the programme director for social security, I got a red stop it would be really unusual for me to insist that the process carried on without taking the actions that came out of the recommendations. I would look at why I had got a stop and what actions I needed to take to mitigate that. It would be really unusual to go forward at risk—and it would be at risk. There would be a stop gate, so why would I try to convince a minister that it was the right thing to do? I would look at what actions I needed to take to get me back on track, undertake those actions and make sure that I had an action plan in place before proceeding.

I am sure that there is no doubt about this, but the reason why the Cabinet Secretary for Finance and the Constitution is cited in the process is that, from a digital point of view, he is the minister to whom we report on issues around digital public services. In the scenario that you have been given, the civil servants working for the Cabinet Secretary for Finance and the Constitution would have made a recommendation to stop a project in an area in which a colleague of the cabinet secretary was responsible for the delivery of the programme. We think that it is right and proper that both ministers would have to be involved in agreeing a way forward.

Okay. Let me pick out another bit. In annex B, the second bullet point under paragraph 1 refers to

“targeted recruitment for talent required to lead our largest programmes (e.g. social security) and increased support to Accountable Officer and SRO”.

What has been the success rate for that recruitment?

Recruitment has always been, and continues to be, very challenging. I feel awkward about giving you a view on the success rate of recruitment to high-profile digital jobs within social security because of who is sitting next me. We managed to find a candidate with suitable qualifications—

It is significant that it is mentioned but there is no quantification of success.

We are finding it difficult to attract the talent that we need at senior levels within IT programmes. To be honest, that is a problem across quite a lot of the country and in different sectors of the economy, not just the public sector. There is a shortage of high-quality IT people.

What is the percentage gap? For example, if you need 10 people to do a particular job, can you get only five or six?

It is not really possible to quantify it in that way. We are talking specifically about the top level here.

Maybe you can do that for social security.

Andy McClintock is best placed to answer that.

We are in the process of recruiting a number of IT personnel. We are heading to a recruitment level of 68 people and we have 15 people in post, with a further 10 scheduled to come on stream by the end of December. It is not easy to recruit talent, as Colin Cook has said. We take a multi-channel approach in terms of where we actually look at the market, which involves looking at a mixture of the civil service, across Government departments, and the private sector, through external adverts. If necessary, we use interim replacements and resource contracts to fill the gaps.

The skills that are needed are in huge demand not just in Scotland but across the UK. If we compete on salary alone, we will always struggle to compete with the private sector because, in numerical terms, there is often a wide disparity.

Early indications are that, although our results have not been as great as we would have expected, we are no different from anybody else. We see the programme as an attractive opportunity for a number of ICT professionals who want to join it. It has a four-year-plus life expectancy, which is, in itself, attractive to staff, and we are seeing a good response rate. However, the response rate does not often translate into people who actually make it through the process because they do not display the right attributes at the final stage, which is interview. At the moment, we are using a combined approach to resource what we need.

We will use the supply market as we go through our procurement process, which will bring supplier expertise to the table.

We also need to be mindful of our legacy. If we overstaff the programme with permanent people and we get over the hump of the programme, we might have a surplus of staff. Therefore, we have to be mindful of the legacy that we will leave for years to come. We are striking a balance by making sure that, along the journey, we get the right people in post at the right points in the programme as the technology starts to develop.

You have highlighted the difficulties with recruiting specific talent into the IT side. The final paragraph on the first page of annex C says:

“In total, 58 assessors have been recruited and trained to carry out assessments.”

There does not seem to be any problems with getting assessors, does there? Does that mean 58 extra bodies? Who trained them?

Anne Moises can talk about that, but it is a reference to the way in which we carry out digital first assessments. That is done by a group of peers made up of IT and other experts within the Scottish Government. We train people in the assessment process and draw upon their expertise in a particular field as part of that assessment.

Is that so in all 58 cases?

Yes.

Yes.

So there is no additional cost, other than the notional cost.

Other than the notional cost, there is no additional cost.

There is the opportunity cost—

The training course is the other cost.

Who did the training?

We got colleagues from the Government digital service down south, who have been running a similar process for some time, to come up to train our assessors.

Was there a cost to that?

Actually no, because they were very good and did it for free.

I like that.

Yes, so did I.

I want to follow up the earlier points about the assurance process and the reports that were made once you did an assurance task. You said that the reports went to the programme director and that they flagged up any issues. How is a report formatted? Let me explain what I mean. How does it identify what the issues are and who is responsible for addressing them, and what is the timescale for resolving issues?

Our report identifies issues; it does not say who in the team is responsible for actioning them. That is why it goes to the senior responsible owner—the programme director. The next step is that we ask for an action plan to address all the issues. The action plan is normally the responsibility of the programme director and it will identify the issues, who has been tasked with delivering on them and the timescale for when delivery will be completed. Our report says whether things are immediate and critical and have to be done before the programme progresses, but the programme director will give us a very detailed response. We follow that up to ensure that the actions have been carried out.

I was interested in Colin Beattie’s question about recruitment. The Scottish Government notes throughout its document that there are difficulties with recruitment and retention, and the committee has certainly investigated that in some depth before. Can you give us any idea as to why it is so difficult to recruit and retain? Is it to do with pay scales and competition from the private sector such that you cannot buy talent? Is it because the talent pool is too small at present? What is going on?

The easy answer is, “All of the above.”

Before I answer the question, it is worth saying that we outperform the market in retention. That is very much a function of the enthusiasm for the type of work that we do, which Andy McClintock described. From a digital point of view, the ability to work on a programme and a project that have a direct effect on people’s lives is a very attractive proposition for people in a competitive market. To have worked on something that is of national importance is a real feather in the cap.

On recruitment difficulties, we do not pay at the same level that some of the large financial institutions do in terms of basic salaries or bonuses, so we are at a slight competitive disadvantage there. There is a limited pool of certain skills in the Scottish and UK markets, and the majority of businesses—particularly digital businesses—report similar difficulties in recruiting talent.

There are things that we need to do in order to improve the recruitment process. We are working very hard with our human resources colleagues to make the recruitment process slicker and faster for boards, because they need to be as quick as they possibly can be in a competitive market. They need to make all the robust checks, of course, but they must be as smooth as they possibly can be in the recruitment process and make their pitch about the excitement of the opportunity clearly and coherently.

There are improvements that we can make in the process, but there is a combination of issues.

That begs a question, which I will throw out as a hypothetical. On pay, could the argument be structured that there is a false economy if you are unable to pay for the best talent, and that that has a causal effect on output down the line?

It also begs a question about the development of talent. I noticed that there are references to the digital academy. I presume that it will take time before that comes online and starts to produce. What is being done underneath that to develop talent at the earlier levels, in primary schools, secondary schools and colleges?

Those were huge questions. I will try to deal with all the component parts, but please forgive me if I miss one.

You are absolutely right that having the right people with the right skills is essential for the success of major IT projects. That is the clear message from Audit Scotland, and we take it very seriously. If we cannot attract people on civil service terms and conditions on a permanent basis to run projects, there are other approaches that we can take, such as going to the market for contractors. For social security, for example, we are developing a benching arrangement with contractors in the private sector so that we can draw on talent. We are also looking at opportunities for secondments from major organisations to increase the pool of talent that we can bring to bear on major projects.

On our approach to training, the digital skills academy is up and running, and it will be extended and expanded. It provides good training on working in an agile environment and agile project management. We have had great support from the Cabinet Office down south and others. We will continue to build it: we are bringing our own trainers on board so that we can expand it, which is vital. We have a commitment to bring new talent into the organisation at modern apprentice and entry level, and we will train people in the right ways to develop IT projects.

10:00  

The bigger issue is also one that is at the heart of the digital strategy. We have an industry across Scotland, the UK and beyond, but we do not have the talent. There are many thousands of vacancies in tech industries, and the number of vacancies is forecast to increase. We need to address that right from the beginning of and throughout the education system, and the cabinet secretary is focused on that. We need to address issues such as the way that IT is taught in schools and the choices that people—particularly young women—make early in their school career that put them into a channel where the route to an IT career does not appear to be the most obvious one for them to take. The digital strategy addresses all those issues.

In social security, we are trying to be more innovative. For example, we have just announced that the corporate centre will be in Dundee, which has one of the biggest universities for exceptional digital and technology people. We are working with the university to see whether we can bring into the social security programme people who come through those courses, interns and so on.

As a corporate contribution, the new provider that will come on board to help with low-income benefits will go into schools to talk about how to get more people into IT and technical jobs. It has a commitment to sponsor some modern apprentices in technology jobs to support that work.

I come from the north-east and have seen what the oil industry has had to cope with. When there is a dearth of talent, there is a danger that, if you bring on talent, it is then poached. How are you addressing that difficulty?

Social security is quite big on considering our risks; we have just had a deep dive on the particular risks around retaining people. We liaise closely with our HR colleagues on what we can do to retain people and on how we talk to the people we bring in about development, encouraging them into the civil service so that home-grown people will be the talent for the future. We are discussing how to encourage more people to come forward.

Our retention rates are good by industry standards, but if we want the best talent and want people to recognise that a role in digital government is good for their career, we should not be scared that they might take those skills elsewhere. I would love to see people circulating around all industries in Scotland, contributing to Government at a point in their careers, particularly in the more technical functions, such as technical architects and cyber expertise. That talent pool would be a good thing.

I ask Mr Cook to provide the committee with the number of current IT vacancies, as Mr McClintock did, helpfully, for social security. I am sure that he can source that number.

I want to tease out one aspect of the role of ministers. The Audit Scotland report is clear that some problems were down to legislative deadlines. Ministers have control over those deadlines, in a way that officials do not. If a stop notice is reported to the minister and cabinet secretary, can they overrule you in light of legislation? Is that a transparent decision-making process that we can follow?

The commitment is that the audit process is transparent, in circumstances where we think that it is right and proper for the responsible ministers to make an agreement.

I would like to explore methodologies, standards and so on in more depth. However, I will first pick up an earlier point about whether we are talking about IT per se, or the wider aspect of social security. You must be aware that the estimated cost of the IT component of the social security transfer of powers is £190 million, which is more than half the entire cost of transferring those powers to the Scottish Parliament. You have to forgive committee members for focusing on the IT aspects, given where we have come from. It is important to make that point.

I want to know more about the digital first service standard. Where did it come from and when did it arise? Is it in place now, and why was something like it not in place before?

I do not know why it was not in place before. We introduced the digital first standard about a year ago. It is built on and reflects best practice in the UK Government digital service—it has a similar feel to the UK standard and is closely affected by it. The UK standard is acknowledged across the world as a robust, good practice standard for Government digital programmes. It has been taken up and adjusted in countries such as Canada and Australia, which are also embarking on major digitisation programmes. The standard looks at the way in which we organise projects, and some of the controls that it gives us ensure that the user is at the heart of projects.

We have been implementing the standard for nearly a year and are now reviewing it. We are taking all the lessons learned from its first year of application, and we will reboot and expand the way in which we implement it going forward. This is the right time to assess whether we have got the standard right and whether it works in a Scottish context. We might simplify the criteria somewhat or, if we think that there is duplication, reduce their number. We are looking at those kinds of things in the review.

You described it as a sort of peer-group review involving a range of people with different specialisms. The standard is a kind of overarching view—it is not a quality standard that is externally recognised and certified. What do you have in place to control the project lifecycle of a piece of software that you have commissioned, and assure its quality? I will come to the social security system in a minute.

I am not sure that I totally understand the question. The digital first standard will ensure that an IT project and, within that, a software development is tested in an appropriate way, that it is based on the needs of its users, and that it is developed and run in an appropriate way. There will be a clear discovery process, an alpha stage, where we test the technology, and a private or public beta. The standard will also ensure that we set up projects so that they can be continuously improved once they are live. I do not know whether Anne Moises wants to add to that.

The digital first standard is not an externally accredited scheme. It is very detailed—there are 22 specific criteria—and is designed to ensure not just that we are doing a project right but that we are doing the right project. There are quite a lot of qualitative as well as quantitative measurements in there. In some areas, there is reference to external standards. For example, there is reference to the accessibility and disability standards in the World Wide Web Consortium. At key points, it refers to standards that can be externally validated if necessary. In general, though, it is about good practice in government. As Colin Cook said, it builds on the experiences of the UK Government digital service.

I was hoping that you would not say that, because I do not want to make any political points on this. The UK Government does not have a particularly impressive record in delivering IT projects. This is not a political point; it is about IT and expertise. I am a wee bit concerned if we are using a standard that does not have a great track record. Why have you not considered embracing recognised industry quality management standards for IT projects?

To an extent, individual projects have done that. Some projects have obtained ISO accreditation, but we have not identified key external accreditations that would apply across all projects. Rather, we have made it a framework—

Why have you not done that? We are talking about recognised industry models that provide assurance and protect us from cost overruns and software that does not work. That is what such models are about, so why are we not deploying them at the heart of what we do?

At the risk of being slightly controversial, I have yet to see a standard that guarantees against cost overruns. If I could find one, I would apply it tomorrow.

I certainly do not want to make a political point—it is written into my job description that I do not—but I think that the digital standards that were developed by the Government digital service are now recognised internationally as good practice in the development of digital solutions for Government. As Anne Moises said, they allow us to go into more detail on a particular technical point and to make sure that a particular standard is met when we think that that is appropriate. We will bring to bear the expertise to do that. Many of the major projects will do that. A number of the reviews that Anne Moises is responsible for bring in external experts who evaluate against industry standards. That option exists, and we will use it effectively and appropriately.

This time next year, we might be looking at the implementation of the first module of the social security system, the cost of the development of which has been estimated to be £8.3 million. Where does the assurance process for that lie? Is it in the hands of the external contractor that has been appointed, or will your team provide that assurance?

What you described as the first module of the social security system will be classified as a major project. It will cost more than £5 million and it will be incredibly important for the Government’s reputation, for a number of reasons, so it will be subject to the major projects assurance process that we have attempted to describe, as well as to the digital first standards. We will be able to discuss that report and the relevant analysis with the committee in a year’s time, if the committee so chooses.

I am sure that Andy McClintock is champing at the bit to come in here, but as well as the digital first standards, which are great principles to work towards, the bottom line is that we have a technical design authority that will be responsible for looking across the piece at the technology for the social security system and what we put in place to support it. We are considering bringing some non-executives on to that board to give us some external scrutiny, in addition to the scrutiny that we get from the centre. I have engaged some contracts to give me some external challenge on what we are doing and how we are doing it. Andy McClintock has brought into his area some key personnel who have specific skills in areas such as cyberdata. I am sure that they will be working to the industry standards that have been mentioned.

We are a growing team with growing capability. The £8.3 million contract that Willie Coffey mentioned will be the first deliverable to be delivered. That will happen next year. To get to that point, there has been a robust, open and fair procurement process, which has taken us a long time. We have had the stop-go gate, and there has been significant input from digital colleagues to the specification. More important, there has been vital input from procurement. Procurement has a valuable part to play in the whole journey from the conception of requirements to the award of the contract and contract management.

As Lisa Baron-Broadhurst said, I have brought in some experts from elsewhere in the UK public sector who have an understanding of programmes of such a scale, including in the welfare and benefits area. We already have experts who have seen where unfortunate decisions have been taken and mistakes have been made previously who are helping us to design a solution that is modular and adaptable for the future, so if I come back to the committee next year, I hope to have a story to tell on what the outputs of that effort and that planning have been.

This is more than just talk. A lot of the effort and foundation work over the past seven or eight months since I came into post has been about getting the right capabilities and not rushing into the award of a piece of work. The £8.3 million contract is a small part of a longer-term investment. I am very clear that our long-term vision is not for a single supplier to have total control of the programme or to provide a total solution. I see a multivendor, multisolution approach that is adaptable for the benefits of today and what may come tomorrow, including things that will change over the lifetime of the programme and beyond my existence. Ultimately, I could be one of the consumers of this benefits platform—God forbid. Whatever I am instrumental in designing and delivering, I could be a consumer of too. I have a vested interest from a number of perspectives.

10:15  

It will be here by summer 2019, which is not that far away, is it?

I feel old already.

To come back to the point, we have been here before and previous committees have been here before. We see a figure of £8.3 million. How robust is that? Do you have a full system requirement in place? Has it been approved and signed off by Government ministers and the users who have been involved? Is that figure robust or will it change? Will you come back next year and say, “Well, the situation changed a wee bit again. We had to adapt this and change that, and now it is £16 million.”

First and foremost, the contract is capped at a maximum value that cannot be exceeded.

It has taken a long time to get the specification to where it is. It has had multiple inputs from multiple parts of Government, including users’ involvement in the early stages of specification. It has had digital input, OCIO input, programme input, policy input and procurement input. In my view, the specification that went to market was as robust as it could be.

The response from the market was healthy. We got down to a shortlist of suppliers and we finished with a supplier that has been awarded a piece of business. Sitting here today, I am confident that the supplier has the product and the capability—along with our capability—to deliver that solution by next year. The delivery approach will not be that it will deliver the solution and then hold us hostage to fortune. We are looking for a delivery model in which the supplier steps back from the delivery. The supplier will deliver the first stage itself, we will deliver the second stage with the supplier, and we will deliver the third stage with the supplier standing behind us. It is a very stepped model. Lisa Baron-Broadhurst and I have seen evidence from elsewhere that the supplier has managed to do that previously.

With that model and a combination of robust commercial skills on the ground, I have a high degree of confidence.

That sounds pretty good. Am I right in saying that no in-house software development is going on in the Scottish Government and that the development is completely external?

Colin Cook and Lisa Baron-Broadhurst have touched on the fact that this is an agile programme. What we do will be done incrementally.

But is work being done by the Scottish Government IT team, or is it being done entirely by the external contractor?

It is a combined approach and delivery model. To be clear, we are not developing software. We are taking an off-the-shelf-product and adapting it. There will be some customisations to that solution, and there are some licences in the contract, but we are not developing software from scratch.

Why is it £8.3 million, then?

There is a combination of elements in the contract over the two-year life cycle, which is a mixture of services, product, licensing and hosting costs relating to where the platform will reside.

It seems to be quite a high cost for something that is being adapted.

Not really. If I was able to show you a breakdown of the contract elements, you would understand why it is £8.3 million. If you looked at what was in the contract, you would see it as good value for the public sector.

I accept that.

Going back to methodology, am I right to say that the external contractor will apply its own system of controls, checks and quality management? It will apply whatever its standards are to the element that it is developing and testing. It will not be applying your digital first standard.

We were very clear in our procurement specification that the bids had to endorse and subscribe to those digital standards. The bids were marked on each of those 22 principles and standards and there were compliance statements on whether they were either fully or partially compliant. That was the overall basis of the evaluation criteria. The quality management that the supplier applies once it is on site will be a blend of its own quality standards and our approach. It is a joint delivery team and there is an agile approach. We are breaking the delivery into small, bite-size chunks.

This represents a proportion of the potential spend of £190 million that has already been mentioned this morning. This is not the whole story. This is an incremental investment in the benefits platform for reuse in the future—not just for the benefits of today but, hopefully, for the benefits of the future.

I think we can take confidence from the fact that, on social security and other projects, we are already demonstrating how having blended teams with expertise from within the Government and from a supplier—working to the way in which the Government wants digital projects to be developed, as defined by the digital first standards—is changing practice and delivering results. The discovery that was undertaken to lead to the contract that Andy McClintock has described is a good example of that. Work has taken place internally and externally, and we have a good result in a specification to go forward with.

Before we get to the point of spending some, any or all of the £8.3 million over the two-year period, all the pieces of work will be broken down, in agile terms, into sprints. Each sprint is a chunk of work that is clearly specified, with a clear outcome and a clear payment at the end. It is not a case of awarding £8.3 million and paying that out with a cheque. The £8.3 million will be broken down into multiple bite-sized chunks and there will be deliverables along the way. If any of you round the table thinks that I am going to sit and write a cheque and pay out £8.3 million for nothing, you should know that it is not going to happen on my watch.

I know a wee bit about the agile methodology and the iterative, bite-sized-chunks approach that you describe, but one of the criticisms of agile is that it lacks evidence and records, such as testing records. Will you address that criticism?

The point that I was going to make is that, if you think that we are not putting in structures and governance around agile, you are wrong. From a programme director’s point of view, I can say that we will still have all the project artefacts that you would normally expect. You would expect a robust plan and a business case, and all those pieces of the jigsaw absolutely have to be there. They will be measured against the plan so that there is no slippage against timescales. It is a myth that, in an agile environment or an agile world, we do not have a plan for delivery. We absolutely do.

I do not mean that, Lisa. I am talking about the software and when it is being tested. The criticism that I have heard of agile is that it lacks an evidence base and a record base to provide evidence for external audit—for example, from step to step—to make sure that the software is working.

The plan for this piece of work is that, for example, security, cyberresilience and fraud prevention will be embedded at each and every stage of every piece of the software that is introduced or is adapted, so we are building in security by design. Testing will be done at a unit level, so each piece of work and each piece of sprint work will have an element of testing in it, whether it is development testing, unit testing or live testing. Testing will be incremental. It will not be the case that we wait until the end, when the last bit of the £8.3 million is payable, before we realise that we have a system that does not work end to end. The work is broken down into packages, phases and sprints, and the incremental use and assessment of the software are done under the same approach.

I understand that the use of different project management methodologies is a controversial area. We are ensuring—and we reflect this in our standards—that agile methodologies are used where they are appropriate, which is particularly in areas that are new developments. Please take my assurance that we will apply agile methodologies in a disciplined way. That will done under an overall framework of governance, so there will be good oversight of how projects are developed, and the fact that this methodology allows for frequent inspection and adaptation of a product and regular releases of software allows us to have that confidence. We believe it is the right methodology for this particular programme.

My final point is that one of the serious criticisms of previous software development projects has been that there has been a lack of documentation by code writers from step to step. That has been a particular problem where personnel have changed and moved on. It has been incredibly difficult to fix, repair and maintain software for which there has been very little documentation. If you are giving me an assurance that substantial documentation and test records will be available throughout the phases of the project, I will take great comfort in hearing that from you.

I reflect on what I said earlier. A large part of the £8.3 million is based on a product that already exists, and it is well documented. Anything that we do to adapt or modify it or integrate it with other systems will be documented by us, with us and with the supplier. It is not a case of our having to document every single screen and code from the outset, because we are not building a system from scratch. One of the key attributes in considering the bid from the supplier was about the element of reuse. We have actually lowered the risk to the programme and enhanced our chances of success of delivery by building on the back of a product that has global use.

I am interested in an issue that Willie Coffey asked about. You say that the £8.3 million is for an off-the-shelf package.

It is based on an off-the-shelf solution.

Mr Coffey said that that seems like quite a lot of money, and it seems like quite a lot of money to me as well. If there is an off-the-shelf solution, or a solution that is based on an off-the-shelf package, it must have been used before by someone else. What did they pay for it?

I cannot share that, because I do not have the information on what other customers might have paid for the software component of our contract. However, I can say that I am satisfied in terms of value for the public purse that the end point that we have reached is commercially the best possible solution or outcome for the Scottish public sector.

Just to press you on that, how certain can you be that you have cut a good deal if you have no idea what the cost was to other users?

Most suppliers, including this one, will enter into commercial confidentiality agreements with all their customers, and it is very hard for that confidentiality to be broken. However, it is fair to say that, with enough market intelligence gathering and discussion among the supplier community, you can get a feel for the investment costs of software and where a price lands in that. My colleagues round the table will attest to my scrutiny and acumen when it comes to commercial values, and I am convinced that the price for the contract is the best possible one that we could secure in the current climate given our requirements and what we need to deliver the software.

You talked about a bite-sized process, which means that you are not just writing a cheque up front.

We are absolutely not doing that.

One issue with the i6 project, which the committee has looked at, was the ambiguity in the contracts, which meant that no one knew what was being delivered or who was responsible for the delivery. There was also an issue with how robust the indemnities were to ensure that any cost overruns would not fall on the public purse. How confident are you that, in your bite-sized process, the contractual documentation and indemnities are sufficiently robust so that, if there is a problem, it does not fall on the public purse?

First and foremost, as I said, the contract is capped at a maximum value. The contract has been awarded with a maximum contract spend that cannot be exceeded. There is a tolerance, but the overall value is capped.

On the indemnities that sit behind that, I am not a procurement specialist, but my procurement colleagues have been with us every step of the way. We have specialist procurement people embedded in our programme and who work alongside us. Every step that we take is taken hand in hand with procurement. I am satisfied with the indemnities in public procurement and satisfied that the framework that we have used indemnifies us.

To take it down a level to the bite-sized chunks, each piece of work will be driven by a statement of work, which will be generated by us and agreed by the supplier and which will set out what is to be delivered in the sprint over a six or eight-week period. I would expect the majority of those to be fixed price but, irrespective of whether they are fixed price or variable, the maximum value that the supplier can get for the contract is capped and, within that, the costs of software licences are fixed and locked.

Are the costs of IT hardware separate from the £8.3 million?

No. The platform solution that we are running will be cloud based, which again is in accord with digital principles and standards. The cost of running the platform is within the £8.3 million. The costs of the virtual hardware that the software will run on are within that, as are the annual hosting costs for the two years of the contract.

So you are confident that the hardware will be encompassed within the £8.3 million.

It is already in the specification. As part of the bid, the supplier has included a configuration for the platform. That is to the supplier’s specifications, based on our users, anticipated volumes, the number of benefits and the number of payments that have to be handled and transacted. That is all architected in the overall solution. There is a hosting element in the £8.3 million—it is accounted for in the cost.

Is there any detail on how the overall cost in the financial memorandum of £190 million has been built up?

10:30  

Colleagues who have been before the Finance and Constitution Committee have already touched on how the £190 million figure has been arrived at, so I will not go over that in forensic detail. I can say that the maximum level of optimism bias in accordance with Treasury green book standards has been applied. We are on a journey to deliver a range of technology solutions to support the programme and, ultimately, the agency. When those things were put together, they were based on what it is believed will be required to get the various technological solutions in place. It is an incredibly complex and challenging journey and nobody is saying that it is a spot-on accurate figure. I think that the estimated figure is based over four years. That is how that figure has been arrived at, but significant optimism bias has been built into it—unlike perhaps other programmes in the past, where optimism bias was at a much lower level.

I have examined the various responses that have gone to the Social Security Committee and the Finance and Constitution Committee on this and have yet to see any explanation as to how the £190 million cost has been built up. Does anybody have that information to provide to the committee?

I cannot sit here and give you a fact-by-fact, line-by-line explanation, but I will take an action away to make sure that the committee is furnished with more details that perhaps give greater clarity on how the £190 million was calculated. I think that finance colleagues before me have attempted to answer some of those questions. I thought that they had done so satisfactorily, but clearly not.

Just to be clear, I am not looking for a narrative or a description as to how the £190 million was arrived at. I am looking for a table on how those costs have been built up and what the different component parts are, which will therefore show how the overall figure has been arrived at. The figure is in the financial memorandum to a piece of legislation that is before this Parliament, so it is quite important that we are able to back that figure up.

Okay.

That is great. We are grateful that you will be writing to the committee on that point.

We have, I think, not spoken about future proofing. You said that you are taking a standard product and amending it in some way. If a manufacturer comes along with an upgrade, a fix, a patch or an update, how future proofed are you when you then have to start—as I presume you would—adjusting the product to allow for whatever you did to it in the first place?

Our approach will be to take the product in its most vanilla standard form and adapt it as appropriate for the social security powers for Scotland. In doing that with the supplier, we will not take the product into a space where it cannot, without reverse engineering, receive routine upgrades and patches. We will make sure that the product is used as much as possible as it is when it comes out of the box, with adaptations and reconfigurations. The supplier is on the journey with us; we will rely on them to make sure that we do not take the product into a space where it cannot be upgraded.

Fair enough.

I draw your attention to paragraph 21 of your submission—I have a couple of questions on detail. You say that

“Data innovation could potentially benefit Scotland by £20bn*”.

There is a wee asterisk there. I cannot find the corresponding asterisk to tell me how you have arrived at that figure. If I cannot find it, maybe you cannot either, so perhaps we should agree that you should stop looking in your papers.

Forgive me; I can write to the committee about that. That is an externally generated figure that is widely used in many contexts, including the city deals, so it is fairly well trailed. I apologise for the lack of the footnote.

That is okay. It just shows that we read exactly what you send to us. If you could write to us with the source and an explanation, that would be particularly helpful.

In the same paragraph, you go on to say:

“Scotland has a world leading set of public sector data”,

and then you say that the data will

“deliver £1bn in public sector efficiencies”.

If I was the cabinet secretary for finance, I would be jumping all over that figure, given his current budget problems. It is not a figure that I recognise. How was it built up? Where do you get it from?

Also, if I can just be a little sharp about this, I point out that so far, we have had hundreds of millions of pounds of failure in IT projects. You mentioned three yourself—the NHS 24, CAP futures and the Police Scotland IT projects. I find it really difficult to accept your statement that somehow the data will create £1 billion of efficiencies because—I say with respect—that is not the committee’s experience to date.

I take your point, and I will go back to reference the figure. It is based on a global understanding of how to use data in the public sector and in delivery of public services, and includes such things as use of predictive analytics to predict when particular health and social care circumstances might arise. It is a potential figure that has been built up by independent experts, I believe. I will come back with the source of the figure.

It is not a figure that Derek Mackay can say will happen in this budget.

I suspect that it is not a figure that Derek Mackay will commit to over three years, but he is definitely engaged in the process of examining how we use data to deliver efficiencies in the public sector.

I just do not like overclaims, so paragraph 21 might need some adjustment.

I am curious to know whether any panellists are qualified IT professionals.

Anne Moises indicated agreement.

Andy McClintock indicated agreement.

Anne Moises is, and Andy McClintock behaves as if he is. [Laughter.]

I am not sure what you read into that.

I am not an IT professional.

IT is a technical area that is, I confess as a lay person, difficult to understand. That must be the case for our non-IT professionals as well.

IT is a technical area that also requires a thorough understanding of user needs and how business processes work. Commercial skills have also to be brought to bear. An IT professional has a combination of all those things in front of them. My team includes a combination of user research, service design, commercial skills and technical skills, where appropriate. That is quite deliberate.

Your argument is that restructuring is taking care of the lack of capacity that existed in the past.

By restructuring, we have identified where we have gaps, which we are now filling with the people whom we need. Some will be from within the organisation and some will be from outside it.

From my perspective as a social security programme director, that is why I have a very good chief digital officer.

Indeed.

I have been in the Scottish Government for just over 10 years; before that, I spent time in the public health sector in England and five years with a commercial software company, so I have a good understanding of the mechanics of software revenue and of all the things that go into software delivery. I bring a mixture of private and public sector skills to the table, which is why I have been selected for what they tell me is this easy job.

Excellent. Do not go anywhere soon. [Laughter.]

I stick with the topic of people and their expertise; what is the role of your group of senior academics who provide challenge and advice?

We set up that group to challenge our approach to the digitisation of Government—specifically for the development of digital business models for Government. It includes Mark Thompson from the Judge business school at the University of Cambridge, and Alan Brown from the University of Surrey. The group’s role is to identify international best practice and to challenge us on whether we are following it, which they have done successfully. We say in the submission to the committee that a recent short research project by their MBA students has helped to show where Scotland is positioned in the international context, and it has provided us with a few important pointers. The approach is to set up and build new parts of Government based on digital business models. The way in which we set up a business in 2017 is not how we would have created a department in 1945, or whenever.

External expertise is helpful, but are you aware of whether any of those academics work in the private sector or have consultancies while they advise you?

At least one of them has a role in a consultancy, but we make sure that there is no conflict of interests in respect of how we use that individual.

I will press you a bit further. What if the academics are sitting around the table with Government at the same time as they hold consultancies with organisations that are bidding for contracts—for example, for social security—or have been involved with Accenture, which was the company in the failed IT project for Police Scotland, which I understand one person has. How do you ensure that there is no conflict of interests, given their involvement to date with Scottish Government IT projects and potential future involvement?

I am not aware of a conflict of interests, although there might well be one. I apologise, but I am not aware of anyone’s direct involvement with Accenture—that was not the example that I was citing.

Those people are employed to look at international best practice and to challenge us about the overall formulation of our approach to development of digital business models. They are not employed to advise on specific programmes of activity, and certainly not on a procurement specification for any activity. That is not their remit. They are looking internationally, identifying best practice and challenging us about whether we are meeting those standards. They are not dealing with specific projects.

How do you check—I assume that you do check—the backgrounds of people to ascertain whether there is a conflict of interests? Have you done that for them all? I ask particularly because I now understand from you that the positions are paid.

We always look out for conflicts of interests and we recuse from an issue that would lead to a contractual award anyone who has a conflict of interests. However, that is not the case here.

I apologise for not being sure who was involved in the Accenture case. We look into people’s backgrounds because we want to ensure that we have the best advisers. The two names that I cited would, under most external scrutiny, appear to be two of the top experts in the UK on digitisation of Government. We respect them and take their advice.

I am sorry, but I have a list of some seven or eight names here. The issue is that there are multimillion-pound contracts in the public sector and some of the academics might well hold posts in the private sector, so you need to guard against influence. I am looking for you to confirm that you have, as a matter of routine, checked the backgrounds of all of those people.

I can confirm that they will have no impact on—

That was not my question.

I know that we have looked into the biographies of those people. They have been proposed as experts, so we examined their credentials as experts. I will make sure that we have all the necessary documentation in place; if we do not have it already, we will make sure that we get it. I will come back to you on that in order to give the committee that satisfaction. Please accept my assurance that those people are not dealing at any level with anything that will result in a direct contractual award; that would not be appropriate.

That is very helpful reassurance and I would welcome that information in writing.

I assure the committee that, for the procurement that was just completed on the £8.3 million contract, none of Colin Cook’s specialists or advisers had any part to play in any part of the journey.

That is very helpful to know, too.

There are no remaining questions from committee members; I think that the panel has exhausted us all this morning. I thank the panel for their evidence.

10:42 Meeting continued in private until 11:11.