Contents
- Attendance
- Decision on Taking Business in Private
- Social Security (Scotland) Bill: Stage 1
- Social Security (Scotland) Bill: Stage 1
Social Security (Scotland) Bill: Stage 1
In item 5 we will hear from the Auditor General for Scotland. Thank you very much for coming. I know that you have been very busy today and have come straight from another committee. I invite you to make opening remarks.
Thank you for inviting us, convener. It is always a pleasure to be here with the committee. I will make sure that my opening remarks are brief, because I know that your time is short.
In May, we published a briefing paper that pulls together the lessons that have been learned from a range of work that we have done on previous IT projects with lessons from around the world, which we hope will be useful to the committee in your consideration of the issue. In that paper, we identified a number of common themes in the difficulties that have been experienced in digital programmes, which we grouped into a set of five principles covering planning, governance, users, leadership and strategic oversight and assurance. It is important for us to be clear that those principles cannot be considered in isolation, because they interact with each other. Alongside them, throughout the briefing, we pulled out the importance of skills and experience as a cross-cutting theme that is critical to success.
In March, we published the latest in our series of audits on how the Government is implementing its new financial powers, of which social security is an important element. Among the key messages in that report was that, in many ways, moving into having social security powers is a step up in terms of the complexity and scale of what the Government is trying to do, and that there are some real challenges still to be tackled.
Since we published the report, things have moved on: we will publish a further report in the spring of 2018, which will look at the progress that has been made, since our report this year, in how the Government is planning and organising in order to deliver its social security responsibilities—in particular, the governance and leadership arrangements and the plans for developing the IT systems. We will also look at costs and at progress to date.
Mark Taylor and Morag Campsie, who are with me today, have been heavily involved in both those pieces of work. We will do our best to answer your questions.
Thank you. The committee has the excellent reports that you have produced, and we look forward to your report on how we will go forward.
What lessons do you think should be taken into account in designing the social security IT system?
In our digital briefing, we pull out the importance of getting some of the planning in very early, whether that is planning for the scale of what the system needs to do, planning to have the right skills and experience in place or planning for the right governance arrangements. Often, when we look at a system that has not gone as planned, we see that the roots of the problems are in the very early stages—for example, Police Scotland’s i6 system and in the NHS 24 system. We are therefore keen to see planning being started early.
Morag Campsie led on the digital briefing work for us, so she might want to add to that.
As the Auditor General said, planning is key, as is getting the right people in from the start. Having integrated teams involving policy, service design and digital experts right at the start is important. In the past, especially in complex policy areas involving an IT problem, the policy has often been designed only for us to find out that it is not easy to design a system to deliver the intended outcomes. Given that many of the benefits that are to be devolved will be managed online and that there may be others in the future, the need to develop a system that is future-proofed and that can easily be changed as we go along is also key. Governance arrangements are always critical, and ensuring that the right level of skills and understanding exists at all levels of governance is critical.
Thank you for that. The committee has met both sides—Westminster and the Scottish Parliament—together, and they seem to be getting along well and doing quite a good job.
Other members want to ask questions. Ben Macpherson will begin.
It is interesting that the briefing paper starts on the need for clear, thorough and effective planning. That is an issue that has been raised today, along with design.
One of my responsibilities is as a member of the Justice Sub-Committee on Policing, which looked at the March 2017 paper that you produced on i6. One of the important lessons from that was about the difference between the waterfall and agile methods for developing IT systems. To clarify for the rest of the committee, paragraph 15 of the report states that in the waterfall method
“software is developed in distinct phases, each leading to the next phase in a sequence resembling a waterfall.”
That creates the potential for a phase stalling if the previous phase is not delivered, whereas the agile development system is, in the wording of your report,
“a more flexible, incremental approach where the team work on small-scale launches of a functioning product.”
I know that the issue is quite technical but, for the benefit of getting it right for social security, it is worth raising. To me, that was the stand-out point in the “i6: a review” report. Do you agree?
Yes—I agree absolutely. I hope that one of the themes that comes through in the digital briefing is the sense that, although it can be tempting to think about a big-bang approach that aims to tackle a big problem all at once, our experience, and that of projects elsewhere, suggests that it is increasingly important to break projects down into manageable chunks and to think about how to build from a good start towards the things that will matter in the future. That is particularly true in the case of social security, as Morag Campsie said, because we know that existing benefits that are within the bill’s scope will come on over time and there is always, in the context in which we all work, the possibility of further changes to the devolution settlement. That is increasingly possible with project management approaches such as the agile approach and with how technology is changing. Much more development is being done though rapid prototyping, and apps have been developed that do particular things, but also interact with each other.
It is also important that, if the agile approach is taken, it is built in at the beginning with the options appraisal, the procurement options, and the skills and experience that are available. Morag Campsie will want to expand on that. We have seen examples of people starting to use the agile approach without fully understanding what the implications are, and having to back up and start again.
The Auditor General is right: the agile approach is being used a lot more in the public sector. It is likely that the agile approach will be used, but there might be parts of the programme that will use more traditional methods. Things can be tweaked to see which is the best fit for what you are trying to deliver. It will be key to ensure that everything is in place at the procurement stage for alignment with whatever method is to be used.
Returning to governance arrangements, when the agile approach has been used in the past, governance boards have often not fully understood it. It will be key to clearly set out where decisions will be made and at what speed they will need to be made, because that is always a critical feature of using the agile approach. Clarity about that and who will be responsible for making decisions will be a factor.
As it was with i6, is it almost essential to use an agile approach? It might be necessary to an even greater extent because of the complexity of DWP data and the systems that will be inherited or built upon to deliver the new benefits in a new IT structure.
The Scottish Government is probably still thinking through which method to use for which piece of the programme. I cannot say that one method would be better than other. However, it is key that that is all thought through and planned, and that the right processes and arrangements are put in place to manage the programme and to scrutinise activity to ensure that everything is being delivered.
You might know that yesterday the Finance and Constitution Committee, on which I sit, took evidence from the bill team and other officials from the Scottish Government about the bill’s financial memorandum. The Finance and Constitution Committee will report to this committee in due course, but it is fair to say that a number of concerns about some of the numbers used in the financial memorandum were expressed, not all of which were resolved. Relevant to today’s meeting were the concerns about the figure of £190 million that is used in the financial memorandum in connection with the information technology costs. How should we treat that figure?
There are two things to say about that, and I will ask Mark Taylor to come in shortly.
First, I know that one of the areas under discussion yesterday was the relationship between the figures in the fiscal framework for funding new devolved powers, including social security, and the figure in the financial memorandum. In our March 2017 update on the new financial powers, we reported about the way in which the figures in the fiscal framework had been reached.
From the available documentation, it is clear that they were not intended as an estimate of the cost; they were a contribution that the UK Government is making to the Scottish Government’s costs. It is important to get that on the record first.
The second point is about the quality of the estimate in the financial memorandum. It is entirely appropriate that the committees of the Parliament subject those estimates to proper scrutiny. We have seen examples in the past where those figures have not stood the test of time as the policy is developed and the new services or agencies are put in place.
As part of our continuing work, we will be auditing the basis on which those estimates have been developed and how they stand up against experience as the work rolls out. I will ask Mark Taylor to come in here as the person who has led that work on new financial powers so far.
When we reported back in March, one thing that we were clear about was the need for the Government to develop its thinking to the extent that it was able to assess the cost that it was expected to require. It needs to recognise the link between decision making—some decisions are still to be made and some approaches are still to be determined—and how those decisions affect the overall cost figures.
We are clear that there is a need to establish a benchmark for the costs and the Government needs to manage against that benchmark on an on-going basis. We also recognise that things happen as decisions are made, so it needs to refine that benchmark and keep it under review.
As we look to our new piece of work, we will pick up on how that plays out in practice. It is apparent that the Government has moved things on since we last spoke and some of that work has been reflected in the estimates that have gone into the financial memorandum. As the Auditor General says, there is still a lot more work to be done around the costs that all this will incur, the management against those costs and the value that is delivered out of that spending.
My question was about how we are to understand that £190 million figure. Correct me if I am wrong, but your answer is that we should understand it as a benchmark. How has that benchmark been arrived at? Why is it £190 million and not £150 million or £390 million or anything in between?
We do not yet know very much about the agency, where it will be or how many offices it will have. We have been given estimates of its annual running costs and its eventual staff size. We do not know anything very much about the extent to which the new devolved benefits will be automated. Pauline McNeill has a question to ask in the chamber later on today about that. We might know a bit more about these things on Tuesday when the Minister for Social Security gives her next statement to Parliament. It might touch on some of these issues; I do not know because I have not seen it. Given all the things that we know that we do not know, how reliable is that £190 million?
That is a good question, and it is one that you genuinely need to direct to colleagues in Government. I am looking at our report from March on developing new financial powers, and one of our key recommendations is that the Government needs to model fmore detailed costs and develop its plans and timescales for the implementation of the social security powers.
When we carried out that audit work, there was not enough for us to be able to comment on the robustness of the work assumptions that were in place. That was six months ago. The figure in the financial memorandum will have a basis in the work that has been done by the Scottish Government, and it is entirely appropriate for Parliament to test that with the Government through its committees. Our work will do that when we report back next spring; we are not yet in a position to give you that assurance.
11:00
You are not in a position to give us any assurance that the figure is robust, so we will just have to discover that for ourselves by putting the right questions to the right ministers at the right time.
Our audit work at the beginning of this year looked at the Scottish Government’s circumstances at that time. We did not consider that the modelling of cost was detailed enough for us to be able to comment on it and we recommended that the model needed to go further
I do not mean to be pejorative in asking this, but how can we know that the figure is anything more than just a guess?
At this stage, this committee and the Finance and Constitution Committee can ask the Government about the basis on which the £190 million figure is put together, with the assurance that we will be looking at it as part of our audit work and reporting back in May 2018.
I have been ploughing through all the lessons to be learned on information and communications technology management—and there are many. It seems to me that this is not about being ICT literate, because a common thread throughout is about thef need to apply basic management principles. There needs to be one team, the people on it need to talk to one another, there needs to be project governance, the end users need to be involved and so on.
From the evidence that we have heard from Jeane Freeman, the Minister for Social Security, a lot of the issues that have been raised have been planned for. For example, we have 2,000 end users on panels, and we can input their views into the system.
In some cases, the use of short-term contracts has led to problems with ICT management systems. Is it your view that the Government should learn lessons from those cases on whom they should employ? I do not know if you are able to comment on whether we have the expertise to carry out the work. The size of the project is probably bigger than anything I have read about so far.
You are right about the scale and complexity of the work. As Morag Campsie has said in our work on digital programmes, a continuing theme has been about the importance of the right skills and experience and, very often, either the lack of them or their poor use.
The Scottish Government’s digital directorate has been working hard to fill the short-term gaps and to develop longer-term capacity in the Government and across the public services. Recently, the chief information officer wrote to the Public Audit and Post-legislative Scrutiny Committee to update it on that issue. I understand that you have a copy of that letter—it is certainly in the public domain. We can talk about that a bit more, if that would be helpful.
It is entirely appropriate for big programmes to make use of contractor staff. Such projects have a big hump of workload that needs to be accommodated, and having staff in post to do that all the time would not be a good use of constrained public money. However, we often see that those teams are not well integrated with the programme and policy staff. As Morag Campsie has said, such integration is critical. We also see a lack of good plans for transferring their knowledge and experience—in general, but particularly in relation to the system that they are developing—to the staff who will take on long-term responsibility for the system. Therefore, the skills and experience that will be needed must be planned for from the beginning, rather than trying to bring those on board in a rush because of tight timescales. There also needs to be planning for how to transfer contractor staff experience to the permanent staff. Those two elements make that a good way of working rather than an additional risk in an already big and complex project.
I appreciate that you may not be able to answer my follow-up question—it is a bit of a hot potato. I am not trying to draw you into the discussion around where the new agency is to be located, but, as you have said, planning is taking place at the early stages—which is now—and will be followed by the establishment of the agency itself. Presumably, planning and managing will go into identifying the start date. Does where the agency is located matter, in the sense of where it might need to draw its skills from?
That, in some ways, is a policy decision, so I am precluded from commenting on the matter—for good reasons. However, you are right that I would expect that when the Government is making decisions it will be thinking about where it will have access to the skills that it needs, as well as considering the interactions with other parts of the public sector. Mark Taylor may want to comment on that, given his thinking on the broader social security programme.
The short answer is that a range of factors come into making that decision. A policy decision that the Government will make is how to balance those factors. Access to skills and to a workforce is among the factors that I am sure that the Government will consider when making its overall decision.
Auditor General, you have highlighted the complexity of the whole scenario, which the committee has also highlighted. As I have said on numerous occasions, there is no big red button that we can press. People just want to know when their benefit money will be in their account and they want a seamless transition from one system to the other.
I have looked at your briefing paper from May and the handy wee infographic on page 5, which shows the five principles for success. I have worked in the real world, in an industry that loved infographics but which did not necessarily read or abide by them. Given your dealings and expertise in this area, do you know whether the Scottish Government has worked towards those five principles so that there are no issues on the day?
I would say that it is work in progress. As I think that the committee knows, we produced the briefing paper because we have reported on a number of different IT programmes, of varying degrees of importance, that have not succeeded. We thought that, to help people learn from our reports, it would be useful to pull that information together. The Scottish Government is taking the issue very seriously and has recently given evidence to the Public Audit and Post-legislative Scrutiny Committee on its progress in making the underlying changes that are needed to be able to do this work better.
Equally, we all recognise that there is no quick fix. We continue to look at the way in which the specific programmes that we audit are being delivered and we very much welcome this committee’s early interest in how that is going. However, I am very conscious that, as Ms McNeill said, often the things that go wrong are the commonsense things that you would expect to be there all the time. Human beings and organisations are not perfect and things do not work as planned. It is often the softer things around culture and leadership that make the biggest difference. The reason for the reports that we have produced so far is to acknowledge, first, both that progress is happening and the complexity and scale of the issue, and secondly, as you said, the potential of this to have a real impact on people’s lives and, in many cases, the lives of those who are the most vulnerable in Scotland.
You said that, in previous programmes, everything happened early on, at the planning stage. From what we have seen, the Government seems to be open to getting it right at this stage. Has there been sufficient interaction with the in-house and external organisations that you mentioned to ensure that we have addressed the situation?
In the March report on the new financial powers, we talk about the good start that has been made. All that I can do, however, is refer again to the unprecedented scale and complexity of the issue. Even starting as early as the Government can start, the timescales are still short, which is unavoidable given the timescales that have been agreed for the transfer of powers. However, there is no doubt that this is a very significant challenge for the Government.
The complexity lies in the fact that, as the committee has heard previously, there are three or four different computer systems that do not talk to one another. In addition, some of the information is in a manual system in some undisclosed place down south. Trying to get all that together adds to the complexity. Do you believe that the Scottish Government has stuck with the five principles, while working towards ensuring that we do not have difficulties at a later date because of information and data being a major issue?
With the caveat that I gave in my answer to Mr Tomkins, we said in the March report that we thought that the Government had made a good start, but the examples that you are talking about highlight how complex the issue is. Until we have done the next round of audit work, I do not feel that I can give very much more assurance around what we are seeing.
What we saw in March and what we have seen since then is a commitment from the programme to learn lessons from other systems. We talk about that a little in the March report and I know that information has been shared with the committee since then about the number of organisations that the Government has spoken to in order to understand and learn those lessons. We are optimistic but yet to be convinced.
The theme of the need to ensure that we have the correct skills, experience and expertise comes up time and time again. How much do you think that we, as a committee, need to scrutinise that? We are discussing principles and relationships with the UK Government and so on, but if the system is not delivered properly, it will have devastating consequences for millions of people.
Could you elaborate on where you believe we are at the moment? The Scottish Government will be making more payments in a week than it currently makes in a year. How do we make that leap and ensure that it is successful?
There is no simple single answer to that. It might be helpful if I talk you through what we will be looking at in our audit work, because I suspect that there will be strong parallels between that and what the committee will be interested in.
We will look closely at the plans that the Government puts in place on the overall programme and the individual workstreams, and we will test them to make sure that we think that they are realistic, that the interdependencies are taken account of and that they are doable, given that the capacity of the civil service is smaller now than it was 10 years ago and that there are a number of other pressures on people’s time, for good and well-known reasons.
We will look at the way in which the Government models the costs of what it intends to do. We will consider not just the costs of the programme but where it sits in the overall financial envelope as we move into a new world in which the Scottish Government will be raising about half of what it spends, which brings with it a lot of volatility and uncertainty.
We will also look critically at the people aspects—the leadership of the programme and the extent to which people are making choices about priorities, working those through their plans and making sure that the right people with the right skills are in place and are being supported to do what is needed on a long-term basis. In relation to the common agricultural policy futures programme, which has been another big area of interest for us, we are very conscious of the long-term strain on people as they have tried to recover from the situation that became apparent a few years ago. A huge commitment is being shown that is admirable and which should be recognised, but people cannot be expected to work in that way indefinitely.
We know that people will be working very hard in the social security programme to meet the 2021 timescale for full transfer. As well as thinking about having the right skills in place, a workforce needs to be developed that can do the work in the longer term. Expertise and experience need to be built up, and the risk should not be run of burning people out to meet very short timescales. We do not see any evidence that that is happening now, but it is one of the things that we will be looking for. We want to ensure that there is a sense of sustainability as we look ahead.
I suppose that I have been around too long, because in local government and the national health service, we have lots of reports of lessons learned, yet we always seem to fall back into making the same mistakes. That is true of all parts of national and local organisations.
To carry on from Alison Johnstone’s point, what is the best way for the committee to scrutinise the social security IT programme to make sure that we do not make the same mistakes that have been made in previous projects? Is there an endemic problem in local and national Government that leads to those mistakes being repeated, or does the same problem exist in the private sector?
I will start with your last question, because it is the easiest one to answer. There is no doubt that the same problem exists in the private sector as exists in the public sector. There have been highly visible failings in banking over the past few years. We will all have read the articles that suggest that most banks still have a deficit to make up in the robustness and resilience of their IT systems, which we all depend on daily. The issue does not affect only the public sector by any means. The skills that are needed are in scarce supply right across the economy.
As far as what the committee might be looking for is concerned, I am not sure that there is much that we can add to the answer that we gave to Ms Johnstone, other than to say that a good starting point would be to develop, with the Government, some clear shared expectations with regard to what the committee is interested in and the frequency with which the Government should share that information with you. The fact that we will be reporting regularly on the programme will act as back-up. We will do so a couple of times a year: first, in the spring update on the new financial powers more generally, of which we know that social security will form an increasingly big part; and, secondly, in the annual report that I do on the Scottish Government’s accounts, which pulls out significant aspects of the audit each year. That tends to come out in the autumn—the next one is due towards the end of this month or in early October. The committee will get assurance from us twice a year about the problems that we see.
It is a case of agreeing with the Government what you expect to receive and how it will provide you with that on a regular basis. You do not want to receive information on the programme on such a frequent basis that you are constantly pulling it up by the roots to have a look, but you need to do so regularly enough to enable you to pick up signs that things are going off track before it is too late to do anything about them. That would be a good starting point for the committee’s scrutiny.
Thank you.
Thank you for answering our questions so succinctly, Auditor General, as you always do. I thank Morag Campsie and Mark Taylor for their answers, too.
11:15 Meeting continued in private until 11:37.