Public Audit and Post-legislative Scrutiny Committee

Meeting date: Thursday, May 25, 2017

Official Report 478KB pdf

---

“Principles for a digital future: Lessons learned from public sector ICT projects”

Item 2 is oral evidence from Audit Scotland on its report “Principles for a digital future: Lessons learned from public sector ICT projects”. I welcome from Audit Scotland Fraser McKinlay, controller of audit; Gemma Diamond, senior manager; Morag Campsie, audit manager; and Lucy Jones, auditor.

I invite brief opening remarks from Fraser McKinlay and Gemma Diamond.

Thank you, convener, and good morning, members. I will say a brief word and then hand over to Gemma Diamond to say a little about the content of the report.

The main thing that I want to say is that this is a slightly different report for us. As the committee knows more than most, over the past few years we have done a lot of work on information technology projects that have not gone well, and, in discussion with the committee and others, we felt that it would be helpful to pull together what we thought might be some of the main themes and principles from our experience and that work. As a result, the team has pulled together a set of principles, which Gemma Diamond will touch on in a second, that we think covers the measures that are not in place when things go wrong and therefore the things that really need to be in place if these major IT and digital programmes are to be delivered successfully. As well as drawing on our own work, we have, as you will have seen, drawn on work from colleagues in other audit organisations around the world and, indeed, other organisations. Interestingly, there is a lot of commonality in the themes.

We are very happy to bring this report to the committee to help you consider how you might scrutinise the whole question of digital public services and IT and, I hope, to inform a wider debate in the public services on how these things might be done more effectively in future. With that, I will ask Gemma Diamond to say a few words about the content of the report.

We want this document to stimulate discussion in public bodies about what has happened in other organisations, the weaknesses that have come up and the things that they might need to look at and learn from. Instead of using a checklist format, we have based the report on a series of principles; there are five main principles that are intertwined and cannot be looked at separately. In all our reports, we have pulled out the issue of skills and experience as a key thing that organisations must have in place, and we have intertwined that through the whole report, given that it affects all the principles that we have set out. Indeed, we have used a little icon to draw attention to its being a key factor.

As I have said, we want organisations to use this report to help them learn from the past and move forward, and we have illustrated our points with a series of case studies, quotes and real-life examples. As Fraser McKinlay has said, we are very happy to take more detailed questions on any of the principles that have been set out.

Thank you very much. That was helpful.

I will kick off with a question to set the scene. The committee considers IT projects only where problems have arisen, but I am keen to get a sense of the overall picture. How much public money has cumulatively been lost as a result of the IT failures that we are aware of, and can you give us a picture of the ratio of successful to unsuccessful IT projects?

I am not sure that I am going to be able to give you that information now, but we can certainly do a bit of work on the numbers. It might be trickier to work out the balance of successful and non-successful projects, because an awful lot of IT development goes on under the radar, but we can certainly see what we can do and write back to you, if that would be helpful.

It certainly would. At the same time, it would be good to get an idea of how that compares with other countries around the globe as well as with the private sector in Scotland, if it is possible to get those figures. I am sure that everything is not as wonderful as it seems, and it would be useful to get that context.

I now invite questions from colleagues.

Convener, I just wanted to emphasise your last point. It would be useful to benchmark ourselves against other countries. I would say that Estonia is a very good example of a country where the use of IT is well advanced, although not necessarily for data purposes. The Government there is literally paper free. There might be other countries that we could benchmark against, as well as benchmarking against the private sector. That would be a useful exercise for Audit Scotland, and it would give us a sense of what we needed to do to get to the top of the league for performance in all this stuff.

We are happy to do that, Mr Neil. I should say that we have done a bit of that for this report; for example, we looked at reports on places such as Estonia, and we know that the Government is looking at Estonia and other places to get a sense of this.

The Estonian experience is interesting. From my relatively limited understanding, it appears that the Estonians had the benefit of starting with virtually no digital infrastructure—

It is a lot easier to do this if that is your starting point.

They were able to fast-track to an entirely digital system because, unlike us, they did not have to unpick legacy systems. That said, there are absolutely things for us to learn from them.

The private sector is interesting. Some of the reports that we link to in our report contain examples of some private sector IT projects that have gone well and others that really have not. I take the convener’s point: it is too simplistic to say that the private sector does this well and the public sector does not. The fact is that we do not hear about it as much in the private sector. We can have a think about whether we can get more information on that, although it should be recognised that, inevitably, it will be more difficult for us to get to grips with. We will certainly have a look, though.

That is very helpful.

The European Commission benchmarks the use of digital in European countries, but that tends to be at United Kingdom level. However, the Scottish Government might want to explore ways of benchmarking against other parts of the UK and the European Commission, given that the information is available at EC level. In our further reading list, we link to one of the benchmarking reports.

I have to say that I was a little bit surprised by this report, because it was not what I had expected. It is as much as anything else an aggregation of best practice but, from what the Auditor General had said, I had been expecting a better analysis of what is happening with public sector IT procurement in Scotland, in view of the well-publicised problems with NHS 24 and other systems that have been installed. I had expected a better overview of the steps that the Government has taken, whether the new structure that it has put in place is working and how all this is going to be pulled together in future. The report does not quite do that. Is there any intention to do that sort of analysis of the public sector in Scotland, which is really what I think the committee is looking for?

That was very helpful, and I apologise if there has been a mismatch in expectations. The short answer to your question is yes, and Morag Campsie will say a little bit more about the work that we have planned in that respect. It is also worth reminding the committee that we are revisiting some of the individual projects that are mentioned in the report, most notably the common agricultural policy futures programme, on which we will be publishing an update report in the next month or so.

Morag, do you want say something about the work that is planned for a couple of years’ time?

As part of our five-year work programme, which we have just published, we are planning to do some work in 2018-19 on the use of digital in health and central Government. We have not fully scoped that work, but it will look at, for example, the assurance framework that has just been brought in and how the Government is doing against the digital first standards and the digital strategy.

In previous discussions that we have had in the committee, we have been concerned about individual projects but also about the overall health of IT procurement. It is easy to get bogged down by one particular project that is going wrong and to dig deep into the morass. We are looking for a rather more overarching picture of how to avoid that situation in the future, and of the steps that the Government has taken and the structures that it has put in place. I would be very keen to see that.

The work that Morag Campsie has described will do that, but there is an issue of timing. A lot of the measures that have just been implemented are quite new. We have reported on those to the committee and elsewhere—in fact, you have taken evidence yourselves. The reason why the work is planned for 2018-19 is that those frameworks and that approach will have been in place for a few years by then and we will be in a much better position to provide some conclusions and judgments on the extent to which they are working.

Practically speaking, would it be too soon to do something on it now?

We could describe the arrangements as they are now, but I am not sure that we could reach many conclusions on how effective they are, as they are relatively new. The point that we make in the report is that there are many things at play that characterise the success or failure of an IT project. Procurement is a very important factor but, as we say in the report, there are lots of other things—you have seen them, as have we—that mean that those projects do not work. We are trying to pull out those factors as, we hope, a useful step for people to look at and reflect on. There will be a more traditional audit product in a couple of years’ time, when we can see how the new arrangements are working in practice.

On a practical level, what sort of expertise does Audit Scotland have internally with respect to IT projects?

We have a team of information and communication technology auditors, who spend most of their time working in individual public bodies—councils, health boards and others—looking at the IT arrangements in those bodies. When we do this kind of work and look at wider IT projects, we sometimes draw on external expertise to help our own understanding. The team here has worked quite extensively on IT projects and we have developed our expertise as we have done the work, but we recognise that we are not IT experts in the same way. We bring our audit expertise in asking the questions that need to be answered and we try to simplify what can quite often be very technical and complex situations and jargon. That is what we try to bring to it and we make sure that we get the technical expertise that we need to ensure that the judgments we make are credible.

Given that you have produced this document, what do you expect will happen with it now? Do you expect the Scottish Government to adopt it?

Yes, it has been well received by the Government and others. It is being picked up in the trade press; publications such as Computer Weekly are running stories on it, so we expect it to be picked up through that route. Locally, we expect our auditors to ensure that individual public bodies are looking at the report and reflecting on it, and we will keep an eye on how it is responded to.

So your auditors on the ground will be using it as their bible.

Yes, absolutely. One of the reasons why we wanted to do that work now is that we have things such as social security coming down the line; there will be a very significant IT requirement to run the new powers that the Parliament has for that. One of the reasons why we wanted to publish something quite quickly on good practice was to ensure that that is fed into the planning for those big, set-piece IT projects and systems. As well as looking at the document in individual bodies, there are some pretty big IT requirements coming down the line and we hope that it will help to avoid some of the pitfalls that we have experienced in the past.

We understand that Scottish Government officials have been commissioned to develop an assurance process for major IT projects or programmes of more than £5 million. How does the document fit into that assurance process?

That is one of the things that we want to look at in some detail when we come to the next piece of work, as the new assurance framework is a very important part of this. It touches on a number of the principles in this report, not least on planning, governance, leadership and strategic oversight. The assurance framework should help with some of the principles that we think need to be in place. We will keep a very close eye on the extent to which the framework is working.

Still on that theme, I understand that there is an Audit Scotland checklist that the Scottish Government uses or is putting in place. How does that fit in with the report? Is it an existing checklist and is it being amended and updated to take the report into account?

09:15  

As you might remember, a couple of years ago we produced a report on managing ICT contracts, which followed up an earlier report that looked at the Scottish Government bringing in the new assurance framework and what its role was. We appended to that report a checklist of things for senior managers and boards to consider as they were implementing an ICT project, and the new report works very much with that. It is supplementary to that and provides a bit more detail of where some of the pitfalls have been in past projects and which areas senior managers and boards really need to look at to give themselves an honest assessment of where their weaknesses are. The report works together with that.

Good morning. I find the report useful in capturing many of the issues, but it is like a second-edition publication of something that I read years ago. Aside from some of the current projects that you have mentioned, we are given the same messages again and again, year after year. Where on earth are we with things such as software project management methodologies and quality assurance processes? I have asked that kind of question before. Although the Scottish Government might have accepted the report, and although the public sector might be looking at it along with everybody else, I am interested in the step change—the what-happens-next—that delivers what we think should be delivered. Across the landscape, from your experience, who embraces formal methodologies either in the Scottish Government or in the public sector? If they are embracing those standards, why do we still get software projects wrong?

I will kick off on that question and then ask the team to come in.

That is the head-scratching thing for us, Mr Coffey. In some places, some of those methodologies are in place yet a project still does not work. One of the reasons why we tried to do something like this was to pull together what we think are the key ingredients of success. Let us look at some of the projects that you have considered recently. NHS 24 and i6 in the police used methodologies—it was not that people had not thought about doing that. In the case of i6, we reckon that the procurement process was pretty good and pretty sound, yet the project still did not work. I do not think that there is an issue with people not being aware of or not using industry-recognised approaches to these things—we see that everywhere—but there is an issue with industry approaches moving on.

You will remember that i6 used what is called the waterfall approach, which is a top-down approach that is used much less often these days. What we describe in our report as the agile approach, with smaller phases of programme, is seen as the way forward. We have tried to emphasise in our report that you cannot get just one, two or three things right but that they all have to be in place. The six examples that we draw on at the start of the report have some of the principles for success in place but lack one or two of them. The main lesson that I would draw is the absolute need to have all those things in place and to ensure that the leadership is right.

It is also important that there is a realism attached to the leadership, which must be both engaged and committed but also a little bit independent and separate. In a lot of the examples, people have been so drawn into wanting the project to work that they have lost a bit of perspective. In the report, we have tried to draw out the importance of being clear about different roles and responsibilities so that there is a mechanism for taking a step back.

The improvement framework is designed to improve the gateway process. On a number of occasions, you have heard evidence that some of the projects that have not worked have gone through a gateway process and people have been told that things have looked okay. The assurance framework is designed to have a much more robust stop-go mechanism. Sometimes, we need to be better at recognising when it is better for us to cut our losses.

That is a long answer to a short question, Mr Coffey. For me, it is about having all those things in place rather than just a few of them.

That is perfectly true. With i6, which you mentioned, and with the CAP project, is it fair to say that the errors and mistakes occurred at the front end? In my experience, if you do not get it right at the beginning, you are hardly likely to get it right at the end either. That boils down to understanding what the users or customers require, and investing time, energy and resource into getting that right. Usually, if you get that right, you get the project right.

If organisations are embracing the standards and methodologies, it is a bit of a mystery as to why they are still getting it wrong. Perhaps it is an issue with skills, experience and expertise. It is easy enough to say that you use and embrace a quality standard, but if you do not know how to use it, it will not be much good to you. Is there something that we need to learn about the basket of five principles? If we were to ask you to tell us whether they are truly embraced across the Scottish Government and public sector landscape, what kind of picture might we get?

It is a variable picture. We see that what works in one organisation will not necessarily work in another. There is not a one-size-fits-all solution. That is because of some of the softer issues such as leadership and the different cultures in organisations. You will see from the witnesses that come in front of the committee that the cultures of organisations are very different, so what works in one will not necessarily work in another. It is therefore difficult to know which principle applies more in different organisations. They all apply, but they apply in different mixes in different places.

You are right that skills and experience are very important. We have remarked on that in all our audit reports. Throughout the process we have said that, at the front end, as Mr Coffey described it, if an organisation sets off on the wrong track, it is hard to change track and pull things back. At the start, an organisation needs the skills and experience of somebody who has done that kind of thing before to say what is needed and to make an honest assessment of the skills in the organisation. If an organisation has not done something before, it is easy for it not to know what it does not have. An organisation can assume that it has the skills and experience but, halfway down the line, it might realise that it does not have what it needs. That is where sharing learning, talking to other organisations and critical friends and mentoring across the public sector are key, so that the chief executive of an organisation gets a realistic expectation of what it is like to do such a project, what skills and experience they need and how they can fill the gaps that they think they have.

Is there an extra dimension in the public sector to do with the time-critical nature of some of the projects? We have mentioned the CAP project, which was incredibly time critical. I do not know about i6, but I suppose that it was, too. The new software for the social security system in Scotland will be pretty time critical as well. I already have concerns that there will be huge pressure to get something working, which is often the first mistake that we make. We do not accept that some systems are complex and need time to be developed and to bed in. If you hurry a solution, you do not get one; you get something that you do not really want and which is no good to anyone.

Collectively, we have to keep saying those messages. We have to try to persuade those who make the decisions about procurement and development of software to really invest and give the developers time to put packages together. There is no doubt that the talent is there to write the software, but you cannot do it in a hurry and you cannot do it unless you completely understand the requirements of the customers or users. This has been said before—I feel as though I am singing an old song—but we need to make a step change to try to improve the software development process. I think that it can be done, but investment at the front end and giving developers the time to do it properly is essential. I look forward to the next piece of work that Audit Scotland will do on the issue.

Following on from Willie Coffey’s points, I think that it is important to put the issue into perspective. We are not talking about just a couple of computers in a back room somewhere—we are talking about £4 billion being spent on ICT in the past five years and a huge sum on procurement. When projects go wrong, it can have a huge impact on the public and on businesses.

As a result, the issue is important and, like Willie Coffey, I am looking for some reassurance that people are going to start getting this. There is no shortage of guidance and it sounds as though there are clear methodologies and industry standards, so is this really about taking ownership within an organisation? When people say, “Oh, we’re having IT problems,” it feels as though they are saying that the problem is happening elsewhere and that it is not really a management but a computer problem. However, everything that I have read so far suggests that it is about management and having the right skill set.

I might be repeating Willie Coffey, but are we going to see this big leap forward? I appreciate that this is not just a problem in Scotland but as far as where we are in Scotland is concerned, is there going to be a light-bulb moment?

Let us hope so.

We live in hope.

To be fair, I am not sure that it is a case of people not really trying to get it. In this piece of work, we grappled with trying to avoid the motherhood and apple pie approach—in other words, to be frank, stating the bleeding obvious. You will come to your own judgment about whether we have managed to avoid doing that.

This is about trying to pull those things out. As far as leadership is concerned, it is very clear that people are struggling a little bit in the move from a world in which the issue was all about IT to a world of digital public services; they are not the same thing. They require quite a different mindset and a different set of leadership skills and behaviours. For example, digital is about how you run your business—it is not about an IT project. IT is an important part of it, but it goes much wider than that, and it involves a big transition. You will have heard all the stuff about its being the next industrial revolution and so on, and there is definitely something in that.

Towards the end of last year, we updated the committee on NHS 24. I think that the NHS 24 experience provides an interesting wee case study; my sense is that, for quite a long time, NHS 24 was very focused on the IT project—as were we, to some extent. It was all about trying to get the IT project to work. Then a new chief executive came in, had a look, said that there were other issues at play about the organisation’s role, how it was staffed, how people were trained and what its job was and then took quite a brave decision not to commit to another date to implement the IT project because of the need to go back to first principles. In a sense, the chief executive did what Monica Lennon has just described, which was to really figure out what it was that NHS 24 was trying to achieve with this thing. It was not about how to get the IT project done. In that context, the hugely important thing for NHS 24 was out-of-hours care and everything else.

It all comes back to Mr Coffey’s point about the environment. As we have reported, the police’s i6 project was a classic case of the political—with a small “p”—environment being such that people were absolutely desperate to make the project work. In such an environment, it is quite difficult to say, “Do you know what? We’re going to press the pause button because we’re not sure that this is right.”

There is something in there for leadership and management to learn. It is about recognising that this is not about IT, but about how they run their business.

I am interested in your point about being brave and basically saying, “Maybe this isn’t the right thing.” In your report, you state:

“The political context contributed to misplaced optimism throughout the i6 programme”,

which we know has been a disaster. On page 9, you state:

“Legislative and ministerial commitments can reduce flexibility in timeframes.”

Can you perhaps flesh that out a little bit more?

As Mr Coffey has said, there is no doubt that some projects have proper, hard deadlines that are not entirely within the control of the Government or whichever organisation is working on the project. CAP is one example. That is why it is really important that we learn these lessons in advance of the social security powers coming in.

In this respect, the lesson is that if there is a hard deadline, we need to ask ourselves, “What is the minimum that we should be asking? What is the minimum that we need to deliver to make this thing work by that deadline?” Too often—and this is where optimism bias comes in—we find out what the minimum is and then say, “It would be really good if we could do this other stuff, too, and if we added in this bit of functionality. While we are at it, we could add this other stuff in as well.” Before you know it, the thing becomes even bigger and more complex than it needs to be.

That is where the need for realism comes in. People need to recognise what the deadline is and what needs to be in place for it to be met and then potentially take a more phased approach to adding things in beyond that. Too often, we tend to throw the kitchen sink at these things.

09:30  

When I look at your reports, I always try to think about the benefits to the public. We talk about transforming public services, and one example that the committee has talked about before is health and social care. All too often, IT systems are detached from each other. I know that progress is being made, but how optimistic are you that we will see properly integrated systems that allow public services to be joined up and enable people to get the best possible experience?

In making users’ experience one of our five key principles, we wanted to make that point about the importance of their being involved in a project. They are a key success factor. After all, if people do not like the system or if it does not do what they want it to do, they will not use it and it will not be judged a success. With the shift to digital public services that Fraser McKinlay has mentioned, there has been a recognition of the user’s role and how important it is to get them involved, and we are seeing that shift in culture.

In our next piece of work, we will look at the involvement of users and the difference that that makes. As I have said, we are starting to see the signs of a shift in culture with regard to the importance of users and the need for them to be involved.

Do you have any good examples of projects where the user was involved not just on the margins and in a little bit of consultation but at the design stage and all the way through the process?

In our report, we cite Registers of Scotland. Although it has had issues in the past, it is now very much focused on the user, not only internal users but solicitors and wider business users that will use the systems, and it has brought those people in to test out ideas and the software. There are examples of organisations that are doing what you have asked about.

Revenue Scotland is another example of an organisation that has learned good lessons in setting up its systems.

With regard to health and social care, which Monica Lennon mentioned, no one is underestimating the challenge in that respect. Trying to get council and national health service systems to talk to each other will be a mammoth task. As you know, we will be reporting on integration a lot over the next few years. As we have found with the police, the public service reform or transforming services bit runs ahead of the systems bit, which gets in the way of people, whether they be police officers, care workers or those who work in hospitals, trying to do the job on the ground.

Again, there is a recognition of the issue’s importance, but in the case of integration, it feels as though people are quite a long way from grappling with it. A lot of the attention has been focused on getting the integration joint boards up and running. We will be looking at the plans, how they help to join up data and systems on the ground and how they ensure that service users get the best service that they possibly can get.

When I read the key principles, particularly the principle on planning, I was amazed at how, in some ways, the report seemed to be stating the obvious with its reference to the need for the right skills throughout a project. Often those skills have been lacking; indeed, we have seen that with i6 and other projects.

Why does that requirement need to be stated? Is it to do with the culture? As an Aberdeen City councillor, I know that officials can sometimes be resistant to getting external advice, and there is always an assumption that a piece of work can be done better in-house. Is that a problem in the public sector and, if so, how do we break through that culture?

It is a key issue, and there might be a bit of that going on. However, there are other issues at play. For example, the market for good IT skills is incredibly tight and the skills are expensive. One issue is what the public sector is able to pay not just to bring the best IT people into the organisation but to cover daily rates, some of which are substantial. Understandably in the current environment, people are not desperate to do that and think carefully before doing that.

For us, the key aspect is ensuring that we have intelligent clients. Particularly when working with some of the big technology providers, who do this stuff day in, day out, the public sector client must make sure in the initial planning stage and particularly in the procurement negotiation stage that it has people on its side who can engage in those negotiations and discussions as equals. That is not an easy thing to crack, and it will almost certainly involve bringing in people from outside.

The Government is aware of the issue and is investing quite a lot in the skills agenda. It is training up digital champions; they are senior people who are not techies—to use that horrible word—but who will lead organisations in the digital age. The Government recognises the problem, but it is playing catch-up to a certain extent.

You have partly answered my second question, which was about what the Scottish Government is doing. I appreciate that it is taking the time to ensure that people are trained and given the necessary skills.

When we looked at the CAP payments process, we saw that staff morale was also an issue. It was not a good situation at all, and there was a risk of people being lost. Is the Government trying to address that and ensure that people with the right skills do not just leave? Is it working to retain them?

In our most recent overarching report, “Managing ICT contracts in central government”, we mentioned that the Scottish Government was setting up a digital transformation service in an effort to plug those skills gaps with a centralised resource of skills and people who could go around different organisations within central Government. That will help to stop staff with skills going out the door. People get great experience on a project and then they go and work in the private sector, and the digital transformation service represented an effort to keep those people. We are very interested in it, and we will report back on how it is working in our next audit report. We are interested in finding out to what extent that has taken off and is being used.

Another key principle is that of leadership. It was interesting to read that ICT projects often fail because of weak leadership. That sounds familiar from my experience as a councillor. Issues often arise with council projects, whether they be infrastructure projects or capital projects, because of a lack of project management or leadership. Those issues have been flagged up in the report, but do you have a model that you could recommend in that respect? How should that work? What should the Government be putting in place to ensure that there is strong leadership on all such projects so that they are delivered on time, on budget and appropriately?

We have not gone as far as to recommend any specific models or methodologies, because we think that it is important for individual organisations to pick an approach that is suitable for them and for the project. That is why we have taken a principles-based approach. As I have said, some of the work that the Government is doing is designed to help the most senior people in the public sector think about their leadership in a digital age. It is not necessarily about how they should lead IT projects but about what the digital age means for how they run their business. As Gemma Diamond has described, specific work is also being done on the delivery of ICT projects.

It strikes me that there might be lessons to learn from other big infrastructure projects, and there might be more that we can do on that. We seem to be managing to build a big bridge across the River Forth. It is a very different scenario, but there might some parallels with regard to leadership and programme management that people could draw on. That might be something for us to think about.

Is it the case that, as well as not having the right governance structure or reporting structure, we sometimes just do not have the people with the right skills to project manage?

There is certainly a strong body of evidence to that effect in all the reports that we have referred to. There are examples of projects down south that have had four senior responsible owners in a year, all of whom brought a different approach. It is clear that that will not work.

To come back to your point about culture, I would say that in public service—and particularly in the civil service—we expect people to be generalists. One minute, someone might be running something to do with culture, and the next minute, they will be running a big IT project. They might bring enormous skills and experience to that, but as we have said in the past—and the committee made this very clear in relation to NHS 24—there is a question whether it is reasonable to have such an expectation of our senior people in the public service, given the scale and complexity of some of the IT projects that we are looking at.

I find this line of questioning very interesting. You have talked about the lessons learned on other infrastructure projects and, looking through the report, I would say that the failures that we have seen are just failures of project management. Ross Thomson and I come from the north-east, where enormous oil-related projects on a much bigger scale than any of the projects that we have been discussing are going swimmingly. As I think Ross Thomson was getting at, if you get the right people in, you get the right project going, and leadership is an important part of that.

You have a whole section on governance in your report, but it does not go quite as far as to talk about accountability. Should that not have been addressed in the report? As you will know from our various evidence sessions, a big issue for us is that very few people seem to be held accountable for failures and the finger rarely gets pointed at one individual. Should the report not say that somebody’s head should roll if things go wrong?

That is a very fair question, and I point out that in various places in the report we talk about roles, responsibilities, governance and so on. There should always be someone who is clearly accountable for the delivery of a project. I understand your frustration when you ask people who come in front of you, “So who’s responsible or accountable for this?” and you do not get a clear answer, but the issue is sometimes not so clear-cut with a big and complex project that has lots of different things in play. In such cases, pinning responsibility on one person can be quite difficult. However, your point is well made and I will take it away and have a think about it.

We talk a little about accountability in the leadership section, recognising where it sits in an organisation, and we refer to an interesting report from South Australia that talks about the difference between responsibility and accountability. An important point for organisations to think about when they put their governance structures in place is where responsibility for delivering a project lies and, indeed, where the accountability lies, because those things can lie in different places within an organisation.

I thought that the South Australia reference was quite useful, but it might be worth pulling it into the body of the report as a principle rather than just have it as an example.

In the following page in the report, you talk about “senior officers” and “stable leadership”, and you mention—quite rightly—the “revolving door” that is often a cause of failure, but you do not go on to say how that can be prevented. When I read the report, I wondered why it did not talk about, say, performance-related pay. Instead of people coming in and being paid vast amounts of money to deliver a project, why are we not talking about having some kind of performance-related pay or golden handcuffs to stop this revolving door and deliver success?

Again, that is a very good question, but there is a balance to be struck. My guess is that the revolving door sometimes happens because things are not going very well. Indeed, there is a link between your point about accountability point and your point about the revolving door. The question is: how long should someone get in order to get a project to work? However, your point is well made. If we can recruit and attract the best people—and that raises big questions about reward and so on—what mechanisms should we put in place to ensure that they stay? A lot of folk will come to projects because professionally they will be enormously rewarding and good to work on, but given the market, there is also clearly a question about remuneration or pay. It is not really for us to comment on the model in that respect, but your questions are helpful in that they make us ask how organisations, whether they be the Government, councils or whatever, think about that topic as they begin to progress a project.

On a slightly different topic, how will projects that are already running such as the renewed CAP project and the NHS project take on the report’s principles? Clearly, they are not going to stop and—as it were—reboot.

It is a really interesting question. For projects such as the work on CAP, which is nearly finished—and we will update committee members on it in a month’s time—the conversation has moved on, but it is very important for organisations that are still in the midst of an IT project to have a look at the situation, make an honest assessment of where they are against some of the factors and ask themselves, “What is our governance structure? Is it working for us? Is it giving us the speed of decision making that we want? Have we captured all the risks, and are we acting on them?”

09:45  

Such an approach would give organisations the opportunity at board and senior management levels to take stock, assess themselves against these principles or ask challenging questions of those who are leading the projects and of project teams. They could look at where they were against some of the factors and have an open discussion about it. We hope that the work will be used in that way to give organisations the chance to have a little pause and carry out that assessment of themselves.

I want to make a very brief point on that. To say that we are concerned would be too strong, but we say in the report that good organisations will not look at such problems, say, “This is an IT thing” and just hand them over to their IT directors. There is a risk that that will happen. That conversation should be happening in corporate management teams and, indeed, at board level, which is where we would expect the issue to be looked at.

Is it your view that if the principles in the document are followed to the letter, a project will be successful? If so and if an organisation were to choose to deviate from those principles and the project were to go wrong, should heads roll for that?

The slightly glib answer to that is that we cannot give guarantees on any of those things. There is no guarantee of success in any of this. Instead, what we would say is, “There are no guarantees of success, but we are pretty sure that if you don’t have any of these things in place, it ain’t gonna work.” As I said in answer to an earlier question, this is about ensuring that the principles for success that we have set out here are present at every stage of the project. This is not a sequence of principles, but a set of principles that needs to exist at every stage of a major IT and digital project.

If some people were to explicitly choose not to do some of these things, we would ask some very hard questions about that. My sense is that people sometimes think that they have all the principles in place, but, actually, they do not. For example, they think that they are doing the things around leadership, that they have a governance structure in place and that they are engaging with users. Indeed, when we turn up in the middle of an IT project, it is very unlikely that people will tell us, ”We’re not engaging with users.” They always say that they are; quite often, though, they are not the right users or the organisation is not engaging in the right way.

A good example of that was NHS 24. A lot of its activity was around engaging with users of its system, but although that was enormously important, there was very little engagement early on with out-of-hours services and health boards, which was the whole point of the exercise. The definition of “user” is enormously important. Now that we have done this work, even if the approach seems a little like motherhood and apple pie, we and, indeed, the committee would be right to ask very tough questions if people delivering a big project had not looked at it and could not demonstrate how they had considered it and put the approach in place.

Thank you.

Willie Coffey has a question.

Fraser McKinlay made a quite crucial point when he said that public sector reforms often run ahead of the ability of systems development to deliver. That is a key message in the document. As for Liam Kerr’s question whether, if the principles were in place, a project could be guaranteed to succeed, I think that the answer is definitely no.

At the very outset of any software development project, Governments and the public sector need to engage with their IT staff as early as possible and certainly before any commitments and announcements are made about delivery of such systems. A software developer’s worst fear in the world is to be given a task and told that they have six months or a year to write something that might take much longer than that. There must be early engagement with technology staff, who have to be able to stand up to Governments and ministers and say, “That can’t be done in that time, so don’t make that announcement.” I think that that is what happens in such projects, not just in Scotland but right across the world, and we need technologists who will stand up at the beginning of a project and say that it cannot be done.

That said, we also need Governments to be able to recognise that and to ca’ canny a wee bit before making pronouncements and announcements about when systems will be delivered. If they do not do so, such projects will continue to fail to deliver on time.

We could always encourage back benchers to do likewise. Mr McKinlay, would you like to come back in on that?

I will probably not comment on that last point, convener. [Laughter.]

I am struggling to remember the example in the document that highlights the importance of policy people and IT people working closely together. That is true of an IT project, and I would say to Mr Coffey that it is equally true before a project is announced. I do not underestimate how difficult that can sometimes be, for all sorts of entirely legitimate reasons.

Coming back to the example of NHS 24, not only did people say, “We’re not implementing this system on that date”, but no other date was announced. Instead, they said, “We’ll take our time and let you know at the appropriate point, when we are surer of it.” At the time, there was a bit of backlash, with people saying, “Oh dear, you don’t know”, “The project’s rudderless” and “Why don’t you know when it will go live?” Again, however, there is a judgment to be made in such situations about whether to take a bit of flak instead of committing to a date that gets missed, then committing to a subsequent date—and on it goes. Our experience with organisations such as NHS 24 is that once an organisation gets into the spiral of naming and then missing dates, morale issues arise, and it is very hard to get out of that situation.

As there are no other questions from members, I thank our witnesses for coming along this morning and giving evidence to the committee. I will pause the proceedings briefly to allow witnesses to move back from the table.

09:51 Meeting suspended.  

09:51 On resuming—