Public Audit and Post-legislative Scrutiny Committee

Meeting date: Thursday, June 6, 2019

Official Report 516KB pdf

---

“Planning for outcomes”

Item 2 is evidence taking on Audit Scotland’s report “Planning for outcomes”. I welcome to the meeting the Auditor General for Scotland, Caroline Gardner, and Fraser McKinlay, who is the controller of audit and director of performance audit and best value at Audit Scotland. The Auditor General will not make an opening statement, so we will move straight to questions, the first of which will be asked by Alex Neil.

I will start with a general question for the Auditor General. I refer you to exhibit 1 in your report, which is on page 3. There are 11 national performance framework outcomes that the public sector and its partners work towards, the first of which is that children

“grow up loved, safe and respected”.

The third one, which is that they “are creative”, refers to “diverse cultures”.

How on earth do we measure “grow up loved”?

I should start by saying that the national performance framework and the outcomes in it are owned by the Government and that, as the committee knows very well, it is not my job to comment on policy.

My view is that it makes perfect sense for any Government to aim to set more strategic, longer-term outcomes for the services that it provides and the things that it can do, instead of focusing on inputs such as the number of nurses or police officers who are employed in public services. The Scottish Government has been notable for the ambition of the outcomes that it has set for itself over the past 12 years, since the original national performance framework was established in 2007.

Having said all that, I think that you are right that setting outcomes should not be the be-all and end-all of the national performance framework and that the Government should have a robust and consistent approach to planning for how it wants to improve those outcomes through its actions and those of other public bodies, and through the other ways in which it can influence and leverage things that will have an impact on the outcomes.

The briefing paper before the committee is called “Planning for outcomes”, because our interest is in whether we can look at what the Government does and see a clear line of sight between the outcomes that are set out in the NPF and all the things that the Government does, such as the strategies that it develops, the legislation that it puts forward and the money that it invests in the capital and running costs of public services.

The two outcomes that you pulled out are the higher-level, less tangible ones. I hope that the Government will be in a position to articulate how it is doing what it has set out; I am sure that it would point to things such as the care review for looked-after children as being part of that work. Our paper sets out the challenges with regard to making sure that the Government’s approach is more consistent and more embedded in everyday policy making, resource allocation decisions and performance reporting and monitoring.

Let us take the example of looked-after children. We can measure whether they are getting well looked after and what their destinations are in later years, so we can see whether they end up having as fair a chance in life as everyone else when it comes to the employment market. We can measure their educational outcomes and their health outcomes, but how do we measure whether they have been loved?

You are right that some things are easier to measure than others. One of the things that we say in the paper is that an outcomes approach—and, in particular, the way in which the Scottish Government is implementing that—means that it needs to be more innovative and take more risks, in a managed way, not just in the ways in which it plans to improve outcomes such as the one to which you have pointed but in how it measures them. Towards the end of the paper, we talk about the importance of asking children themselves what their experience is as part of the measurement framework. That is not the only part of the measurement framework, but it is important, particularly for the sorts of outcomes that are included in the latest national performance framework.

If you want to audit whether the Government has achieved its aims and outcomes in the national performance framework for children growing up, for example, how do you measure that? I realise that one thing that you can consider is the opinion of children, but children cannot tell you whether they are more loved or less loved than their predecessors, so we do not know whether we are measuring progress or regress. As an auditor, how do you measure whether the Scottish Government has improved love?

Our starting point is to look at whether the Scottish Government has set out how it plans to improve each outcome and how it will know whether it is succeeding. As we say in the paper, that means that it needs to have baseline information, that it needs to understand what the gaps are in that, and that it needs to go through the cycle of setting how it plans to measure what is improving or not improving, doing that, and then tweaking its plans as needed. That is its job.

The performance framework is now 10 or 11 years old. Surely the Scottish Government has defined in the past 10 or 11 years how it measures whether it is achieving greater love.

I think that that particular outcome is a newer one that was added in the latest version of the national performance framework, which, as you know, is now set out in legislation. More broadly, as we say in the paper, we have seen some examples of good practice but, equally, we think that there is room for that to be much more embedded in policy making and performance reporting and that, although there are examples of good practice, it is not consistent.

Do you agree that, if there is meaningless guff like this, it will destroy the credibility of any exercise before we start?

I completely agree that the value of having a national performance framework is not in setting outcomes but in doing the underlying work, which is to do with planning for how you expect to improve the outcomes and then monitoring whether that is happening. That is definitely more challenging for some outcomes than for others.

Alex Neil has had a good chance at that issue, so I will bring in Willie Coffey.

I want to continue on the same theme. Who, if anyone, is doing any work to ensure that targets or outcomes are meaningful for us? We have heard that they can be pretty aspirational or very prescriptive. There can be specific targets—it can be demanded that something be reduced by 1,000 by such a time, for example. Who is doing work to ensure that the targets that we have given ourselves collectively are achieving anything for us and achieving the outcomes that we seek? Is any work being done to review and refine that process? Are you the person to do that, or should the Government do it?

I think that it is for the Government to do that. Fraser McKinlay has been involved in work that is under way. I ask him to pick up that question.

A lot of work is currently going on. “Scotland’s Wellbeing—Delivering the National Outcomes”, which was published last week, is the first report of its kind to pull together a broad sweep of progress that is being made against the national performance framework and the outcomes, as opposed to progress on individual outcomes. I take Mr Neil’s point that the framework has been around for some time in various iterations. For the latest iteration, which was launched last June, a lot of work went into looking at not just the outcomes but the 81 measures that sit underneath them to ensure that they are meaningful and tell us something.

To pick up Mr Neil’s point, there is no doubt that some of this stuff is difficult to measure and to audit. It takes us into the territory of hearing stories and personal testimonies from people, and recognising them as legitimate evidence both for measuring progress and for us as auditors.

We have dipped our toe in that water. For example, we did a report on self-directed support a couple of years ago. A lot of that report was based on going out and talking to people who received services in their own homes and getting a sense of how that was going. That in itself is not enough for us; to use the jargon, we always need to triangulate the evidence with more quantitative stuff.

However, we must bear in mind that the latest iteration is a joint Government and Convention of Scottish Local Authorities enterprise and that a lot of work is going on to ensure that, underpinning the 11 outcomes, which are, in a sense, a statement of intent and aspiration, as has been said, we can have some sense of progress. As Caroline Gardner said, our challenge to the Government, councils and all partners is that we still too often find good and strong statements of intent and vision but not enough of a plan for how they will be delivered.

I am sure that we could come up with a few examples of targets not being met. Does anyone look at whether the outcome was positive or negative for people when we did not meet a target or did not deliver something on time? That is what I mean by asking whether we review what the targets mean. If there is no negative outcome for people when a target is not met, why do we have the target? Do pieces of work include such a review process for outcomes and target setting? That would involve looking back at whether targets were appropriate.

The question is interesting. When things do not go as planned or when targets are missed, that is mostly reviewed. However, to come to your point, the accountability systems are difficult, as our briefing says. Almost by definition, such things need to be delivered by multiple organisations and multiple people. I genuinely think that saying that one body is solely responsible for delivering a target or outcome is hard these days. That can be frustrating for the public and for MSPs in your scrutiny work, because it can be difficult to point to a single person who is accountable for delivering something.

Our accountability systems need to catch up with an outcomes-based approach, because it is unacceptable to say that nobody is responsible and that, if something has not worked and we have missed a target, that is a systemic issue. We need to be sharper than that.

There is a tension, because I see the opposite effect when accountability systems drive people in public services to do things that are not necessarily in the best interests of outcomes but which are in their organisations’ best interests. We need to surface and grapple with that tension.

That is helpful.

I will draw an example of the point that Mr Coffey raised. When we looked at mental health services, Anas Sarwar asked whether somebody had counted the number of, and whether there was evidence of, suicides by people who were on a waiting list for mental health services. Such an example goes to the heart of Mr Coffey’s question. Are there terrible outcomes for people for whom the Government’s targets have not been met? The committee has touched on that in evidence sessions.

The Auditor General’s briefing mentions the importance of meaningful engagement with public sector workers. Will you provide examples of best practice in that? I refer members to my registered interest as a member of Unison.

We see examples of that on a small scale in most of the performance audit work that we do. In almost all public services, the quality of the service that is provided depends on the quality of the interaction between the person who receives the service and the person who provides it. Repeatedly, we see good examples of public sector workers of all sorts being encouraged to listen to what is important to the people they provide services to and to look at meeting their needs.

A couple of years ago, we published a report on self-directed support, which found good examples of people sitting down with a person with disabilities, listening to what would make a difference to that person’s life and thinking about how they could use the money and other resources that were at their disposal to provide tailored support to give that person the best outcomes and help them to develop their independence. However, we also saw examples of people not being encouraged or trained to take such an approach and of people not having the flexibility to do more than provide the services that the council already had available.

For the groups of people who were affected, the approaches were as different as chalk and cheese. A key recommendation in that report was that councils needed to understand better the approach that was being taken locally and to ensure that people were trained and supported and had the resources to do what was needed and to listen to the voices of people who were affected. However, we often see it from the other end, from third sector organisations that we talk to as part of our work. We do a lot of that engagement ourselves.

09:15  

For example, a leadership team from Audit Scotland spent some time talking to people from the Wheatley Group about the way in which its housing service takes a think yes approach, in which it looks to meet the needs of people locally. Where that can be done through the flexibility of the housing service itself or through things that it can easily put in place, it can work very well. We were told of an example of a gentleman with severe hearing problems who had the television turned up loud enough to disturb his neighbours. The housing officer was able to simply buy him a pair of wireless headphones, which meant that he could hear his TV and his neighbours were not bothered.

Equally, we heard examples of where what was needed was some involvement from the health service or from social care services. In some circumstances, it was much harder for the housing officer to engage local public services in asking what the person’s needs were and how they could best be met. Consistency, flexibility and innovation are needed in those situations to make a reality of self-directed support, which takes us back to the culture and the ways of working that we talk about in the briefing, and the sorts of evaluation that Fraser McKinlay was talking about—listening to people and seeing whether what matters to them is being delivered.

Nobody is saying this is easy. It is clearly complex, but those are the things that can make a difference.

Those are some excellent examples. Although it might be simplistic for me to say that people on the front line sometimes know best, that is a truism across Europe. The general criticism that I sometimes find at surgeries and through talking to the public is that politicians come out with policies but they are not really talking to people on the front line. Without making a party-political point, which I would of course not do, issues such as the workplace parking levy have caused some controversy among public sector workers—I am a member of Unison, as you know. Auditor General, I know that you cannot talk about the policy per se, but is that the kind of example in which talking to front-line workers in the public sector, or indeed the private sector, before developing policy would be quite helpful?

Without getting into the specifics of the workplace parking levy proposals, Governments generally tend to come up with better policies by talking to the people who will be affected by them, and doing so consistently, not as a one-off consultation. The Scottish Government has made some serious commitments to that through its membership of the Open Government Partnership, its community empowerment legislation and its approach to public service reform, which is built on participation as one of the four pillars that go back to the Christie report of 2011. It is an example of where it is not the aspiration that is in question but the consistency of following it through.

There are some examples in other countries, such as America, of holding town hall forums to discuss policies. Michael Russell is looking at the idea of citizens forums and so on. Is that a development that you have picked up from other European countries that we could look at more carefully in Scotland?

As you say, the Government is investigating lots of that. Fraser McKinlay can probably tell you about how we see that from the work that he has been leading.

We see that in many different guises. The whole citizens assembly idea is one version of that, and we see lots of good community engagement locally.

The challenge is getting to that engagement much earlier in the process. As you know, I do a lot of work in councils. Many councils are very good at engaging with communities and service users when they have an idea of what they want a service to look like. The challenge is in involving people much earlier in the process, and I include staff in that. Before decisions are made and proposals are put out for consultation, there needs to be an earlier conversation about what is important and what matters to the community.

In my experience, when organisations do that, they are often surprised that it is not necessarily the thing that they thought was going to be the answer. I therefore take your point about the answers often lying as close as possible to the front line of service delivery. In the end, all outcomes are local and individual, to some extent. That is where the whole community empowerment and engagement agenda is headed and the next challenge is to do that much earlier in the process.

Auditor General, I would like to link money and performance. In your report “The 2017/18 audit of the Scottish Government Consolidated Accounts”, you recommend that

“The Scottish Government should prepare a performance report that clearly links to the financial resources outlined in the Consolidated Accounts.”

Your briefing paper discusses that point, saying:

“Scottish budgets should be clear about how spending ... is expected to contribute towards ... specific national outcomes”.

We have talked about some of the difficulties in defining the national outcomes.

You may recall that, when the committee raised the issue with the Scottish Government as part of its scrutiny of the consolidated accounts, the Government said that it would include brief material in the consolidated accounts and signpost individuals to more detailed sources of information. What is your view on that proposed approach?

You will be aware that all of this work is rooted in the recommendations of the budget process review group. It is recognised that making a reality of the national outcomes means investing money and other resources and that we need to be able to track how that is doing in order either to continue investing or to reinvest somewhere else if the investment is not having the desired effect.

In Audit Scotland, we are clear—in line with the budget process review group recommendation—that there needs to be a much clearer link at both the budget end of the cycle and the financial reporting end of the cycle with performance on the outcomes that the Government plans to achieve and how well it is doing in practice. We are also clear that that is not a simple thing to do. It is complicated for a number of reasons, which we outline in the briefing.

I would be relaxed about the Government, in its financial report, signposting information that is held somewhere else. It is always a challenge to make sure that financial reports are accessible to readers, that they are not overburdened with detail and that the high-level messages are clear and apparent to them. My concern would be to ensure that the signposting was clear—that is easier to do these days, with technology allowing us to make direct links to different sources of information—and that the information that it was linking to was fair, balanced and rounded.

We give an example in the briefing paper of where the Government’s reporting of its performance can sometimes highlight the positives without highlighting the things that are not working so well. That has the effect of limiting parliamentary and public scrutiny and makes it harder to stop doing the things that are not working and reinvest in things that are more likely to have the desired effect on outcomes.

The approach sounds fine to me. The issue is to do with the detail of how it is implemented to make it accessible and ensure that it gives the rounded picture that is needed.

Do you think that the approach will be sufficiently transparent on how spending is directly leading to improved outcomes?

I think that it will be, as long as the other parts of the package are in place. If the budget is clear about which outcomes financial investment is intended to improve and how that is expected to happen, it will be straightforward at the end of the financial reporting cycle to link back to that and say, “This is what we expected to do and this is how it worked out in practice.” The Government needs to make that link across the budget cycle. As I said, the information needs to be rounded rather than selective. The Government should not simply pull out the things that are working well; it needs to show the whole picture.

Is there not—

I think that Fraser McKinlay wants to come in on that.

I just want make a couple of points, if I may. The budget process review group said that to identify specific budgets for each of the 11 outcomes was a difficult and probably not terribly helpful task, because of their interrelated nature. I absolutely sign up to that, having been involved some years ago in work that tried to do that. However, the convener mentioned the “Children and young people’s mental health” report, for example, and we do not know how much money is spent on that area. It seems to me that that is not good enough.

It seems to me—and I think that this is the direction of travel—that rather than trying to do an abstract exercise to identify the budget and the outcomes in an area such as children and young people’s mental health, it would be more practical to figure out the collective resource that is being applied. We could pick any of the topics that we have reported on over the past few years. I think that we had a similar discussion about how we just do not know how much money is being invested in economic growth.

The report that was published last week on the delivery of the national outcomes is a good report and it gives a good picture of progress, showing the places where we are doing well and the places where we are not doing so well, but money does not get a mention anywhere in it. We need to get better at joining the two things up.

As Caroline Gardner said, there are many different ways of doing that, but it will be difficult to set all of that out on a single bit of paper. That seems like a big gap to me, not least because, apart from anything else, it is quite hard to make a judgment about where we should be investing more money. Part of the conversation has to be about shifting resource from one place to another in order to make a bigger difference to the delivery of outcomes.

I was going to come back to the previous comments about the complexity of all this and the number of different elements that go into achieving a particular outcome. Is it possible in all cases to be able to identify the stream of money that goes into a particular service to achieve a particular outcome, when there are so many bodies looking at different aspects and putting money into different areas? If you try to figure out where every pound is going, do you not reach the point where you just tie yourself up in knots?

Yes, you get to that point eventually, but my point is that we are still miles away from starting that process. There is a place in the middle between not really knowing anything and spending for ever trying to tie down where every pound and penny came from and who is responsible for it. We do not expect perfection in any of this, but, as we say in the briefing paper, the money is a means to an end. The question is how we allow Government and its partners to make decisions about where the money should be invested, and in the absence of a decent understanding of how much we are spending on things in the first place, that seems to me to be very hard to do.

From the work that Audit Scotland has done, should it be possible for the Government to identify at least the broad stream of money that is going into an area and to measure the outcomes directly against that? There will be anomalies and gaps where other entities are putting in money, so the process will always be flawed, but what we are looking for is just the general thrust. Is that, in crude terms, correct?

In short, yes. Indeed, we quite often find that that is what we are doing in our reports. In the absence of available data on spend, we will do some analysis. That analysis will, of course, be based on assumptions, but as long as those assumptions are reasonable, you will get something that you can work with.

As Fraser McKinlay has just touched on the point that I wanted to ask about, I will ask my questions now.

The committee has looked at the issue of data in relation to a number of the Auditor General’s reports; indeed, I raised the matter with the First Minister at the Conveners Group meeting with her just a couple of weeks ago. The starkest example of this is, as Fraser McKinlay has said, the children’s mental health report. We discovered that we had no idea how much in total was being spent on children’s mental health; there was no data specifically on the outcomes; and there was no data on why referrals to child and adolescent mental health services were being rejected. Moreover, even where data existed, it did not seem to be consistent across the 14 health boards or shared in any way.

We also discovered a lack of data on self-directed support and, very worryingly, Audit Scotland told us with regard to the early years and childcare report that, despite the huge amounts of public money going into the increase in funded childcare hours, there was no business case and no sense from the Government of the outcomes that were expected to be achieved.

Those are just three examples, but is the situation even more widespread? Now that we are living in the era of big data and evidence-based decision making, can you give us a sense of where the Scottish Government is in making evidence-based decisions with our money?

Those questions are really important and very timely, given that, as you have said, the approach to information is moving away from the need to collect it as a separate thing to its being generated by the things that we do in our everyday lives. You can see that in, for example, the Lothian Buses tracker, which knows where your bus is and when it is going to arrive. However, we do not seem to be using that very approach very much in public services.

09:30  

I would pull out another very important example in addition to the handful that you have already given us. I have reported a number of times on the slow progress that is being made with the Government’s 2020 vision for health and social care, which is about helping people to spend much more of their lives well cared for and happy at home rather than relying on acute hospitals for their care. Most of the data that we have on health and care is still about acute hospitals, and most of the political focus is on what happens to the amount that we spend on the national health service and on waiting times for acute services. However, there are big gaps in some of the basics around health and care in the community.

We are just finalising our work on primary care workforce planning, and we have seen that the Government does not have good information on the number of general practitioners around the country and the amount of care that they can provide. Without knowing that, it is hard to genuinely plan for how we avoid a situation in which hospital is the first resort rather than the last resort for people who are just about coping in the community. Those gaps relate to things that ought to be relatively easy to capture. We ought to have better information about how long people wait for GP appointments and the extent to which primary care practices can prevent people from being admitted to hospital unnecessarily or get them back home again quickly.

As part of making a reality of the outcomes approach, there is an important question for Government about how it uses data and digital capability in exactly the way that you describe to get more nuanced real-time data about what is happening across the country rather than rely on the old approach of quarterly data collection and publication. We can move well beyond that now, but that is not yet built into the outcomes approach well enough.

Your report says that the Edinburgh parallel computing centre and the Scottish administrative data research partnership are doing some work to see whether we can match data with the performance framework outcomes. How far advanced is that work?

Fraser McKinlay can say a bit more about that. It is worth noting that a big part of the Edinburgh city region deal is about making it a digitally enabled area. Work is under way.

If truth be told, I am not sure that I can tell you much more about it, but we can follow up on that in writing if that would be helpful. There are other examples of such work—the University of Glasgow’s urban big data centre does similar work.

As well as the gaps in data and understanding in what are pretty obvious places, there is an issue of the data not being terribly well joined up. It is about people having access to different sources of data and making sense of that at a level that means that they can really make a difference in terms of the outcomes.

This perhaps sounds a bit techie, but I think that it is important. In 20 years, we will look back and think that we were just putting a finger in the air to decide how much money is spent on certain initiatives in the public sector, without having evidence to back that up. There is a front-page article in The Herald this morning on exactly that issue. The article says that the European Commission has withdrawn money from public services in Scotland and that

“Skills Development Scotland do not have rigorous enough controls over how the money is spent and what is achieved with it.”

That goes right to the heart of the issue. The Commission has obviously decided that there is no evidence of outcomes from the money that is being spent in Scotland, so it has just taken it away. Will you address that point?

The committee will be aware that we have reported on problems with the management of European Union funds, including structural funds and agricultural funds, in recent reports on the Scottish Government. We will pick up on the detail of that again in a report to the committee that is due in the autumn.

In general, you are absolutely right that, in the chain of money from Europe through to the Scottish Government and on to the bodies that spend it, accountability is important. As with all public spending, the money is there for a purpose. I would like to pull back the frame and say that we are not talking only about funding that comes from Europe for particular purposes or new announcements from Government on money that is focused on reducing waiting times, for example; we are talking about a budget of about £42 billion a year, and the way in which all that money is spent is crucial to whether or not outcomes are improved. Fraser McKinlay and I are saying that we would like a much clearer line of sight on how the money is intended to improve outcomes. Obviously, we need data on whether that is working so that we can fine tune and make adjustments as needed.

It is maybe a bit of a wake-up call for us, in relation to our other spending, that the European Commission has decided that, because Scotland is so poor at evidencing how the money contributes to outcomes, it will withdraw the money altogether.

In principle, you are right, but the picture is a bit more nuanced than the one that you have described. The funding is “in suspension” and “in interruption”, which are technical terms that the EU uses, until it receives the assurance that it needs. However, in general terms, to make the national performance framework a reality, the Government needs to know what it intends to achieve with the funding that it provides and whether that is being achieved.

My questions relate to the same point. Despite the commitment to an outcomes-based approach, your report seems to suggest that performance in certain areas of the public sector is still measured on inputs an awful lot rather than on outputs and outcomes. Politically, will that change?

That is a difficult question to ask us. You are right to say that, in some ways, the problem is political—with both a small p and a big P. However, the Government’s approach is the right one. In response to Mr Neil’s first question, I said that whatever we think of the individual outcomes in the national performance framework, it has to be right for Governments to focus on how they intend to tackle the big challenges that their countries and societies face. Such challenges tend to be long term and require a joined-up strategic approach. There is widespread consensus that the approach is right, but, as we say in the paper, there are real challenges to making that a reality. In some ways, making the commitment is the easy step; the hard step is doing the difficult long-term work of improving the things that the Government has set out as its aspirations.

One such challenge will no doubt be politics. If I sat down with every member who is sitting round the table, there would be broad agreement on most of the outcomes in the framework in relation to people being healthy, active and well educated. On a Thursday lunch time at First Minister’s question time, the challenge is to come back to the short-term things that are not working well, which could be the case under any Government of any complexion, rather than focus on progress towards the longer-term things. It would be better for everyone involved if there was a clearer line of sight and direction of travel between where we are today and where the outcomes are. That would make it easier to focus on the long-term changes that the Government is trying to make for the country. Equally, there is a role for politicians in asking themselves where the balance lies between today’s political advantage and the longer-term changes that individuals and parties want to see.

Thank you for that. I appreciate that it is quite difficult for you to answer that, but people who are watching this meeting will be asking themselves the same question. In paragraph 12, you talk in similar terms by saying that

“long-term outcomes may bring difficult decisions into sharper focus”.

You give the example of health outcomes being affected by the move to a “community setting”. Bluntly, that could hurt in the first instance, and some big and brave decisions will need to be made before outcomes improve. The question that will be on people’s minds is whether it is realistic to think that such decisions will be made, given the political environment and political cycle in which we find ourselves. Do you have any thoughts on that matter that you can share with us?

I do, and I am sure that Fraser McKinlay will be able to share his view in a moment. There are two things. I understand why the issue is so difficult. On the morning of one of my reports on the NHS being published, I will come out of a BBC studio, having talked about the need for a shift in care from hospitals into the community. An Opposition spokesperson who will be going in after me will say to me, quite frankly, “We know you’re right, but we have to criticise moves to downsize hospitals or close wards.” I understand why the situation is difficult, but doing that makes the shift harder to achieve. You know that as well as I do. However, saying that that means that the situation will never change is a counsel of despair.

In my view, it is right to take a longer-term view of what Government and public services are for. One step in doing that is to set out the outcomes that the Government wants to achieve. That work will be helped along by putting more rigour into planning how the Government wants to improve outcomes and be accountable for what is working and—just as important—what is not.

There will be political knockabout—we all know that this is a rough old trade. To an extent, politics is about making such disagreements public and being accountable to the public. I think that having in mind that longer-term picture has to be one of the things that makes that more possible.

The stuff that we say in the briefing about engagement with communities is critical, because that is where the conversations really need to happen. That is not about Government, health boards or councils just going out and doing a sales job; it is about having an honest conversation with people in local communities to explain the rationale and the evidence base for a decision and, to return to David Stewart’s point, what the alternatives might be. Too often, that does not happen and the first that communities hear about something is after a decision has been made. If the nature of the conversation with people is that they are just to be told that, for example, their leisure centre will close, of course they will be up in arms. The conversation has to be heard much further upstream, to explain what the challenges and alternatives are. In that context, difficult decisions might be more palatable, although we recognise that none of it is easy.

The principles that apply when making big decisions in local places are the same as those that apply in Parliament. For example, by the end of this parliamentary session, about 50 per cent of the Scottish budget will be spent on the NHS. At what point will we, as a country, think that that level of spend is still okay or not okay? If we continue on that trajectory, we will need to talk about the impact that that might have on other areas. Obviously, I offer no policy view about that one way or the other, but, as we say in our briefing, those are the kind of big discussions and decisions that Government and, ultimately, Parliament need to have and make.

At the start of the meeting, Alex Neil mentioned the first national performance framework outcome, which is that children

“grow up loved, safe and respected so that they realise their full potential”.

I think that, conventionally speaking, I have stopped growing up. That probably happened quite a long time ago, as it did for a large proportion of the population, so it may be that a lot of us fall out of that outcome—I do not know. How does the Government know whether I have realised my full potential? How would you audit that?

You are still loved, Bill—by someone. [Laughter.]

How do we know that Bill Bowman has reached his full potential, Auditor General?

I will resist the temptation to personalise the question and talk instead about the outcome in the NPF as it is stated. I recognise the challenge that you and Alex Neil are putting to us. To a great extent, it is a challenge to Government rather than to us. It is for any Government to set its own policy. I think that it is a good thing that the Scottish Government is doing that in a longer-term way that is focused on outcomes. What flows from that is a requirement that it can say how it will measure whether the outcomes are being achieved and, upstream, how it intends to achieve that.

I will turn the question around. From this morning’s discussion, you can perhaps see clearly where people have not met the outcomes, and we allocate resource to try to help those people. There could be a large amount of unrealised potential that we are missing out on by focusing on more obvious aspects. Do you have any suggestions about how we might tap into that?

We say in the briefing—this is a truism—that there may be trade-offs between outcomes. If a decision is made to prioritise one set of things in society, others will, by definition, be less of a priority. To use an example from our briefing, there may be a trade-off between economic growth and sustainable development, including the environmental impact of emissions. We suggest that there needs to be more surfacing of the trade-offs and more clarity about how what takes priority is being pursued in terms of the investment of money and time and how progress will be tracked.

The particulars in the briefing are clearly a matter of Government policy, rather than something that we can defend for you—that is not our job. However, if those are the policies that the Government has set, we can say what we think needs to be in place in order for this committee and the Parliament to be able to do its job.

Last week’s report on delivering the national outcomes has a series of indicators to do with realising people’s full potential. None of those in itself will be able to tell you whether an individual has fulfilled his or her potential, but when all those indicators—a lot of which are about inequalities regarding where you live, your socioeconomic circumstances, whether you are a boy or a girl or a man or woman and whether you are disabled—are taken together, they can give you a picture of whether, broadly speaking, people have the opportunity to fulfil their potential.

I accept that there is a wee bit of a leap of faith in all this, but if the inequality gaps across all those measures are being narrowed, it can be argued that more people are more likely to be realising their full potential.

09:45  

I think that I ask this question every time that we meet, Auditor General. The performance improvement model in exhibit 3 in the briefing shows a cyclic process for continual improvement. For me, the key part of the process is the review, or the study part after something has been done, to see whether we have done it well, met targets and objectives and so on. Is that review being done consistently across the public sector? Is there enough evidence to support whether that is happening? Does a bit more need to be done to require the public sector to demonstrate that it does that?

I agree with you. The model is quite a simple but powerful way of taking the next step of agreeing the national performance framework, which is now in statute. In the briefing, we give a couple of examples of where the model has been used well. The most notable example is probably the NHS patient safety programme, where advances in important things that affect and protect people’s lives are being achieved by disciplined application of the model.

We also say that the model has not been consistently applied. The convener has highlighted two or three of our reports that identified instances where being clear about the scale of a problem and how progress would be measured were not thought about at the beginning. If those things are not done, it is impossible to know what effect you are having and therefore whether you should do more of the same or try something else instead. We want to see consistency and rigour in the use of the model.

Should the review part of the cyclic process include what I spoke about earlier, which is thinking about whether the targets, objectives and outcomes that we set ourselves are appropriate?

Yes is the short answer.

Does anybody do that?

Timing is part of the issue, because some of the outcomes will be intergenerational—they could take 30, 40 or 50 years. A risk in the discussion about outcomes is the general sense that we will wait for 30 years to see whether something has worked. That clearly would not work—it certainly would not work for us as auditors. The cycle need not work exactly in real time, but we do not see enough of, in particular, organisations reviewing as they go along whether the intended objectives are being met. Again, that is sometimes where the politics comes into play, if I am honest. Sometimes a review will require people to say, “We thought that was what we were doing, but actually we’re going to shift that a little bit”, but having that conversation in such a way that it is not defined as a U-turn or a failure is quite tricky.

The model in exhibit 3 is effectively step three of the three-step model that we described in paragraph 19 of our briefing. Equally important are steps one and two. The first step is setting out clearly what you are trying to achieve. The second step is creating the conditions for that, to make sure that everything is in the right place. The third step is about operationalising improvement. As Caroline Gardner said, I think that our assessment is that that is not done consistently enough.

I thank you both very much for your evidence this morning.

09:48 Meeting continued in private until 11:25.