Good morning and welcome to the 24th meeting of the Finance Committee in 2014. I remind everyone to turn off any mobile phones, tablets or other electronic devices. We have received no apologies, although Gavin Brown is not yet here. I hope that he will turn up soon.
Agenda item 1 is evidence on the Scottish Government’s draft budget for 2015-16. The session will focus on outcomes and performance budgeting. I welcome Colin Mair, from the Improvement Service; Gareth Davies, from the Chartered Institute of Public Finance and Accountancy; and Fraser McKinlay, from Audit Scotland. Members have copies of the submissions that were received before the meeting.
We will move straight to questions from the committee. [Interruption.] Gavin Brown has just joined us.
I apologise for being late, convener.
As the witnesses probably know, I usually start with a few questions before opening out the session to colleagues, and I propose to do that today.
Thank you for the submissions. We will start with Colin Mair, whose submission is at annex B to our paper 1. Paragraph 2 of your submission is headed “Profusion and Prolixity”.
And pomposity, perhaps.
The word “prolixity” does not come up often.
Paragraph 2(i) says:
“Bluntly, there is a huge amount of”
frameworks to support and look at outcomes
“and we seem to have proceeded by accretion without deletion ... the newer focus on outcomes seems often to have been retrofitted to service or sectoral performance frameworks”.
I sense an element of frustration. Do you feel that, although more and more people are coming up with ideas for looking at outcomes, those ideas are never rationalised into something that can be delivered more effectively?
The next paragraph says:
“It is often unclear who the end user of such frameworks is intended to be, i.e. who they are for, and what they are for.”
Will you expand on your thoughts for the committee?
Thank you for the questions. We reviewed all the performance frameworks that we can identify at the national level, and we have moved on to identifying all the local ones. We will pull all that together in a final report on what we are doing on performance.
I appended two or three overhead shots to my submission to summarise some of the mapping work that we have done. Members will see that a staggering amount is going on. There is a huge number of overlapping performance frameworks.
What I said was less a critique of anybody and more a point about frameworks having evolved at different points for different purposes, after which we have added new purposes and fitted on other things. The most recent arrival on the scene has been an outcome focus, but we have often posed the question the wrong way round. We have asked what the outcomes of services are as opposed to asking what outcomes we want for the Scottish population and how services contribute to them. We have begun to create silos around outcomes, even though the idea of an outcome focus was to help us to break out of the silos that we started from.
A lot of any performance framework in a major public service is made up of indicators on what is needed to run that service well. There is a lot of political and public interest in running public services competently. Are we meeting service standards that we have committed to? Are we using the available resources as efficiently as we can? An awful lot of the performance armoury is focused on that level. There is nothing wrong with that, as public services are massive businesses that need to be run competently.
We sense that, of all the audiences for performance measurement, the one that wants the holistic overview—as the committee seems to—is still the weakest voice. The sectoral or service voice remains much stronger and the business management voice remains very strong in how we go about performance measurement and management. In Parliament, although the Finance Committee wants an integrated and holistic view of outcomes, I suspect that other committees are interested in distinctive indicators that relate to health, education and children, for example.
In a way, we all want to come together but, at the same time, we all want to look in particular lines of sight. Therefore, in performance management, we get a kind of compromise between those things.
The Community Empowerment (Scotland) Bill, which is being scrutinised elsewhere in Parliament, will place a common duty on public bodies to work together to improve outcomes. Once that duty is in place in law, it will introduce a dynamic behind reaching a collective view of the outcomes that we are trying to improve and the measures that we will use to show that we have actually improved the outcomes.
As I say, I did not particularly intend the first part of my submission to be a critique. It is simply an honest recognition of where we are. It is a recognition that the business management interest properly remains powerful in public services, and that those who want an holistic view of what we are achieving overall for the Scottish population are, frankly, probably not the strongest voice in performance management at the moment.
You go on to say in paragraph 2(iii) that
“outcomes are often defined in service terms”—
as you have just touched on—
“rather than independently in terms of the life chances, life outcomes, and quality of life of people and communities.”
Is the issue that sometimes we cannot see the wood for the trees in relation to outcomes?
There is a very natural service orientation. If someone is running, say, a health and social care partnership, it is tempting for them to say, “What outcomes are ours?” and then build them into their performance framework separately from everyone else and operate to that. The trouble is that health and care outcomes are massively influenced by a wide range of social and economic factors. They are not solely influenced by the organisation of health and care services, and therefore we begin to try to narrow the thing down. If we are serious about improving life for people in Scotland and particularly those who suffer the greatest inequalities, we need a much better integrated overall focus.
Community planning is in part supposed to provide that, but we have not quite reached the point at which the collective discipline that community planning could bring is being fully brought to bear. As you will know, Audit Scotland has done an interesting audit of community planning partnerships, and one issue that it found was about how to pull everything together with a simple set of outcomes for the local population and then show that we are moving forward to achieve that.
Do the other witnesses wish to comment on the issue?
Yes—just briefly, convener.
I agree with everything that Colin Mair said. Audit Scotland’s view is that the outcomes approach is a good thing and that we should not lose sight of that. It is difficult to imagine a world without it. Certainly, at the most senior levels in the public sector, it has become the way that public services are talked about and viewed.
Our observation, based on all the audit work that we have done on community planning, policy areas and individual bodies, is that the national performance framework and everything that is set out around it are the tip of an iceberg, and the rest of the iceberg is not quite in place to support that outcomes-based approach.
Obviously, we have a close interest in the money. Colin Mair mentioned the audit work that we are doing on community planning partnerships. The Accounts Commission for Scotland and the Auditor General for Scotland will publish another national community planning report later this year. Community planning partnerships are still in the early stages of figuring out how to use their combined money, people and assets to progress and to deliver better outcomes for the community. If I was to pick one thing that really needs to be pushed forward and that we need to crack if we are to make a real difference to the outcomes-based approach, that would be it.
I echo and agree with the comments that have been made so far.
From my perspective and understanding, outcomes are a consequence or result of action or, arguably, in some cases inaction. I would tend to put the matter in the framework of governance. One of the key principles in the document “International Framework: Good Governance in the Public Sector” is:
“Defining outcomes in terms of sustainable economic, social, and environmental benefits”.
Therefore, to my mind, an outcomes-focused organisation should be doing outcome budgeting. Outcome budgeting could be seen as evidence that an organisation has an outcomes focus. I would tend to tie this to a large extent into the overall governance of an organisation or of public money generally.
I agree totally with the point that it is the holistic impact on society as a whole that matters. Outcomes do not affect just the individuals or service recipients. The issue is about organisations having an awareness of the consequence of their actions on all stakeholders, whether that is other public sector bodies, third-party suppliers such as voluntary sector bodies or employees. It is about awareness of the consequences of their actions as a whole and how they reverberate through society.
Paragraph 7 of the Audit Scotland written submission states that
“there is scope for the Scottish Government to demonstrate a more systematic approach to implementing the outcomes approach.”
It then refers to modern apprenticeships. Can you expand on how you think that that approach can be delivered?
Yes. The example of modern apprenticeships is a good one because it is where we see some tensions between the policy that is set out and outcomes. The national performance framework is very outcome focused. Once we get into the nitty-gritty of the measurement of the delivery of the outcomes, the challenge is that a lot of the indicators that are used are not outcome based.
We use modern apprenticeships as an example because the headline target is 25,000 new modern apprenticeships. That is a good thing. We are not saying that it is a wrong thing and we are not even saying that it should not be a target or objective. However, what it does not do is measure the outcome of what those 25,000 new modern apprentices are going to do for their communities and for the economy as a whole.
We see a disconnect there and in other places where we grapple with an overall outcomes-based approach that tends to be longer term, a bit more diffuse and, if we are honest, a bit more difficult to explain in political terms; and with targets that tend to talk about numbers of things, whether that is teachers, police officers or whatever. There is a real tension between the outcomes-based approach and the measurement and system of performance management and information that is, as Gareth Davies said, designed to support it.
As you can imagine, convener, we have constructive and robust discussions with the Scottish Government as we go through our audit process. Our approach comes from quite a simple place really. We have recently published reports for the policy areas of self-directed support and reshaping care for older people, which have long-term—10 years-plus—policy outcomes. We looked at them relatively early on, three years in. We got a bit of a challenge back from the Scottish Government that we were looking at the situation too early. However, our question is a simple one: “How do you know?” How do you know, in a long-term, outcomes-based approach, that you are making the right progress in the things that you are doing and that the money that you are spending is making the difference that you need it to do?
In summary, that is where we believe that the focus needs to be now in order to make the outcomes-based approach real and more meaningful in a very practical sense.
Do you want to add to that, Colin?
Yes. I endorse what Fraser McKinlay has said. I think that we are often stuck with measures, even when they look like outcome measures.
In our written submission, we alluded to how we measure children’s educational attainment and inequalities in educational attainment, which is the secondary 4 and 5 tariff score. Those scoring systems were produced by the Universities and Colleges Admissions Service—UCAS—and they are an utterly selective understanding of educational achievement and accomplishment; they are what is salient to a university and not what is not salient to a university. For that reason, they do not include vocational qualifications, even though the Government and the Parliament have recently committed very strongly to massively strengthening and valuing up vocational qualifications in Scotland.
In essence, we are still measuring what used to be standard grades—national 4 and 5s—and highers and advanced highers, and we are saying that that is how to measure children’s achievements. Many children leave school after their fourth year and go to college to do a vocational course, but that fact just vanishes off the face of the earth as part of our assessment of educational outcomes.
We have a positive destinations outcome, but that more or less means that the young people are not in prison, detained under mental health legislation or unemployed. Someone could be on a zero-hours contract. Is that the positive destination that we sought for children through the education system?
An awful lot of our measures are, at the very best, proxy measures that are not actually telling us much. A lot of our concern about educational attainment and inequalities in education comes from how we measure. We do not measure things for the kids whom we worry are suffering inequalities; we measure only those things that tend to suit the more academic stream of children in the system.
So your view is that, if we are not measuring something effectively, we are not going to be targeting the specific area in order to achieve the outcomes that society wants to see.
10:15
Or worse. In the cynical view that what is measured is what matters, people become driven by the particular measure that we are using at that point in time and lose sight of the outcome. Because certain targets now exist and people want to show that their school or council is doing well against those targets, a big drive goes on behind that. Whether that is the right thing for children, the future economy or their communities and society is probably a different proposition.
One of the challenges is to work back from policy and ask what outcome we are achieving—that applies to the whole range of outcomes to which Gareth Davies alluded—and whether we are using our total resource intelligently to enable us to achieve it, whether that is a local partnership or Government itself taking the overview at national level.
Again, there is no disagreement from me. I will mention something that is in Audit Scotland’s written evidence. It might be too early to mention the logic map, but it is a key part of what we are talking about because we are trying to establish how output targets contribute towards outcomes, and logic mapping would help us to build that. More important is the evidence or assumptions that underlie the outcomes because, if somebody produces a logic map, we can start to see what evidence or assumptions support the relationships between the achievement of the output and the outcome that we want to achieve.
More to the point, as Fraser McKinlay said, the earlier we are aware of whether something is succeeding, the better. We can then rearrange our resources or activities to try to achieve our outcomes better. It might turn out that we could spend a lot of time going down the road towards an output that we set a long time ago that will no longer be relevant or will no longer help us to produce the outcome that we want.
Being able to examine the evidence and assumptions that underlie the logic of the output measures is key.
Paragraph 13 of Audit Scotland’s submission says:
“The Scottish Government should map the pathways that connect each portfolio’s contribution to the national outcomes.”
That was the issue that you just touched on, Gareth, but what should we do less of? In his submission, Colin Mair touched on the fact that there is accretion but not deletion. Can we achieve more effective assessment of outcomes by removing some that are not as effective?
The way to identify what is effective is to challenge or question what the evidence is for the output in the first place. I would start by considering what different outputs people have.
It is a case of making steps towards that. I suspect that most of those present would accept that we will not solve everything overnight, although we might make steps towards that.
One thing on which the committee has previously picked up—I think that Audit Scotland referenced it as well—is the use of incremental budgeting. In a changing environment, that does not tend to lend itself towards challenging or rethinking where the resources are going. You should consider the budget process from the point of view of asking what the underlying assumption is of the budget model that you are using. Is it incremental budgeting or should we move to priority-based budgeting?
In an era of change and financial pressure, perhaps we should move towards priority-based or zero-based budgeting. One example of that is Shetland Islands Council. It has significantly reviewed its operations to decide whether its budget reflects its priorities and has gone through a zero-based budgeting exercise to restate where it wants its resources to go.
On that, I ask Fraser McKinlay whether there is much sharing of best practice on outcomes throughout Scotland.
I guess that there is not as much as we would like, convener. A wee while back—earlier this year, in fact—we did a report on developing financial reporting that considered some of those matters. We touched on priority-based budgeting and zero-based budgeting. There are some places that take such approaches. Aberdeen City Council has been doing priority-based budgeting for quite a while. There has been a lot of activity on that. Colin Mair will be able to say a bit more about the work that the Improvement Service and others have done on outcome budgeting in particular.
It is difficult; it is not an easy thing to do. If it were easy, we would have done it a long time ago. It is difficult in a practical sense. Let us take, for instance, money spent on housing, which touches on virtually every other outcome that exists. Everyone recognises that good-quality housing is fundamental to good health, good employment opportunities and everything else that goes with managing inequality. How do we attribute the spend on housing to the various outcomes that we are trying to achieve? That is not a straightforward exercise. There will always be an inherent tension due to the likes of people like me, in Audit Scotland, who will continue to be interested in whether the money that individual organisations have is spent well, properly and efficiently. I make no apology for that. How do we reconcile that view of the world with one that has to be much more about aligning resource and thinking about how resources are being directed towards budgets?
To be hopeful, I should say that our sense is that, particularly through the community planning work, people are grappling with the issue in a way that I do not think that we have seen before the past couple of years. The letter that John Swinney, Alex Neil, the president of the Convention of Scottish Local Authorities and the chair of the national community planning group sent a year ago that clearly set out the expectations on community planning partnerships and their use of the totality of the resource that is available to the partners has galvanised action. We can see lots of activity going on as people try to get their heads around that. However, it is really tricky. It is tough.
I will let Colin Mair respond, then I will open out the session to colleagues round the table.
To pick up on a latter point that Fraser McKinlay made, the issue is about the use of resources, not just budgets. Where people have sought to share every single budget line with each other, the net effect is that a staggering amount of time has been taken up and not much light has been generated, even if a fair amount of heat has.
We need to be honest about the fact that we are going to continue to run core, large-scale and expensive public assets and services. In local government, about 50 per cent of the budget is education and, of that, about 47 per cent goes directly to run schools. A huge proportion of the total budgetary resource goes straight into the school system. Understandably, the public are deeply concerned that schools be maintained and that the schools in their area are of a high quality and have the necessary staff complements and so on. The question might not be whether to take money away from a school and do something else with it. Instead, it might be whether to use the resources that the school constitutes in different ways with local communities, so that we get more value towards outcomes out of what we do.
At this stage in our discussion, if we go straight into budgets, which are often seen to be to do with the numbers around financial flows and so on, we might miss the most spectacular achievements. We work with a number of neighbourhood and community planning projects around Scotland and see people using resources in creative, imaginative and flexible ways at very local levels. They are getting on with work within a community and between communities to do things differently for and with those communities. There has probably been no change to the formal budgets of those organisations; the change is simply a result of people using resources in a smarter way.
One of the biggest resources that the public service in Scotland has is that it employs about 25 per cent of the total labour force. It is also overwhelmingly the major procurer in the Scottish economy and it has the largest asset base in the country. We must ask whether, given that we have those capacities, we are using them to create the outcomes that we say that we want. We often put community benefit clauses into procurement processes, saying that we want people to create modern apprenticeships, employ people from deprived areas and so on. It is worth asking whether the health board that puts in such a clause actually does that itself and, if not, why not, and whether it intends to address that. We know that ensuring that communities have better economic opportunities is a significant step towards those communities having better health, achieving more in education and so on.
I encourage the committee to think about resources and not just finances, although I accept that your role is to scrutinise the Scottish budget.
However, in paragraph 8 of your submission, you say:
“Examining both international and Scottish data, we can find no systematic evidence that the organisation and quality of public services is the key or main determinant of the pattern of outcomes in any society.”
Yes. There is no question but that the pattern of economic outcomes is driven more directly by macroeconomic and fiscal strategy than by how local public services are organised. The range of literature shows that that conclusion is reinforced by research. However, that is not to say that public services cannot have an impact. I think that they can and should have more of an impact with regard to creating opportunities for people who currently lack them.
A good example is the new Scottish police service, which is about to locate its new headquarters in Dalmarnock in Glasgow, in the Clyde gateway. The fact that the police have been willing to do that has opened up a site that will become hyperactive with private investors coming in precisely because the police presence constitutes an anchor. Once a big public body shows confidence in an area, the private sector starts to show confidence on the back of that.
That is really intelligent use of the Scottish police service’s capacity. It needs a headquarters. Where is it going to base it? Answer: pick an area of deprivation for once rather than an area in the more pukka parts of town and then create an economy around the headquarters because cafes, shops and so on will grow up to service the office workers coming in. If that then gives rise to confidence in the place among the private sector, we will start to have incredibly positive investment flows into a part of Glasgow that, frankly, was being written off 20 years ago as going nowhere at all—it was just contaminated land.
How we use our asset power is a really important part of how we stimulate economies to give people opportunities that will then support their health and wellbeing within their communities.
My first question relates to Mr McKinlay’s submission, which states:
“there is evidence of greater focus on outcomes, both nationally and locally”.
Earlier in the submission, you specifically say that your
“audit work has demonstrated”
the impact of the NPF
“in aligning resources and action across different parts of the public sector in some policy areas”.
Will you say a little more about that and cite those positive examples? If Mr Davies and Mr Mair want to comment on that as well, that would be useful.
As I said right at the start, the NPF with its outcome approach is a good thing. It is a really important first step and it is embedded, in many ways, in how public services are thought about and run, particularly at the most senior levels.
One good example that we cite in the submission is renewable energy. When we did a report on that a year or so ago, we found clear evidence that all the organisations that are involved in renewable energy reflect the overall policy objective and desired outcomes. That is a good example of an area in which there is clarity and consistency of approach right the way through.
In a recent community planning report, we reported on how Glasgow has coalesced its community planning partnership around three priorities. I do not remember them all, but I think that they are on alcohol, in-work poverty and something else that escapes me just now. That is a real achievement, because that is not to say that Glasgow is ignoring everything else. Clearly, everything else that the CPP and public services in the city of Glasgow have to deal with remains important. However, they based that decision on an understanding of the communities of Glasgow and an understanding of the data on those communities. They also had an interesting approach that involved tackling different things in different parts of the city, in recognition that Glasgow is not one homogeneous place. They had got round the table and agreed that those three priorities are the three things that are most likely to make the biggest impact on inequality and outcomes in the city.
There is an awfully long way to go for that CPP to actually make that deliverable and make the difference, but it is one good example of a local outcomes approach, and by no means the only one. The missing bits of the jigsaw are all the supporting bits to do with the money. The CPP is at the early stages, as are all CPPs, of figuring out how it then targets the combined resource in Glasgow to those priorities—how the bodies spend their collective money and use their buildings and people to achieve those outcomes.
There is also the performance management area, to pick up Gareth Davies’s good point about governance. In relation to how the partnership is governing all that, there is still a long way to go. However, when we did the last national community planning report, which is getting on for 18 months to two years ago, we talked about an opportunity to deliver a step change. There was a real sense then that people were genuinely committing to the approach. That commitment is there and the real challenge now is to put in the infrastructure to support that approach.
10:30
You seem to be saying that the prioritisation of outcomes is important. This perhaps relates to Mr Mair’s point regarding the proliferation of outcomes perhaps being a bad thing and the need to be focused in order to pursue an outcomes-based approach.
That is a really interesting question. The Accounts Commission considered the West Lothian CPP last week. As you will know, that CPP is long established, with a good history of very strong partnership working, and it provides great examples of the co-location of public services and other things. The CPP is taking a different view. It is adopting quite a broad front on outcomes, and it has not narrowed down to two or three key priorities. We have highlighted a challenge or risk to that partnership that, although it is entirely up to it to decide to do things that way, it needs to figure out how it will use its scarce resource to make progress on a wide range of fronts.
I am a wee bit cautious about saying that one model is better than another, but it is important that people have a model and a plan for how they will implement it. In particular, they should understand how they are going to organise themselves, their people, their building and their money in order to deliver the outcomes that they have set for themselves.
Does Mr Mair or Mr Davies have some perspectives on that?
I agree with what has been said. A key thing is the role of CPPs in locality, total place or community budgeting, as it is sometimes referred to, which is basically being able to say how much public service money is being spent in a particular place. As Colin Mair has said, it does not stop at the pound signs. Are the resources being used to best effect?
As far as CPPs are concerned, there should be a duty of best value for an area that involves noting the resources in the area and asking whether we are getting the best for that area. It clearly also goes back to what Fraser McKinlay said about how well attuned any CPP or equivalent body is to the needs of an area. Engagement is key in that regard. I know that the Improvement Service has just done something on engagement.
My sense is that three really positive things are happening at local level. One is that most of the community planning partnerships in Scotland have reached a smaller number of outcomes that they say are a priority, with massively fewer performance indicators. If we are about
“demonstrable improvements to people’s lives”,
in the words of the statement of ambition of last year, let us have half a dozen indicators telling us whether people’s lives are moving on or not.
Secondly, there is now more targeting. There is a recognition that some of our communities are living very good lives and have excellent outcomes. They are marginally self-sustaining around those outcomes, and they use public services when they want to, but they are in no sense dependent on them. There are other communities with a far higher level of need for properly organised and responsive public services. There is now more of a focus on identifying communities where, across a range of outcomes, people are not doing well. Let us commit to helping those communities to move on by working with them in new, more flexible and different ways. There is often measurement around that, which is the total place approach to which Gareth Davies was referring.
Thirdly, we have almost but not quite stopped using the national performance framework as cake icing—the idea that, whatever we propose to do, we should slap on a dod of national performance framework. There was a cynical referencing a lot of the time with regard to the national performance framework. I have been looking at some of the papers that went to Parliament. I suspect that the amount of references by the civil service to the national performance framework is liturgical rather than real, with the assumption being that people have got to say this kind of thing. We need to move away from that and actually focus on outcomes to which we are truly committed and that we truly intend to change.
The localism that Fraser McKinlay and Gareth Davies have emphasised is really important. It is when we get down to the level of community and we engage with communities in different ways that we begin to see new routes to achieving outcomes and new capacities that communities can bring to the table. That is part of the resourcing question as to how we take outcomes forward.
You have spoken about community partnerships, localism and engagement. Surely those outcomes will mean something for people only if they feel that they are relevant for them. How involved are people, particularly at community level, when it comes to the outcomes that they would like to see for their communities—for them, for their children and for everyone who lives in their areas?
People are more involved. The interesting relationship is between engagement and empowerment. How empowered are some communities to drive forward, and force people to prioritise, the outcomes that they see as important for their communities? Engagement has become hugely better; a lot of time, effort and energy has been spent locally on engaging communities.
Yesterday, I was at an all-day event with a neighbourhood in Fife. A lot of effort and energy went into that and the community was fantastically active and constructive in engaging with outcome priorities—not for the whole of Fife but for its area. That localism, in combination with the willingness to be open in engagement, makes the difference. If people in a bit of Kirkcaldy are asked what they think about the whole of Fife, they may—perfectly reasonably—say, “Not a lot.” They are—rightly—interested in what happens in their bit of Kirkcaldy. Localism allows people to engage more.
In the classic old-fashioned budget consultation, the aim is to save £20 million and people are asked to tick some of 86 options. That is not engagement at all; that is a tick-box exercise. As we are getting much more local with communities, we are getting engagement of a far higher quality than in the past.
Engagement is a prime topic. Yesterday, I was at an event about tenant participation in the housing revenue account, where some of the ideas were interesting. It was emphasised that engagement should be a head-to-toe culture in an organisation; bodies do not need just formal panels—front-line service delivery staff are normally the first to get feedback from people on how happy they are and what their aspirations are, as has been said. The issue is how that information flows through an organisation and leads to service delivery and service planning decisions. A large part of client engagement can come from the front; to an extent, that comes down to the governance or culture of an organisation.
My next questions follow on from the discussion of community engagement. The witnesses are probably all aware of the Scottish Government’s Community Empowerment (Scotland) Bill, which will place a duty on ministers to publish reports regularly on progress to achieve national outcomes. Do you welcome that? Should that happen? Will it be positive?
I think that such reports will be positive. We have a national performance framework, but it is not used for national performance management. I noted in my submission that we want to be fairer, but I have no idea how much fairer we want to be or by when. To drive the system, it might be more interesting if the national Government was clearer about its level of ambition, the timescales and its expectations about how the public service should deliver. That would provide a dynamic in the system.
We have a framework of outcomes that says that we will become fairer, smarter and so on over time. If we asked how much smarter we would like to be by 2020, that might be helpful to feed into the system, and it could be linked to the local level. I welcome the proposals, but the framework might need to be tweaked and focused to make it useful.
I echo the point that the principle of enshrining the outcomes approach in legislation is good. As members would expect, we are interested in how that will work, and what governance and accountability arrangements there will be. If people are not seen to be fulfilling the duty on outcomes, what will happen? If ministers of any future Government set out with a bunch of input targets and measures rather than outcome measures, what would that mean?
In our most recent community planning national report, we asked what the accountability framework is for partnership and community planning. We are between a couple of stools—we say that there are national outcomes and a national approach, but that is all about place. We have not bottomed out the inherent tension in that. The step is good and useful and we will be interested in how it plays out in practice.
Another planning issue concerns the spending review periods that the Treasury sets. It creates a challenge for organisations if they are not sure about or there is a lot of uncertainty about future funding and they are trying to think five to 10 years ahead.
I do not think that, even in the best of worlds, there will ever be certainty about funding so, to an extent, it is a case of encouraging people to plan for the medium and long term despite the uncertainty rather than say that it prevents them from doing anything. Service planning towards achieving outcomes is important, because otherwise planning will probably always be done on a year-by-year basis.
Thank you.
I want to ask about the national performance framework. In round-table discussions that we have had in committee with economists, they said that the NPF was a progressive development and that it was recognised internationally as something to be revered. Do you concur with that view? Do you see the NPF as progressive and groundbreaking and, if so, for whom?
Shall I?
Yes! [Laughter.]
I am sensing a bit of a drum roll, but my answer to the question is yes. It is interesting that “the Scottish model of government” is now a phrase that people recognise and welcome. It will be really interesting to see what happens at the next Holyrood election and whether, in broad terms, the approach is embedded almost regardless of who the Government of the day is. As I said earlier, it feels as if it has become embedded. To be fair, the notion of outcomes, single outcome agreements and so on had been around even before 2007, so the outcomes approach is not tied uniquely to any one Government or political party. There is a lot of international interest in and research on the approach.
I concur that the outcomes-based model is a very good and progressive model, and it is difficult to see how we could ever go back to one that is not outcomes based. As I said, there is an awful long way to go to realise the potential of that framework. We get quite a lot of pushback in the work that we do, because our job is to hold Government, primarily, to account for how it does things. Quite often, in committee or in other places, it says, “But this is all really difficult, you know,” and we say, “Yes, we know it is, but this is what you’ve decided to do.”
Now that we have started down the path of outcomes and an outcomes-based approach to delivering public services, we need to continue down that path all the way to making sure that what is happening in individual organisations and in places and towns all makes sense and all stacks up to deliver the outcomes approach. It is not enough just to have the national performance framework and Scotland performs, as good as that is; we need the infrastructure for running public services day to day, including resources and budgeting, to really make it fly.
I add that the models that were adopted by American states were among the models for Scotland performs. For example, the Virginia performs model was an underlying model for the Scottish system. What is interesting is that people in the States do not take Virginia performs as a measure of the performance of the Virginian state Government; they take it quite literally—because they are sceptical about Governments doing anything for the economy or anything else in America—and simply say, “This is how well Virginia is doing.” It is often pitched at international inward investors as a way of saying, “Look, this is the kind of place you might like to be because we’re not very unequal, our health’s fairly decent, our kids are well educated and our labour market’s flexible.” Many of the indicators are therefore taken not as measures of the performance of public services but as simple statements of how the place is doing.
At one level, we might want such a running checklist on how Scotland is doing. If it was not doing well in areas that we would wish it to be doing well in, the public services would have to think about how they could contribute more and better to make the place the way that we would like it to be. However, I think that a genuine tension runs through much of the discussion about outcomes, which is to do with whether they are simply statements about the world of our country, our society, our economy and so on—that is fine, because we should have such statements and they should be accurately measured—or whether they are judgments on the performance of public services.
The most recent programme for international student assessment—PISA—data that was reported to Parliament was on the educational performance of kids in Scotland compared with that of kids in other Organisation for Economic Co-operation and Development countries. One of the points that was made was that the variation in Scotland is disproportionately within a single school rather than between schools, whereas in other countries inequalities in achievement mean that one school is doing really well systematically and another school is doing really badly. In Scotland, with our big secondary schools, all the variation is occurring between different people using exactly the same school resource. Some people are using the school resource and getting fabulous results by any international standards; other people are using the same resource and are falling off a cliff by any international standards.
10:45It may well be that the factors that are shaping outcomes in Scotland are not just—or even predominantly—the public services, but we should have a statement for ourselves about how we are. In Virginia, that includes public attitudes data as well. Are Virginians less hateful people than they were, say, 20 years ago? Are their attitudes more progressive? Are they more egalitarian now than they were? A whole series of things are included, which tell the rest of the world what the place is like, how they are doing and what the key trends are. I do not think that we are sure whether our national performance framework is a Virginia performs or a performance management tool for public services. I am not sure that we have ever bottomed that out.
That indicates the difficulty in separating correlation from causation. As Colin Mair says, the national performance framework is widely admired and tries to measure Scotland as a society on the important fronts. However, as he indicated, it is difficult to see from that which elements are down to public sector performance and which may be due to other factors. Going back to the idea of a logic map, it is key that we try to establish the actual causation rather than just whether there is some correlation. Establishing that causation will, in the long term, lead to the transformation of public services.
Mr Davies mentioned Shetland Islands Council, which—I do not think that he said this—is having to prioritise in the light of reducing budgets and so on, as well as that being a good exercise to do. How does that fit across the board with single outcome agreements as they relate to community planning partnerships? Is there always a clear understanding, or are rules followed regarding those priorities?
To a large extent, it hinges on how environmentally aware an organisation that is making changes is. Is it considering all the consequences and impacts of its actions? Shetland Islands Council is a particular case, given the level of its reserves, but it is basically trying to find financial stability to protect itself going forward. There are financial pressures in other areas as well, including in Aberdeen as Fraser McKinlay said, and organisations are being driven to say that the incremental budgeting process may not be suitable for them now. It may have been suitable in the past, during periods of more stability, but when there is significant financial pressure you probably have to examine what is happening more fundamentally.
On liaison with partners and CPPs and single outcome agreements, whenever somebody thinks about changing service provision, it is a case of whether the consequences of any action or inaction have been figured through into the service planning and how much that has been taken into consideration. If somebody thinks that their target is just to save money and that is what they are doing, that is fine—they may save money—but that might push another burden on to the health service, for example, and that is not desirable from a public money point of view.
When we look at tight budgets, do we prioritise services or outcomes? You will be aware that all public authorities are in the middle of a three-year planning exercise that, in many cases, will take very substantial sums of money out of public budgets. Within a council, education is evidently a priority because it is about children and it is a national priority, and social care, given its relationship with health, is also clearly a priority. We prioritise those services. One danger of that is that we do not prioritise services such as environmental and land maintenance, even though those are the parts of councils with the best track record of creating entry-level employment and social mobility. The people who come in at the entry level are advanced through the system and, 15 years later, they end up running quite complex public service businesses. They did not come in as graduates; they just came in through the entry-level route.
If one of the outcomes for every community planning partnership in Scotland is to create a better flow of entry-level employment and get people from unemployment into employment, and if we are prioritising in that way, we might say that we should preserve services such as land maintenance, facilities management, catering and cleaning, as we can help people into employment in those areas. However, I suspect that, in reality, we will end up prioritising areas that largely involve graduate professional employment, because they are seen to be the highest priority in service terms.
People need to do a really complicated juggling act involving public and political expectations on services and outcomes. I was once challenged by a Scottish Government minister to write an outcome-based manifesto. I tried to do it and it came out as complete mince—I would not have voted for it. Which party does not want Scotland to be smarter, healthier and so on? Once we get to that level, no one disagrees. Elections tend to be fought locally and nationally on issues such as keeping a local school open or making sure that there is a classroom pupil to teacher ratio of 26:1 or whatever. People get that, but the outcome thing is much harder to turn into attractive politics.
My sense is that the issue is really complicated and that, if we were prioritising outcomes, some of the judgments that we would make would be quite different from the typical priority judgments that we make about services.
That is absolutely true in councils, as Colin Mair’s examples demonstrate clearly. The issue is even more complicated in community planning because, increasingly, with the exception of the places where bodies have coterminous boundaries, such as Fife and Dumfries and Galloway, the council is now the only genuinely local body that sits round the community planning table, with the exception of third and private sector interests. Health boards and colleges and the police and fire services are all now regional or national bodies. There is a strong tension for, say, Greater Glasgow and Clyde NHS Board, which sits on eight or so community planning partnerships, in trying to balance its requirement to deliver its HEAT—health improvement, efficiency and governance, access and treatment—targets and do what it needs to do for the whole of Glasgow while contributing meaningfully to eight single outcome agreements across the diverse communities in the greater Glasgow and Clyde area.
Colin Mair gave a helpful example of the complicated nature of prioritising services or outcomes, but the situation is even more complicated in the community planning partnership arena than it is in councils.
Colin Mair talked about local communities doing things anyway. How does that measure up? If the national performance framework is the pinnacle, or the “tip of an iceberg”, as I think that Fraser McKinlay called it, are we talking about that level of action in local communities? Certainly in the region that I represent, that is where the real progress and action are happening and where the exciting work is being done, as people literally decide that they can build houses or run a renewable energy scheme such as a small hydro scheme and earn some money for their communities. How does that fit into the national performance framework, in every sector on the way up?
That is built into the focus on local place that we now have. In the national performance framework and the literature that goes with it, there is a strong commitment to Scotland having active communities. From my point of view, one of the more exciting developments in the past five years is that communities are working in a much more empowered way with public authorities, but some of the time they say that they do not need to work with them and that, if the public authorities just keep out of the way, they can get on and do things for themselves. That feeds into national outcomes. Communities that are active and feel in control of their lives and feel that they can achieve things are more likely to have better mental health, greater economic participation and so on.
However, the pattern of that type of community activity across Scotland is often uneven. In some areas where we most want to see it, we need to do some work to help communities to organise and to feel empowered and feel that they can take control. One danger of a model that simply hands things over to communities to let them get on with it is that, although some are very well situated to do that, others are not, and we need to be careful that there is a proper pattern of support for the participation, engagement and empowerment of communities where that is necessary.
I think that this brings us back to the question whether an organisation has an internal focus. Is it focusing on how it provides or measures services, or is it more interested in measuring how the communities involved are doing and the impact of what it does? One example that assesses need and outcomes is the adult social care outcomes toolkit—or ASCOT—which asks individuals how they are doing in, I think, 10 different areas to get an idea of the impact that is being made. Over time, that impact can be seen, and I would regard that kind of focus as being external as it is all about getting feedback from the client or community. A measure that is about seeing a particular number of cases in a particular number of hours is internally focused, and that sort of thing might be partly what is behind people beginning to ask whether their organisations are community focused and what they are trying to measure.
I was particularly interested in Colin Mair’s point about how we sell outcomes to the electorate. After all, they cannot be measured easily and the electorate is more interested in the number of nurses in a particular hospital, whether a particular hospital is open or closed and so on. Listening to all this, I wonder whether when we talk about outcomes the language is so vague and general that we can all sign up to it but, in practice, we just need to keep it in the background and focus on inputs and outputs.
No, because that would make us focus back on our organisations, and one of the most powerful accusations that have been made of the public sector historically is that it has tended to vanish into itself instead of focusing on the people whom it is supposed to serve. One good thing about an outcomes focus is that the point of what you do is to make people’s lives better. We are getting towards asking, “In what ways do we want our lives to be better and what is the likely contribution that public services can make towards that aim?”
Of course, public services will not make a total contribution. After all, if the decisions that were announced at a party conference yesterday were implemented, they would make some communities in Scotland poorer. You might agree or disagree with that, but that is what would happen; there will be some impact in those communities. Moreover, other aspects of fiscal and macroeconomic policy and indeed global economic pressures are going to have an impact on Scottish communities as they will in other communities across Europe and elsewhere in the world.
We should not assume that public services create the pattern of outcomes in and of themselves; they do not, and all the evidence suggests as much. If, however, the kids in a disadvantaged community are not attending school, do we simply conclude that that says something about those kids or do we ask the much harder question: why are we running a £5 billion education system in Scotland that perhaps 20 per cent of its users find offputting? A private business that was found offputting by 20 per cent of its market would not be unduly happy about the situation. Can we run schools in ways that make them more engaging? That sort of approach would lead us to look quite hard at issues such as the point in kids’ lives at which they seem to become disengaged. It does not seem to happen at primary school, so it must happen in the transition to secondary school. Is that transition being managed well enough? Are we offering some children the learning opportunities that they want instead of forcing a standard set of learning opportunities on them? If we start off with an outcomes focus and take it quite seriously, we can start to ask the hard questions that we could evade by looking only at inputs and outputs.
Who should be taking that outcomes focus? Should it be the Parliament, or should it be councils, headteachers and so on?
It should be happening the whole way through the system. Indeed, the more local you get, the more genuinely people get and are committed to outcomes. They become real; they are about people whom they mix with day in, day out; they are not abstractions or performance measures. They are real people whom you encounter in your day-to-day work, whether it be education, social care or whatever. In a way, the local probably gets it better but talks about it less, while the national gets it less less well but talks about it a hell of a lot. That is perhaps the way you would want it to be, because what happens locally is what will most materially impact on the quality of people’s lives and the quality of opportunity that is available to them.
11:00For me, the issue of outcomes is not about political accountability alone; it is about managing services and working with communities in new and different ways. If anything, ensuring that people are empowered and focused is more important than political accountability.
I think that a real change is taking place. If we abandon the outcomes focus now, we will be doing so prematurely. I think that it is growing legs at the moment and is beginning to motor along—I apologise instantly for that completely mangled metaphor. We often abandon things in the public service just when they are about to pay off.
I totally agree with that.
A headteacher in one of my local schools said to me that it is like having two separate schools. That ties in with what you were saying earlier about the same resources producing quite different results. The council and we want to measure the success of the school and examine higher results and so on, but the headteacher has the freedom only to move periods around a bit within the week—the number of periods is decided for them. If we push more freedom in decision making down to that level and ask headteachers to take an outcomes-based approach, does that not become impossibly hard for everyone else to measure?
For some kids, you will be able to properly measure their progress by looking at standard grades, national 5s, highers, advanced highers and so on. For other kids—this relates to how empowered leadership at a school level is able to be—should we be offering vocational qualifications from secondary 2 onwards, if that is what kids want to do? What is our problem with that? You have to do Spanish, but you cannot do car mechanics. I absolutely fail to comprehend why that would be a sensible judgment to make, if part of the school wants that sort of opportunity.
If we valued those things up, I do not think that you would find that headteachers would want to stand in the way of developments of that sort. However, just now, we are saying that someone must go through four years of one sort of schooling before they can escape to the vocational education that they always wanted. That does not fit with what the Government and Parliament have said about their commitments to vocational education in Scotland. I would say that a headteacher who feels that they have two schools should think that they are failing if they are catering to only one of them and they have a duty to cater for the other school, as it were, and ensure that the educational offerings are attractive to and positive for the range of pupils who are attending the school.
You are right to suggest that we are locked into a way of thinking about educational performance and that that gets imposed downwards on people. I would prefer greater empowerment to tailor the education to suit the kid. That is what curriculum for excellence says, philosophically speaking, and we now have to ensure that, in practice, education is fitted to the child rather than the child being fitted to a form of education that we have already decided on.
Mr Davies, you used the word “awareness” earlier. That suggested to me that, as long as the schools and the other organisations were aware of the national outcomes, that would be enough. Is that enough, or do we need to pin them down more?
That is a difficult question to answer, in many respects. In a previous CIPFA submission on the issue of governance, we recognised that there might be a difference between the national priorities and the national level of service that people want to see as a bare minimum across Scotland, on one hand, and local variations on the other. We have suggested before that, if there is going to be local variation, there should be a clear reason for that variation and it should be justifiable. People should be able to justify to central Government, for example, why that variation has occurred and why it is appropriate for the locality.
I take the point that you made in your initial question that outcomes are always going to be seen as a bit nebulous. There is always going to be the quality aspect that people are going to say cannot be measured.
A balance must be struck between the issue of awareness and—to use the motoring analogy that was used earlier—the issue of who is driving. When someone is driving a car, generally speaking, they are looking out—they are not looking at the dashboard all the time. That might be a weak analogy, but it strikes me as having a bit of truth to it. The question is the extent to which you want people to be driving what is happening locally.
Yes. I think that you also mentioned that you accept that the causation—the linkage between spending and outcomes—is difficult. I have been in Parliament since 2011, and that has certainly come up with all the budgets since then. Can we link the spending—I guess that that involves looking at the dashboard—with the outcomes out there and the bigger picture? Do we simply have to accept that that link is pretty loose a lot of the time?
Nirvana would be having perfect information and being able to measure outcomes perfectly in a way that lets us say, “That’s what we’ve got.” I think that, certainly in the short term, proxy measures are the best that we will get. For me, it is all about challenging how good the proxy measure is. To be honest, I suspect that there will be that gap, but narrowing that gap is the important thing.
I am interested in the term “proxy measures”, which came up somewhere else, as well—in relation to schools, I think. Is it inevitable that an organisation such as a school or a hospital will start to bend its performance to meet the proxy measure rather than the outcome, and that we therefore need to keep changing the proxy measures to pin them down?
That has certainly been suggested in one draft publication from CIPFA. It was commented that some organisations have changed the output measures, partly in response to seeing what the causation is and partly to prevent an overly output-measure focus that detracts from what people are trying to achieve. It is a matter of trying to say to people, “Yeah, your output measures are not the be-all and end-all of what you’re doing,” because they can drive innocent—and sometimes underhand—distorted behaviour. For example, in England, in order to reduce waiting times and get people into what were called beds, people took the wheels off all the trolleys and said, “That’s a bed.” [Laughter.] That does not exactly seem to be going in the right direction, but that is an example of being output-measure focused rather than outcomes focused.
That strikes me as a good example of innovative thinking that we should probably try to harness.
To come to your original question, Mr Mason, we have found that, when people talk about the outcomes approach, the mindset is that it does not matter what we do to get there. I have repeatedly heard the challenge from people in councils, health boards and other places that the end result is all, so why are we—the auditors—interested in the inputs and ouputs? We will always be, because the outcomes approach is about all those things. We cannot look at only the outcomes without thinking about the inputs, outputs and the activity that drives the outcomes. For me, it is about having an overall, whole-system view of the world. This will sound a bit flippant and trite—I apologise for that—but it is a mindset thing.
To continue the school example, I do not think that anyone is suggesting that a community planning partnership will take over the running of a school, but it is reasonable and legitimate that everyone who works in that school should think about their job in terms of outcomes. If a person works in a school canteen, their job is not just to hand over the food to the kids as they come through and get their lunch every day; they should have a different kind of outlook. They should ask, “How does wee Johnny look today? Has he been here for a couple of days?” There should be a wider approach and a wider mindset for everyone in the public services. They will still need to deliver the service—to come to Colin Mair’s point—but they will do so in the context of a wider set of outcomes and what we are all ultimately trying to achieve.
It is about that kind of cultural thing. The measurement issue is tough, and we are grappling with it. How do we audit some of that stuff? How do we audit a culture of outcomes and good partnership working? Things are not as straightforward as they used to be, but we need to find the measures that capture those softer cultural aspects, which will, I reckon, make the difference to people’s lives.
If that is the challenge for you as the auditors, it is also a challenge for us.
Absolutely.
How we oversee all that and make any measurements is a challenge for the Finance Committee and the whole Parliament. It is very easy to say, “Well, there’s a waiting list of X.” We all do that kind of thing, but do you have an answer for us?
We report to Parliament and the job for parliamentary committees is, in a sense, to challenge the people who are running the place and, in part, to figure out what scrutiny to apply. The bit that we all struggle with is knowing what the measures are in the first place that will help us to do that. That is not my job and it is not necessarily the job of committee members; rather, it is the job of the people who are running systems to figure out what sensible, nuanced and sophisticated measurement approaches we can develop. Examples are out there and we are seeing those approaches in some places. Through its work, the committee will continue to apply that scrutiny.
When you look at the budget papers, it is striking that, although there is lots of good stuff, with lots of detail, and much of it is transparent, the numbers part remains almost entirely disconnected from the activity and outcomes part.
You mentioned community planning partnerships, which are the other area that I want to ask you about. I remain pretty sceptical about whether they are just talking shops. The police and everyone else turn up and talk about a theme, but they then go away and do whatever they were going to do anyway, although with a little bit of cake icing as Mr Mair said.
People understand what a council is. They elect a councillor. If something goes wrong with the roads, they go to them or shout at them or whatever and the issue will perhaps get sorted. Community planning partnerships are such vague animals. To whom are they accountable? We understand the council bit, because it has an input.
As far as I am concerned, the health secretary cannot oversee the input of 32 bits of the health service and assess whether they are having good local input into Clackmannanshire, Glasgow, Orkney and all the other places, can he?
To be fair to the Scottish Government, it has been pretty clear that the Scottish Government elements, including health and all the agencies such as the enterprise agencies, Scottish Natural Heritage and other organisations will contribute meaningfully to community planning. For example, that is part of health boards’ local delivery plans and the accountability process.
The system is designed to work so that, as well as the people in the partnership holding each other to account, the individual bodies are held to account for their contribution to the partnerships. That is the theory. On that point, we probably agree, because I do not think that we are there yet. We saw examples in the CPP audits not that long ago in which the health board was not even turning up half the time to the community planning partnership meetings. That is the point that I was making: what is the accountability mechanism for partnerships? Something needs to happen to make that better.
In a way, CPPs have evolved. They are by no means the finished article from the viewpoint of anyone at this end of the table, but they are progressing. Everyone is agreed on the need to have something that pulls together across public agencies a common focus on outcomes at local level. If you do not call it a CPP—I do not care what you call it—that is fine, but if nothing is there, we are back in our boxes doing our own thing down our service lines. It would be a shame to lose what has been a genuine progression across the last period. The best partnerships are quite hard-nosed, they have a small number of priorities and they are very clear on how they will achieve those and so on. We need to capture some of that practice and roll it out to the system rather than do in community planning partnerships.
There may have been a talking phase—I see that everyone is smiling—but, frankly, I think that that was needed to create a basis of trust for people to do things. That may seem a bit luxurious in retrospect, but that was a necessary period in which people got together to talk, so that they could then plan together. I am more positive about the situation. There is a lot more clarity in the community planning partnerships and their meetings are much more purposeful. This year, there will be a lot of emphasis on sharing information about how we are using resources and what impacts those may have on outcomes and on different services. Therefore, we are getting somewhere with them.
I cannot necessarily talk so much about CPPs, but where there is a structure and incentives to co-operate, that co-operation between bodies can happen. I understand that a lot of effort went into pulling together the Clyde valley city deal, to agree the governance structures and arrangements and to move ahead with it fairly quickly. That is an example of how different organisations covering one geographical area can pull and work together. Therefore, the challenge for the CPPs is how they get a structure that achieves that same objective.
Thanks very much.
11:15
To follow up on some of the issues that John Mason has been raising, we got on to the idea of political expectations being part of the equation. I have watched service providers develop plans for the service, only for them to be ripped up by the politicians because they did not suit the political agenda or their local interests, which is a matter that I have raised a couple of issues about at this committee. Although politicians will sign up to policies such as single outcome agreements, national performance frameworks and other such ethereal indicators of the country’s wellbeing, I have heard the service providers and the people who have given evidence here repeatedly say that we do not collect the data well enough to know where the evidence base is on which we are making the decisions. Some of the decisions are counterintuitive, but we do not have the data to allow us to judge whether they are the right ones. How can we improve data collection so that, whatever we are discussing, we know that what we are talking about is on a firm footing?
Excuse me a second, Michael. I did not want to interrupt you, but I am afraid that I must suspend the meeting. There is a sound issue and it may be that what is being said is not being recorded, which would create a problem for the Official Report. We will take a minute or two to try to sort out the matter, and our witnesses can ponder their responses to Mr McMahon’s question.
11:16 Meeting suspended.
Michael McMahon is absolutely right in saying that we need to improve data collection, and a number of national initiatives on that are under way. National Government, local government and health colleagues have a programme on improving evidence and data, which is trying to get better local data available more quickly so that we can better understand what is impacting on what. Work is also going on between the Scottish Government, the Improvement Service, the Society of Local Authority Chief Executives and Senior Managers and others to develop a framework for benchmarking between community planning partnerships. A very small number of outcome indicators will characterise that framework, which will look at variation and diversity across Scotland’s CPPs on those core indicators.
That work is throwing up the fact that we have some bad measurements that are not helpful, which we will have to get rid of, and the fact that in other areas we have no measurements at all. Given the focus on outcomes in Scotland, it is odd that the first reliable comparable measure that we have of how kids are doing in education is at the age of 16. Given the amount of investment that we are making in the early years, it is utterly puzzling that we wait until children are 16 to discover whether that investment is paying off. Local measurements are being made at earlier stages across Scotland, but they are not standardised so we could not bring them to you and say that you could make a comparison across Scotland with a single Scottish figure.
Part of the issue is in deciding how much standardisation we want and how much data collection should be locally driven with people using whatever measure suits them locally as long as it drives their practice locally.
I reassure you that a lot of work is going into improving our measurement base and our data base and into speeding up the data flow, which is often quite slow at the local level. If it would be helpful to the committee, we will happily copy you into details of that work.
As you can imagine, we love data in Audit Scotland. It is enormously important and there is loads of it. The issue is not that there is not enough data; it is the nature of that data and, most important, what we are doing with it.
Michael McMahon made a point about the political element of data. Data can be quite challenging and there is an issue about how data is used to select priorities and, when priorities are selected, what is deprioritised. Colin Mair gave some examples of spending decisions that need to be made. Very often, when you use data about the community and community need, the thing that gets the biggest bang for your buck is not always what politicians or people in the community think is the best thing to do.
Data can be very challenging, which is why it is important that it is robust and used well and that people take care in how it is presented. Some of the examples that were given earlier of things not happening are a result of people not taking enough care about how the argument is presented. We—I include Audit Scotland—need to be more mindful of the context in which the data and analysis land, so that you are pushing at an open door with us. There needs to be more and better use of data, and, as Colin Mair has said, we support all the things that are happening.
11:30
CIPFA supports the idea and application of benchmarking. The important thing about benchmarking is that it is not an end in itself. Its role is to drive service change in effect, otherwise why would you do it? In doing it, the one thing that we tend to focus on is the importance of consistency. For example, in its submission on the integration of adult health and social care, CIPFA advocated that a national performance management framework be set up for data that is client based and focused not around how the service is being provided but more on the outcomes or outputs for the clients. Regardless of how that was done, that would allow you to compare consistently the different ways of doing things and maybe establish which ones work best for which reasons.
This might be a question for Mr McKinlay, but I am interested in whether anyone else can come up with an idea. When evidence is produced on the basis of which a decision is justified, you can at least make a judgment on whether that evidence leads to the conclusion that people think it leads to. However, when people counter that decision, how can we ensure that the arguments in the other direction are evidence based?
For example, I was recently frustrated when a health board decision in my local area was overturned. Having been given the task of designing a service, the health board produced a report but it was overturned without any evidence to counter the outcome that the health board had produced. How do we prevent ourselves from getting into a situation in which evidence is produced, outcomes are set and inputs are measured but a decision can be overturned without any evidence in the other direction?
I will sidestep neatly around that specific example, Mr McMahon, but the point that you raise is an important one. We have talked a wee bit about the political context in which all of this operates, and there is no point in fighting against that. That is the world in which we live. In local government, national Government and Parliament, politics is what makes the world go round and we need to find a way of working with that.
One of the tensions that we cite in our community planning report arises from the fact that people make decisions and judgments from slightly different viewpoints and perspectives. A council is designed and elected members are there to do the best for their local area, but that will not always be the best for the health board for that area or for the national health service as a whole, and so it goes on. Exactly the same challenge exists in the Scottish Fire and Rescue Service, in Police Scotland and in other places. There are now real tensions between the regional and the local—and “local” could mean right down to ward, neighbourhood or street level. There are always discussions, negotiations and tensions in the process of deciding on the best answer for a particular place and in how people come at the issue.
Evidence can show that people have a strong case that something is the best thing for their neighbourhood. I live in Haddington, so I might have evidence to show that something is the best thing for Haddington, but it might not be the best thing for Musselburgh, Dunbar or anywhere else in East Lothian. That is the kind of tension to which I do not have an answer, and I do not think that there is a silver bullet.
Colin Mair talked about relationships and trust, which are hugely important, as is understanding where your priority focus is. If a community planning partnership is focused on inequality, we must ask where the most problematic pockets of inequality are and what we can do collectively about it. There has to be a clear sense of priority—everything else follows from that.
It would be odd for me to come to Parliament to defend politicians—you can do that yourselves. However, if we are not leaving things to the market—in which case I decide about my preferences and choices—but doing things collectively in a democracy, we will elect people who will appoint people to health boards and other bodies to make collective decisions on our behalf.
Unquestionably, part of making a decision is asking whether there is evidence for it. Another part might be saying, “We, as a community, just don’t like this,” in exactly the same way as I do not buy food that I dislike the taste of not because of a rational case for not buying it but because I do not like it.
The most spectacular example of that is probably the Kerr report from back in 2005, which contained an entirely rational analysis of the Scottish health service and was evidence based. It said that we should remove quite a number of accident and emergency units and reconfigure hospitals, but it was utterly hated by a lot of the public and by a significant number of clinicians whom the changes would affect. The honest truth is that it was probably done in by people who marched down streets and said, “Hands off our A and E unit!” That could be said to be utterly irrational, but such services are provided for and on behalf of the public and I suspect that if the public do not like something we must factor that in.
Politicians have an important role in reflecting public opinion, some of which is not entirely rational but concerns values, feelings about an area, historical sensitivities and so on. That is part and parcel of how we make decisions in our private lives, so it would be profoundly unlikely for that not to be part and parcel of other decisions. As long as politicians are well engaged with the communities that they serve—that is critical—some of the non-rational and non-evidence-based stuff will be important.
Often, one of our difficulties—Fraser McKinlay alluded to it—is that we have a lot of policy-led evidence. Because we have done something, we have evidence about it. Therefore, we often end up not with evidence-led policy but with policy-led evidence. That is almost inevitable in a mature system of public services. The challenge that Gareth Davies talked about therefore remains important. There is opposition at every level of the system that will challenge whether the evidence is good enough and whether the interpretation of it is reasonable. Evidence rarely resolves anything unless active people do things with it as part of scrutiny and challenge processes.
I will talk briefly about one aspect that might cut slightly across or cause difficulties for CPPs’ role. In responding to the independent commission on strengthening local democracy, we suggested that defining what locality means for each service should be considered. We probably would not design a trunk road network for Scotland by thinking about 5km blocks of that at a locality level. When we talk about services, we need to be clear about the scale of locality that is meant and the area that is being serviced. That might bring in the challenge of deciding whether the locality is the local town or the region, for example. What locality means for each service might be different.
As has been mentioned, Audit Scotland’s submission says that we should
“map the pathways that connect each portfolio’s contribution to the national outcomes.”
At first blush, I whole-heartedly agree with that statement. However, after listening to the evidence, I wonder how achievable and useful such an exercise might be, particularly when the witnesses have said that housing, for example, touches on everything. Would we end up with a diagram that showed that housing linked to all 50 national indicators? I initially agreed with the statement, but how useful would the exercise be?
That is a fair question. As you said, the statement is probably easy to make and more difficult to achieve. The exercise would need to be done in a way that avoided what you described with the housing example. We could end up with a complicated spider diagram in which everything was connected to everything else. In a sense, that is always the case, so the predominant links—where the biggest changes and differences would be made—would need to be identified.
I am sure that Gareth Davies will have a view on this, but my view, whether we are talking about outcomes-based, performance-based or zero-based budgeting and so on, is that it is not an exact science. We can get a wee bit sucked into thinking of it as an exact science, when it is in fact us trying to shift ourselves along the road a bit from an incremental approach to budgeting, in which we figure out what we spent last year and take 3 per cent off that. We are getting closer to linking what we are trying to achieve overall with the resources that we are using. It will never be perfect. Gareth Davies spoke about causality. We will never get there and nor should we try—we would just tie ourselves in knots—but it is worth having a go.
The committee may be aware of some interesting work on this issue at the Royal Botanic Garden Edinburgh, believe it or not. The director of corporate services there has been doing a lot of thinking with the University of Edinburgh and others about exactly how to do that kind of budgeting. They are looking at it from the organisational perspective, which is probably a bit more straightforward, because it is about that organisation’s contribution and mapping that pathway all the way up. As a methodology and an approach, however, it looks quite interesting.
Modern apprenticeships came up earlier as an example of the sort of outcomes that we might look for. I think Audit Scotland’s view was that we were focusing heavily on existing performance measures but that there was little focus on long-term outcomes. Will you expand on that? Imagine that it was your role alone to decide what measures there ought to be. You are the cabinet secretary and director in charge of it and it is entirely up to you. Without being exhaustive, will you give us examples of the sort of measures that ought to be in there if we are going to treat it seriously?
I cannot possibly imagine what it would be like to be in such lofty positions.
Our challenge to the Government on modern apprenticeships, as I said earlier, was to ask what difference the number of 25,000 modern apprenticeships would make. I will not get the exact wording here because I cannot remember the detail, but if the overall ambition for modern apprenticeships is to contribute meaningfully to sustainable economic growth, how will having 25,000—as opposed to 20,000 or 15,000—do that? As well as an input measure, which I have no difficulty with, we need to devise a measure of the net result and effect. That is really important, Mr Brown, because that leads us to much more qualitative measures of the nature of those apprenticeships. What does a good apprenticeship look like? What is the outcome of a good apprenticeship? Where does it end up? Once they have done their apprenticeship, to what extent do these young people—in the main—stay in employment and progress through employment for five, 10 or 15 years into the future?
If we start from the other end, we begin to ask more challenging questions about not just the number of apprenticeships—important as that is—but what it is about those apprenticeships that will really make a difference and therefore what activity we need to put in place to support them. What is our methodology for that and what is the measure in the end? I cannot, off the top of my head, think of the exact measures but, as a process, that is how I would describe it.
I think that we are running over so I will try to roll my questions up into one. This has been a really interesting session and I have got a lot out of it.
My question is about the national performance framework. Everyone praises it, yet I took the Improvement Service paper and most of Colin Mair’s comments as quite a radical critique of it. That is certainly how I read your paper and heard some of your comments, Colin. What do you think we need to do it? In a sense, you have redefined outcomes. In one part of your paper you say that what we call outcomes are sometimes really objectives. I take it that you mean that we need to focus our attention more on a smaller number of outcomes. Perhaps you can clarify whether that is the case.
I suppose that the related question, since I said I was going to roll my questions up into one, is the extent to which, if we do need to revise the national performance framework and make it far more focused on a smaller number of outcomes that are really about improving people’s lives—particularly, to my mind, and probably to yours, the lives of those in our society who are more disadvantaged—to what extent do we have to drive that through into community planning partnerships and local authorities? There is always a tension between localism and national objectives that we are all committed to. That is me trying to roll my thoughts into one question.
11:45
I did not write the paper as a radical critique of the national performance framework, but if that is how it has come across, I suppose that it can be interpreted that way.
The point that I wanted to make in the paper is that we have a framework that is useful and has shifted the culture in a way that gets people to focus outwards. That is good. However, if you, at the national level, wish to make a contribution to change—I would imagine that you would—the question is, what are your ambitions and expectations? One of the challenges that community planning partnerships face is the number of measures that have no targets associated with them. I think that that comes through in the Audit Scotland work. We want to make the place healthier, but we cannot really say by when we want that to happen. If that is a legitimate challenge at the local level, it would seem to be a legitimate challenge to parliamentarians as well. What is your ambition and your expectation of the public services that you govern in Scotland?
I would welcome narrowing to a tighter focus and I would start with the strategic objectives in the national performance framework. People understand the objectives of making Scotland wealthier, healthier, fairer, smarter and greener. That is fine, but what flows out of that in terms of your priorities over a period of time, and how do your priorities at that level sit with the various political priorities around inputs, outputs, service standards and so on? All of that needs reconciled. That is what we are asking the community planning partnerships to do, and we are sometimes quite critical if they do not do it. In that case, ditto for Parliament. You should have a view about key outcomes that you want achieved, timescales and targets that you are willing to commit to and be accountable for, and a view about how those things fit with issues such as pupil teacher ratios, never being able to close a primary school and so on. You have to consider how you balance those political commitments with the key outcomes.
I am sympathetic to the idea that reducing inequality should be at the heart of what you do. Making things fairer cuts across everything else. It is not a separate set of outcomes. For example, we need to be greener in a fair way—people should enjoy a quality environment consistently across Scotland—we need to be wealthier in a fairer way, and so on.
Even if we forgot all about the national performance framework in this discussion, we should acknowledge that it has spawned a lot of on-going work. Big things are going on around health inequalities and the educational attainment gap in Scotland. Why are those problems arising? How do we combat them? Those issues are not explicitly referred to in the national performance framework, but they unquestionably flow out of an outcomes focus. The framework has driven quite a lot of on-going work that will, no doubt, be reported on to various parliamentary committees in due course.
I would stress those two points. First, we should be more dynamic, and the national level should make a contribution by saying, “Here are our expectations for the system, and we are willing to be accountable for them.” Secondly, we should be aware that a lot of stuff that is not formally within the national performance framework has been driven by it.
That has exhausted questions from the committee. I have a few further questions but I will impose a self-denying ordinance, given the time that has elapsed since the beginning of the meeting—that will be a matter of relief for the bill team in the public gallery.
Do any of the witnesses have any further points that have not been touched on?
I want to emphasise something that was touched on briefly.
We are doing a bit of work on how councils and the community planning partners use their economic capacity as employers, procurers and asset holders. We are doing quite a lot of measurement around that—with a group of councils at first, before rolling it out—to see whether that is a resource that we are underusing with regard to achieving outcomes. Could we use those capacities much more forcefully to achieve the outcomes that we say are priorities for our areas? That bit of work is on-going across the next year, but the pilot stage will come to an end in a couple of months’ time. We would welcome your interest in that work and would be happy to report to you on the pilot stage.
We should not neglect that economic issue. Public services are big economic entities and should partially be judged by their economic impacts.
Given the interest in the committee about some of the issues that you have raised, we would be interested in that.
CIPFA is planning to work with a charity partner on developing a practical structure for an outcomes approach and outcomes budgeting and also for the assessment of preventative interventions. We, too, would welcome the opportunity to keep you updated on that.
I want to close by returning to the issue that I started with. CIPFA sees an outcomes focus as part of good governance. You might not want to read the whole international framework document, but I can sum it up in this sentence: act in the public interest at all times. If people and organisations have that focus, that will take us a long way towards where we want to be.
One of the striking things about this session and, indeed, the whole debate, is how much agreement there is—in the break, Colin Mair and I were saying that we need to find something to disagree about today.
It seems to me that people absolutely sign up to the analysis of what the issues are and what needs to be done. I welcome the enormous amount of work that is being done to try to make things happen, and happen more quickly. That is the key point for us. We need to put in place the infrastructure, the processes, the systems, the people and everything else that is needed to take things to the next stage.
Given that your continuing interest will be around scrutiny of the budget, we will continue to push, challenge and support the Scottish Government—and future Scottish Governments—to do more to link outcomes with budget and spend. If there is anything else that we can do to support the committee in that, we will be delighted to do so.
I thank all our witnesses for answering questions so comprehensively today, and I thank my colleagues for their questions. It has been a fascinating session.
We will have a five-minute suspension to allow our witnesses to change over.
11:52 Meeting suspended.