This briefing helps parliamentary committees and SPICe researchers use the National Performance Framework (NPF) to improve committee scrutiny of the Scottish Government budget. It also assesses recent changes made to the NPF and summarises developments over the 12 year history of the Framework.
In light of the Budget Process Review Group's report published last year, and recent changes made to the National Performance Framework (NPF), this briefing sets out some practical suggestions on how Parliamentary committees can use the NPF to improve their budget scrutiny.
It demonstrates how committees can:
use the NPF to help provide a focus to budget scrutiny;
identify key questions relating to the different stages of the policy/spending process;
link these key budget questions to the NPF's outcomes and indicators;
use the NPF to improve the depth and scope of budget scrutiny.
The briefing also provides some background information on the original aims of the NPF, as well as a summary and assessment of recent changes.
The Budget Process Review Group, set up by the Scottish Government and Parliament in 2016, recommended that the National Performance Framework (NPF) be used more widely by the Parliament to evaluate the impact of Scottish Government budgets.
Its report and recommendations were agreed by Parliament in May 2018, with the Finance and Constitution Committee subsequently sending budget guidance to each of the Parliament's subject committees, stressing the importance of performance information to effective budget scrutiny.
However, during last year's budget scrutiny process, of the Parliament's eleven subject committees only four actually mentioned the NPF in their budget reports or in correspondence with ministers.
If "outcomes-based budget scrutiny" is to be anything more than a clever sounding piece of public policy theory, then it is important to note some practical ways in which the Parliament can incorporate the NPF into its scrutiny. This short briefing aims to do this. It also provides some background information on the history of the Framework and assesses the extent to which recent changes made to the NPF have helped or hindered budget scrutiny.
At the start of the third Session of the Scottish Parliament, and as part of the 2007 Spending Review, the Scottish Government introduced a new outcomes-based National Performance Framework (NPF) to underpin the delivery of its agenda. In June 2008, the Government launched Scotland Performs, a website designed to present information on how Scotland is performing against the range of indicators and targets outlined in the NPF.
When the NPF first appeared it was hailed as a “world-leading example of a wellbeing approach to government”. Although inspired by the existing performance measurement system used in the US state of Virginia, Scotland’s NPF was regarded as genuinely innovative by many commentators at the time. It’s fair to say expectations were high, with Oxfam telling our Finance Committee in 2011:
“The National Performance Framework offers an exciting opportunity for Scotland to bolster its leadership in innovation in socio-economic policy and progressive legislation. Through the NPF, Scotland can develop the most coherent and forward-thinking framework for orientating and evaluating public policy in accordance with social, environmental and economic prosperity.”
The Government billed its NPF as a “10-year vision” for Scotland, with the original Framework centred around the Government’s purpose. This was supported by 11 “purpose targets”, 5 "strategic objectives" and 55 "national indicators".
For more information on the development and original aims of the NPF, see Allan Campbell's National Performance Framework and Scotland Performs briefing from 2012.
The Scottish Government is required to review its National Outcomes every five years, as set out in Part 1 of the Community Empowerment (Scotland) Act 2015). During the first review in 2018, the Government also took the opportunity to change the structure of the NPF and implement a series of other changes. Legislation only requires the Government to consult on the National Outcomes, so the Parliament neither accepted nor rejected the new NPF structure, layout or content. Indeed, during the subsequent Parliamentary debate various MSPs voiced some concerns about the new NPF.
Some key differences between the original NPF and the new version are:
Removal of purpose targets: the Scottish Parliament's Economy, Jobs and Fair Work Committee was surprised to find that all time-limited targets had been removed from the NPF. The Committee asked “how will the impact of policy be measured if we are moving away from the previous specific time-based purpose targets? What will the benchmark be?”. The Scottish Government's Chief Economist confirmed that the new NPF will not have any time-specific commitments “because it is about continuous improvement”.
Simplified structure: as well as removing the Purpose Targets, the new NPF no longer includes Strategic Objectives. The Government argued that the old version was overly complex, with people finding it difficult "to understand the relationship between the different layers of the NPF and wanted a simpler and more accessible framework"1 . The new version includes a revised Purpose, 11 National Outcomes and 81 National Indicators which feed in to the Outcomes.
Presentation: the new NPF website has also been simplified, to the extent that some useful information is no longer available. Previously, the National Indicators and National Outcomes web pages included useful narratives such as “what is the Government’s role”, “why is this national indicator important”, “how is Scotland performing” and “who are our partners”. These are no longer included on the new site. Also removed are technical notes which had previously explained the statistics used for each indicator.
UN Sustainable Development Goals: the new NPF makes links between the new National Outcomes and the UN's Sustainable Development Goals (SDGs). The UK Government signed up to the SDGs in 2015 and the Scottish Government committed to implementing them through its National Performance Framework.
For more information on the background to the new NPF, and the consultation process leading to the Parliament agreeing the new National Outcomes, see Ishani Erasmus's National Outcomes Consultation 2018 SPICe briefing.
The budget process, and policy-making in general, is usually incremental in nature (see Rose, 19901, and Baumgartner, 20122). In other words, Governments tend to continue programmes over a number of years, tweaking policies here and there, but with budget lines generally inheriting levels seen previously.
There are generally only marginal differences in departmental budgets between years - the Government will normally (although not always) allocate a little bit less or a little bit more to its various policy areas.
Realistically, large portions of the Scottish budget are always going to be allocated to health, local government and other core areas. Therefore, the budget process offers only limited scope for choice and change, and even then it is usually at the margins.
Nevertheless, budget scrutiny by the Parliament is an important challenge function to this process of policy continuity and budget repetition. At the very least, it provides an opportunity for Ministers to reflect on and justify to Parliament exactly why they are continuing to fund certain programmes and organisations, and crucially what outcomes they are hoping to achieve by spending public money in these areas.
The Finance Committee, in its guidance to subject committees, recognises that subject committees cannot undertake detailed budget scrutiny in every area within their remits. It therefore recommends that they agree a focus each year, either on a specific policy area, or indeed budget areas that have seen significant reductions or increases between budgets. Furthermore, if the committee identifies national outcomes or indicators that are seeing little or no progress (or indeed a deterioration), then it could be useful for committees to focus on budget areas that may impact on those outcomes.
For its 2020-21 pre-budget scrutiny, the Economy, Energy and Fair Work Committee decided to focus on Regional Selective Assistance (RSA), Scotland's main grant scheme for business. RSA is managed and delivered by Scottish Enterprise, and is available for capital expenditure and investment projects which will create and/or safeguard jobs.
As recommended by the Finance Committee, one of the main reasons for focussing on this area is that there has been a significant change in the RSA budget over recent years - a 50% real terms reduction between 2013-14 and 2017-18. The Committee will explore what impact this has had on the Government's national indicators or outcomes.
Another reason for selecting this area for scrutiny is that Committee work earlier this year had questioned whether RSA funding is always consistent with the Government's commitments to fair work and inclusive growth. Questions arose about value for money, especially when RSA funding has gone to businesses that have subsequently collapsed.
Finally, looking at the NPF's National Indicators Performance Overview, it is striking that none of the eight indicators relating to the business and fair work National Outcome show any improvement over the most recent reporting period. Three indicators - the percentage of high growth businesses, the total number of businesses and the percentage of innovation-active businesses - have actually seen performance worsening, and the living wage and employee voice indicators have both seen little change over recent years despite being the subject of considerable policy focus.
Once a focus for budget scrutiny has been selected, it is important to identify questions the Committee should be looking to answer during its inquiry. The following list provides some examples of key budget questions committees could incorporate into their scrutiny:
Pre-budget decision making:
How was the amount of budget decided upon? Is it simply the same as last year's allocation?
What research or consultation led to the budget decision?
Why did the Government or government agency decide to spend (or continue to spend) money in this area rather than direct money into another area?
What equality and carbon impact assessments have been undertaken during the decision-making process?
What issue/problem was the policy/programme hoping to address?
Was evaluation built in to the programme from the beginning?
Delivery:
Who decides how money is allocated at a delivery level? Do service users have to meet certain eligibility criteria?
How did recipients hear about these funds or programmes? What efforts were made to ensure all potential beneficiaries know about available funds and benefits?
Was any public procurement exercise followed?
The application and approval process from the recipient's perspective – is it too bureaucratic or burdensome? Does it favour larger organisation?
With limited resource, how were priorities made at a delivery level?
Counterfactual:
What are the opportunity costs; i.e what other policy options were considered but rejected? What else could this money have been spent on?
Who didn’t receive funding who perhaps should/could have?
Evaluation and lessons learnt:
What did this money buy in terms of inputs and outputs (e.g. number of grants awarded, training courses delivered, homes insulated, children inoculated, etc.)?
How have these inputs and outputs contributed to national outcomes?
What has been the customer/user experience of the support they've received - do they feel they've received a quality service?
Which regions, sectors, communities, etc have benefited most from the spend?
What evaluation took place during and after delivery/spend?
If the programme was continued, why? What lessons were learnt and how has the programme changed as a result?
How much money was committed, how much was spent - was there an underspend, or conversely was there sufficient funding to meet demand/need?
These suggestions are simply meant as a guide rather than a definitive list. Each committee, and each inquiry, will have their own areas and questions they wish to see answered.
Returning to the case study, the following NPF indicators are of particular interest to the Economy, Energy and Fair Work Committee, and it is possible to see that many of them could be impacted by a programme of funding like Regional Selective Assistance.
| High growth businesses | Productivity |
| Innovative businesses | Employee voice |
| Economic participation | Carbon footprint |
| Employees on the living wage | Research and development spend |
| Gender pay gap | Contractually secure work |
| International exporting | Economic growth |
Taking a selection of the questions listed in the above section, it is possible to frame some of these within the context of the National Indicators. For example (relevant indicators highlighted):
To what extent are the views of employees in recipient businesses taken into account when 1. designing the RSA eligibility criteria, and 2. deciding how money is invested within companies (employee voice).
Is the gender pay gap, contractual security or the guarantee of paying the living wage to employees considered when deciding whether or not to approve grants?
What assessment is made of the carbon footprint of new technologies funded by RSA grants?
Do RSA grants have any impact on the level of research and development seen in recipient businesses?
Given the lack of progress in the number of high growth businesses and the number of innovative businesses in Scotland, why was it decided to reduce the Regional Selective Assistance budget?
What impact has RSA had on the productivity and international exports of recipient companies?
Whether sufficient numbers of smaller high growth businesses apply for RSA, or does the application process favour larger, more established businesses?
In the process of answering these, the Committee can engage with a whole range of individuals, organisations and businesses, for example workers and management within recipient businesses, Scottish Enterprise staff, academics, business organisations, trade unions, etc.
From the evidence received, it should be possible for the the Committee to take a view on the extent to which RSA has contributed to the Scottish Government's indicators, and the following related National Outcomes:
We have thriving and innovative businesses, with quality jobs and fair work for everyone.
We have a globally competitive, entrepreneurial, inclusive and sustainable economy.
The Committee may come to the conclusion that RSA makes a positive contribution to these outcomes and that its budget should be maintained or increased. It may take the view that the recent reduction in funding has had a negative impact on national indicators, or, alternatively, it may take the view that the money could be better spent elsewhere.
The Scottish Government published its NPF "baseline" report, Scotland’s Wellbeing – Delivering the National Outcomes1 in May, to coincide with its Medium Term Financial Strategy. It aims to bring together existing evidence and analysis on a number of key issues, trends and features of Scotland's performance which the Government feels are “important to consider when making decisions on policy, services and spending.”
The baseline report provides a snapshot of current performance across a range of outcomes and indicators. The chapters are structured around six broad headings:
Scotland's people and reputation.
Scotland's natural and economic resources.
Fair and equitable working society.
Educated, skilled and realising our full potential.
Living healthy lives.
Community well-being in Scotland.
These do not correspond to Government departments or Parliament committees, so it is not clear how Committees can focus on those chapters most relevant to their remits. Furthermore, the report does not include commentary on policy or Government spending decisions. Committees may struggle to see how the baseline report, or indeed the new NPF website, supports efforts to identify linkages between budget decisions and outcomes as recommended by the Budget Process Review Group.
In fairness, the report does not claim to measure the contribution of Government policies to the delivery of national outcomes. Instead, it is more a gateway, or a “starting point”, in to the large range of social, environmental and economic data used in the NPF and beyond, and an accessible and well-illustrated snapshot of how Scotland is currently performing across these indicators.
For example, the baseline report states there has been a reduction in business innovation activity over the latest period – with the share of businesses innovation-active decreasing from 50.4% in the 2012-14 period to 45.0% in the 2014-16 period. It does not explain what the Scottish Government intends to do about this, nor how much of its budget is currently spent trying to change the situation.
Instead, it will be up to Parliament's committees to assess the effectiveness of budgets in achieving outcomes, and recommend improvements based on evidence received during the pre-budget scrutiny process . Hopefully, some of the suggestions and examples in this briefing can help them do this.