Economy, Energy and Fair Work Committee

Meeting date: Tuesday, September 10, 2019

Official Report 660KB pdf

---

Pre-Budget Scrutiny 2020-21

I welcome our second panel of witnesses as we move to item 5 on the agenda, which is pre-budget scrutiny. Starting from my left, from Scottish Enterprise, we have with us today Dr Wendy Hanson, who is the team leader for grant appraisal, and Kenny Richmond, who is the head of insight and economics. From Highlands and Islands Enterprise, we have Lorna Gregson-MacLeod, who is the head of planning and partnerships, and, last but not least, Rachel Hunter, who is the director of service delivery. I welcome all four of you and thank you for coming in today.

I will start by asking some questions about budget scrutiny and our understanding of the evaluation that you do, which requires familiarity with a number of different concepts. Those concepts include gross to net, the difference between direct, indirect and induced impacts and the forecasting of future impacts, in which trying to look into the future is fraught with difficulties. All of that can be confusing for people who are assessing what your agencies are doing. Does that prevent proper scrutiny?

I do not think that it does, because we use a mix of appraisal and evidence to consider what the potential future benefits and impacts of activities could be. We then mix that with evaluation activity, which considers what the benefits and impacts actually were. Using both approaches gives us a good steer on the type of impact that we are getting from the investments that we are making.

Scottish Enterprise produced publicly available guidance on the evaluation of economic impact. Are you aware of other agencies that have done that?

The guidance that we produced draws on best practice in the form of the appraisal and evaluation guidance that has been produced by HM Treasury and the United Kingdom Government. However, although it draws on that, we have adapted it to make it relevant to the types of activity that Scottish Enterprise and Highlands and Islands Enterprise provide. While following best practice methodologies, it is really tailored to the types of activity that we do.

Does it also relate to what the Scottish Government’s analytical unit does?

Indeed. We look at the same types of indicators and measures, and we consider gross and net impacts. Some of the work that we will do with the Scottish Government’s analytical unit around consideration of our future evaluation approaches will make sure that those methodologies are followed.

Does anyone else wish to comment on that topic?

My comments follow on from Kenny Richmond’s point. We do many evaluations using those common methods, and we look beyond Scotland for good practice. The characteristics of our region make it unique. However, it is really important to us that we learn from approaches in other areas. We do some cross-border EU project work, and we are looking at approaches to supporting and evaluating that.

For example, at the moment, we are carrying out a female entrepreneurship project called W-power, working with people in Sweden, Ireland and other areas to better evaluate and test what we are doing with female entrepreneurs, to see whether it is the right approach and how we can measure the benefits. That is broader than just looking at jobs and turnover—it also involves behaviours and access issues.

We also lean a bit more heavily on the qualitative evaluation approach, because of the nature of our region, the lack of statistics and the difficulties in finding control groups and so on. We get a lot of value from that qualitative and cross-border work.

Do you think that the way in which you present your evaluation makes it accessible to those who may not have the skill that others have for understanding such evaluations?

They are technical documents, but all our evaluations have an executive summary that is very accessible. The language is accessible, and the summaries are not very heavy on tables and statistics where they do not need to be—they try to draw out wider community and social benefits, and they draw attention to business behaviours. The views of the businesses, social enterprises and communities that we are dealing with are very important, and the approach of using an executive summary alongside the fuller report makes the evaluations more accessible.

Given that the money that goes to enterprise agencies and the amount of money that the agencies give to companies is sometimes a little bit contentious and needs to deliver value for the public pound, can you give us any examples of evaluations that you have conducted that have led to significant changes in the way in which you deliver grants to the private sector?

That is a great question. We should never have to wait for an evaluation to know whether a project is delivering at its optimum. Good project management during the life of a project means that we are getting real-time information as we go. We tend not to see evaluations as providing us with startling or brand new insights that mean that we have to make a significant change, because we are working with our management information and evidence in real time. Evaluation evidence tends to provide indications of where we may want to tweak things and gives us evidence of where we are making the biggest impact.

All our evaluations tend to include a set of recommendations, and we pull together action plans around that. Those tend to be tweaks around the edges rather than more fundamental changes.

We have some examples of that. In our evaluations we always ask for recommendations for how to flex our approach if need be. That is a critical part of our evaluation material. One example over the past five years might be our community-led development evaluation, in which the evaluators pinpointed potential changes to our measurement frameworks and what data and measures they capture. They suggested that we should be looking at more long-term impact assessment, because a lot of that is about the communities generating income from assets, which can take some time. For community-led development, there were several key recommendations on how we could improve value for money in relation to where we put our expenditure and on looking to put more of it into the assets that can generate income for communities to bring stability and build growth. We have taken that on board and we want to do more work to tease out that long-term impact model. We are currently carrying out a pilot project on that.

I have a specific question about the people who undertake the evaluations, as many of the evaluations are contracted to the private sector. How do you ensure that private contractors are giving you honest and robust evidence, and that the evaluations are not skewed by the contractor’s commercial interest in getting more work from you?

All the contractors follow the best practice methodologies. In that respect, we hope that they are looking at things effectively. We do not find that contractors are looking to change the results or evidence either to please us or to please businesses. Sometimes, when we get recommendations back, they can be slightly challenging, although without suggesting dramatic changes. In addition, when our contractors speak to businesses, we ensure that any evidence is anonymous so that businesses feel okay to talk honestly and to provide objective views. That is translated through our contractors. We are pretty confident that we are getting objective views from our contractors.

11:00  

Okay. As there are no other views on that, I will move on to a question about your evaluation online portal. Over the past 15 or so years, the level of evaluations has significantly decreased. Prior to 2010, Scottish Enterprise alone was averaging about 40 per year; that has now fallen to about five. In 2019, there have been no reports, and in 2018 there were just two. Is there any particular explanation for that?

More and more of our evaluation evidence is highlighting that businesses benefit from a combination of support rather than from just one specific programme or project. Therefore, we have increasingly been moving away from contracting evaluations that focus on a specific project and have been looking instead at evaluations that consider a bundle of activities. In recent years, that has included the evaluation of internationalisation and exporting support, which considered a whole range of projects through which companies were supported. In the past, we perhaps would have evaluated those as individual projects, but our evidence shows that businesses benefit from that combination of support. Therefore, our evaluations are evaluating that combination of support more and more.

We also do internal process evaluations of individual projects that consider management information and so on to get a handle on exactly how the individual projects are operating. However, for large impact studies, it makes no sense to consider those projects individually, as the support network for business is more complex in terms of the types of support that are needed and how that combination works.

Nonetheless, can you clarify that that is not constraining your scrutiny efforts in relation to the decisions that you make about, for example, regional selective assistance?

That is not the case at all. For example, we have considered RSA and the evaluation of grants as a whole. We have been doing some evaluation evidence—which is on-going—that considers our grant support as a whole, including RSA, research and development grants and training-plus grants.

That is an example of why we are looking across the piece rather than focusing evaluation on a specific product. Marrying that with the on-going management information that companies receive gives us a good steer on both the overall impact of that combination and on how the individual grants are working in terms of activities.

The Scottish budget process obviously is—and has been—evolving, which has resulted in a far more outcome-based approach to scrutiny. Have you had to modify your own measurement frameworks to reflect that? If so, how?

I will cover Scottish Enterprise.

We are making sure that our performance measurement framework links us closely with the Scottish Government’s national performance framework. For example, we have headline target measures around jobs, R and D and exports, all of which link to the Scottish Government’s national performance framework.

Beneath that, where possible, we also link our internal management information to the national performance framework, which gives us an idea of the activities and outputs as well as the longer-term outcomes. We have, therefore, modified our approach to link with the national performance framework.

How difficult has it been to make that change? Every time you talk about statistics, it seems to be enormously difficult—first, to collect them and, secondly, to collect them in the form that you want. How big a transformation has that been?

It has not been a significant transformation, because we have been monitoring a lot of the measures in the national performance framework for a number of years. The measures that we consider are focused primarily on the drivers of productivity and creating quality jobs. Those drivers have been known for a number of years and we have been monitoring them for a number of years, so it has not been a significant change for us.

Similar to the situation that Kenny Richmond outlined, Highlands and Islands Enterprise has always had an alignment with the national performance framework. As a result of the Enterprise and Skills Strategic Board work, we are getting closer alignment between the agencies. A common performance framework is under development, but it has not resulted in a lot of new measures. When it comes to how and when we collect the data, there has been some requirement to adjust measurements. It is not a huge impact. In relation to the social community impact and the inclusive growth model that we want to pursue, some of the things that we would like to do as an agency require system changes. However, that will be woven into our business transformation programme, which is under way. That requires adjustment both to information technology and to data collection. It will take a number of years to come through, but that alignment is there.

You mentioned inclusive growth. Nowadays, that is a key theme in Scottish Government policies. How do you measure that?

Highlands and Islands Enterprise and Scottish Enterprise have a range of target measures that relate to matters such as community investments, generating income from assets in communities and average wages. That last relatively new measure has been in place for the past three years. We publish those targets and we track them. We track the quality of jobs and their salaries.

We also look at where the investment is being made. We have a strong place-based approach. We are looking not just at the numbers but at where the impact is felt across our region. We have particular targets for fragile areas. We will work with Scottish Enterprise and the emerging south of Scotland enterprise agency on developing the pilot more accurately to measure inclusive growth.

Numbers alone do not tell the full story. Our board is keen that we look at ways of weighting our measures. For example, three jobs in South Uist have a far greater benefit and impact than three jobs in the middle of Inverness. We need to account for that. We have been doing a lot of data modelling and will be working to develop that pilot to work out ways of weighting for inclusive growth impact on our range of measures. However, formally, we will probably focus first on jobs.

Given the nature of the measures that you are dealing with, inclusive growth can be difficult to evaluate. You mentioned the quality of jobs. Do you evaluate that on the basis of salaries?

Yes, that is one mechanism. For example, our business plan target this year is related to jobs that pay at least the real living wage. However, we also monitor high-value jobs, which we define as jobs that pay 20 per cent above the Scottish average wage. For instance, we monitor R and D jobs, which are higher-skilled, quality jobs.

As Lorna Gregson-MacLeod mentioned, for the companies with whom we work most closely, we also track the extent to which they follow, for example, the Scottish business pledge criteria. That is like a proxy for quality jobs. A quality company that follows those pledge elements is more likely to offer quality jobs.

You are right in saying that there is no single measure of a quality job or inclusive growth, so we have to look at that basket of measures, and the Scottish business pledge elements are a good proxy for that.

It is important that statistics are constant across the public sector. Is the way that you collect statistics on inclusivity consistent with what is done elsewhere in the public sector?

That is another good question. The introduction of the national performance framework, which is a number of measures that are proxies for inclusivity, will drive different parts of the public sector—at least in Scotland—to follow the same types of measures. The bigger challenge is how we compare what is happening in Scotland to what is happening in other parts of the UK or other parts of the world, where there is still no consistent definition of inclusive growth. That makes it more of a challenge, but at least we can start to coalesce around the national performance framework and take that consistent approach.

We should be satisfied if, to start with, we can get the approach consistent across Scotland. From what you say, it sounds as though there are different approaches across the public sector. Is that correct? To some extent, the national performance framework might be starting to pull it together but, when we get down to the base and look at evaluations, is there a difference between what you and other areas in the public sector do?

I am not aware of what evaluation approaches other parts of the public sector are taking in looking at inclusive growth specifically.

So you have developed your own model.

Yes. We are working jointly with Highlands and Islands Enterprise and are looking to ensure that the approach that we take is as consistent as possible. Through the analytical units and enterprise strategic boards, we will be looking to ensure that we and the other agencies—Skills Development Scotland, the Scottish Further and Higher Education Funding Council, Highlands and Islands Enterprise and south of Scotland enterprise—use similar approaches.

Although it is commendable that you are developing all that, I would have hoped that there would be a uniform approach right across the public sector.

The Scottish Government has been leading work on developing an inclusive growth diagnostic that looks at the range of inclusive growth measures that all parts of the public sector can use, including us. There is a movement towards having a more comprehensive view of inclusive growth.

Are we moving towards having not only the same measurements but the same evaluation methodology?

We still have some way to go on that, but we are definitely getting there. Progress is definitely being made.

Perhaps I should have said at the start that the sound system is operated for you, so there is no need to press any buttons. If any members of the panel want to say something, they should simply indicate that by raising their hand, and I will try to bring them in.

We would like to be able to verify some of the activity that you highlight in your annual reports. For instance, in relation to regional selective assistance, Scottish Enterprise has shown that the trends in the number of offers accepted, planned capital expenditure and the number of jobs are upwards. However, if we took one year out of that trend—for example, 2016-17, when 65 offers were accepted and there was £37.5 million of capital expenditure and 1,600 jobs—how could we as a committee verify those numbers subsequently?

There is verification of what happens in a project. During the course of a project, we track and monitor the conditions to do with jobs and capital expenditure. That goes on all the way through the life of the project.

The project will enter a conditions period, which depends on the type of grant that was awarded. For job creation grants, we look at a three-year condition period for small and medium-sized enterprises and five years for large companies for them to maintain all the jobs in post for that length of time if, from the initial investment, those jobs are 18 months from the final instalment being paid. Research and development grants tend to vary according to the size of the company. There could be one year or two years on job conditions, for example.

To stick with RSA in particular, once we enter the condition monitoring period, we still regularly meet the company either via the account management route, colleagues in the business gateway or members of the team, who go out and assess whether the company is still doing what it said it was going to do and that it has maintained the jobs for the period that it said it would.

The historical evidence that we have on RSA to date is that the companies not only create the jobs that they have said that they would create, and keep them; in general, there is around an extra 10 per cent leverage on top of that in respect of other jobs that have been formed on the back of the RSA. It probably overdelivers against what we predicted and forecast at the beginning of the grant.

Obviously, if we ran into an issue with a company that was not doing what it said it would do, we could look at the conditions of the contract and consider clawing back funding appropriately relative to the number of jobs that there are compared with what it said there would be.

There are mechanisms for us to be absolutely spot on and know what the company did. The payment profile is a good real-time reckoner for us because, if we pay out against what we contracted to do, the company is definitely delivering what it said it would deliver.

11:15  

I accept that you guys monitor companies’ performance and claw back funding, where necessary. I also accept that it can take three to five years before whatever was planned in relation to jobs and capital expenditure is delivered. My question is this: where can we see that evidence? In 2012-13, for example, you will have said, “The planned number of jobs is X and the expected capital expenditure is Y.” Five or six years down the line, where in the annual reports can we see what actually happened? I think that you said that you get a 10 per cent bigger bang for your buck. Where do we see that?

We do not publish that information, but we can pull it together and share it with the committee. Basically, it is just management information. It is not something that we hide, but we do not publish it, I guess because we have not had requests for it in the past. We can easily share it with the committee.

If you are saying that you are delivering 10 per cent more than you expected to deliver, surely that should be something to shout about.

Yes, potentially.

Yes, it is a good-news story. We have never been asked to report on that statistic, but we track it internally, project by project. It struck the organisation that it would probably be useful to track it on a more global basis for different grants and products. There are many measures that we could collect; we could certainly produce something that would give the committee the confidence that it is looking for.

Thank you.

HIE said in its annual report:

“We approved £54.6m to support 564 new projects representing a total combined investment of £185m. This will deliver well over 1,000 jobs”.

How did you arrive at those numbers?

The analysis in the annual report is captured at the time when the investments are made, through the economic impact assessment work that we described in our submission. That is the data that goes into the annual report. As you said, it takes time before the actuals come through. We capture actual data at the end of year 3, for most things, although for things such as distilleries, which require a longer stream before they come to fruition, we take a different approach.

Audit Scotland highlighted the issue that you raised. It is a good point. In the data in the annual report, we talk about what has been approved to be forecast—if you like. This year is a holding year, in that we have committed to doing a bit more analysis, so that we bring in more of the actual data next year, because such an approach is necessary.

Given that doing things on a year-by-year basis is not that helpful, we tend to do the analysis over three or five years, which provides a richer picture and a story to tell. We do that regularly, looking at the internal regional impact and how different measures are playing out across different types of investment, sector and scale of business. We regularly do such analysis internally, and we are committed to drawing it into our annual report next year.

Can changes in inclusive growth really be measured objectively? To my mind, that is almost impossible, given that “inclusive growth” is not an accounting or financial term but an outcome towards which people want to strive. Does the panel agree?

You are right, to a certain extent. As we said, there is no single measure of inclusive growth, and what the concept means to one person might be slightly different from what it means to another. I guess that that is why we use a basket of measures.

We ensure that companies that receive RSA grants adopt the fair work first criteria, so that, at least, is one way of getting an indication that RSA supports good-quality jobs.

Looking at that basket of measures gives us a proxy for the types of benefits that we are getting and the types of jobs that are being created.

I agree that it is challenging; we are doing this pilot because there is a gap. Kenny Richmond was right to highlight the diagnostic as a tool to identify the challenges that an area has; it is not particularly about measuring what you are doing for that area or for a group of people. With the pilot, we are trying to look at the characteristics of the people and the place and we are seeing whether we can work those characteristics into how we measure our impact. There are ways of doing that; it is not scientific, but we can combine it with other evidence-based research.

For example, we can look to our research on young people to tell us whether they feel that there has been a change in the area that they live in, whether they want to come back or stay in that area, and whether the job opportunities are there but the academic offering or the infrastructure is not. The role of HIE is to have an impact on that. We can measure what is happening through our research and through the pilot and the basket of measures. If we combine all those approaches, we can get a better story to tell and a better indication of what we are doing and whether we are doing it in the right way and in the right places.

Inclusive growth evaluation is challenging. I lead the eight area teams across HIE and we have a place-based approach. Inclusive growth helps us to flex our approach. Recently, we invested in a caravan park on the island of Fetlar, which is a small community in Shetland. An economic development agency would not normally make such an investment, but by doing so, we made the community hall, the cafe and the local heritage centre more sustainable because of the increased footfall coming through that small community. The impact is difficult to measure but, as officers on the ground, we can see that community confidence has been boosted, which helps communities to develop more projects. I agree that inclusive growth is difficult to measure, but it has certainly focused our minds on how HIE delivers services on the ground.

Evidence from previous sessions indicated that the key performance indicators for each enterprise agency are set and monitored internally. Will the panel members explain how the KPIs are set? For example, are they signed off by the Scottish Government? What role does the Scottish Government have in agreeing the KPIs? Also, do panel members recognise that if KPIs are set, monitored and measured internally, there is a degree of concern over their transparency?

We use a range of evidence when we set the KPIs. They are linked to the pipeline of potential activity in the year ahead, what we have achieved in the past, the wider macroeconomic environment and what we know our budget will be. We use a mix of criteria to set those KPIs, which are agreed by our board. We are stretching where we can—that is why we put the KPIs within a range. The KPIs are shared with the Scottish Government. They are not signed off by the Scottish Government as such, but the Government sees them and can comment on them if it wants to. It is difficult to make assessments of what we will achieve, but by giving that range, and basing the KPIs on a range of criteria, we think that the KPIs are robust and stretching. We publish our year-end results, which gives transparency around how we have progressed against those KPIs.

It is a similar process for HIE.

We would also look at dealing with our sponsorship team in the Scottish Government and liaising with the strategic board. We look at pushing some of the boundaries on targets—for example, this year, we increased our fragile area job target beyond what we thought would normally be achievable. We will flex to try to push investments in those areas.

We are in a demand-led organisation and have a good pipeline of commitments on our books. That is how we generate the measures for the next year. Lorna Gregson-MacLeod is right. We try to push certain targets, particularly on jobs in more remote and rural areas.

You share the targets with the Scottish Government, but does it typically give you detailed feedback, or sit down and meet you to discuss the targets in detail?

It does so in general terms. Colleagues of mine will meet the relevant team in the Scottish Government to discuss the targets. Generally the feedback has been that they look okay and that we should go ahead. We share our targets with the Government before we publish our business plan.

I want to explore some of the numbers that Scottish Enterprise helpfully provided in annex A of your written submission. I am not sure whether you have those numbers in front of you. I want to check some of the trends underlying the numbers. For example, on clawback of RSA, there is a significant jump in the 2019 numbers. In 2018 the clawback was £2.3 million. In 2019 it is down as £6.7 million. I want to check what underlying trends sit behind that increase.

That number also struck me when I saw it. It includes the recovery of funds from a company that went bust in Dundee, which is repaying us £6.3 million. If we take that amount off the figure, it is actually down at £400,000 which is quite a normal, average year for clawback.

Does that explain why the other grants that were provided to that company in Dundee were also clawed back?

Yes.

In table 2, there is a drop in RSA grants from £17.1 million last year to £12.6 million this year, but an increase in other grants from £72 million in 2018 to £102 million this year. I want to get a picture of the underlying drivers behind those numbers.

Which table are you referring to?

It is table 2 on RSA expenditure.

Bear with me while I look at it—sorry.

It is basically showing a drop in RSA grants from £17 million last year to £12 million this year, compared with an increase in other grants from £72 million to £102 million.

Okay. There is a slight drop in RSA grants, which is partly due to foreign direct investments. Sorry, that is expenditure. I apologise. I was looking at the wrong thing.

Expenditure fluctuates greatly depending on how the companies are progressing. Clearly, an RSA grant, like all the other grants, pays out over a number of years. An initial award is announced of so much money to be spent, but that is drawn down only over two or three years. It may be that the £12.6 million paid out on RSA in 2019 is a reflection of fewer grant offers having been made, one, two or three years previously, or just of a slowdown.

On the other grants that have increased, I can speak, for example, on the research and development grant, which I also look after. There has been a huge increase in activity in that particular area. More companies are bringing forward more projects with greater complexity and ambition, and that has driven up spend in the area of R and D, which is part of the other grants line in table 2.

That is helpful. Thank you.

Most of my questions have been covered, but I never resist an opportunity to ask more. A table that I cannot see here is one on the decline in the number of jobs, because while the RSA line is declining—as highlighted by my colleague, Dean Lockhart—the number of jobs relative to the money that is spent is also declining. Is there, in your evaluation, an explanation for why?

There is not a specific explanation in the evaluation. Wendy—do you have one?

The schemes are demand led; companies ask for the products and processes that meet their ambitions. If they are not bringing forward as many jobs-related projects, the number of jobs might be skewed down the way.

11:30  

Some of the grants that we offer are not jobs related. For example, the research and development grant is not a job-creation grant, although through that grant we can sometimes create or safeguard thousands of jobs or, in another year, hundreds of jobs. That is specific to the grant offer that we have been asked to support.

RSA is linked to jobs, though—

Yes, it is, but much depends on the breakdown of what companies have applied for, which might be skewed towards more capital-intensive projects that might not have many jobs related to them, or towards aid for job creation. I cannot give you the breakdown of funding in that regard today.

I would be interested to see that, because the trend is showing a decline, over time, in the number of jobs that are being generated. I am interested in whether the underlying causes are understood. Is there just a general shift, or is it happenstance? We do not know.

I would be happy to provide written feedback. Of course, when the state aid rules changed in 2014, under the general block exemption regulation it became very difficult for large companies to apply for RSA funding, because they could not meet the new GBER rules. Therefore, in effect, 99 per cent of the time the scheme supports SMEs, which tend to have lower job counts. The number of jobs that are being created is decreasing naturally, because we are relying on the SME base rather than on large companies that come in and set things up. There are exceptions, obviously, but I think that that is why there is a declining trend. I would be happy to provide more evidence on that.

Colin Beattie asked how you assess the quality of jobs. Will you explain the process? How, at the start of an application, do you assess how many jobs will be created or safeguarded? Is payment of the Scottish living wage a requirement for a company to receive RSA?

As part of our due diligence on RSA grants, we look at what the company is asking for and the return that Scotland will get. With the RSA scheme, there is always a jobs element; the company will say exactly how many jobs it proposes to create and safeguard as part of the project. We ask for information about that—in particular, evidence is required to prove that the jobs that the company says it will safeguard are genuinely at risk.

The job numbers are then set in stone, in a contract, so that we can track and monitor the situation throughout. The grant is paid on delivery—when the company creates new jobs or safeguards jobs—throughout the contract’s lifetime.

From 1 April, there have been conditions that all project-related jobs in our large grant schemes must be paid the fair living wage, and that no zero-hours contracts are allowed in the projects.

Such issues are brought up with companies right at the beginning. As soon as we get wind of an RSA application, we go out and meet the company at an inquiry meeting and ensure that it is clear about the rules and eligibility criteria. I am pleased to say that, so far, all the companies have risen to the challenge. To be fair, I point out that 93 per cent of RSA projects were already paying the real living wage before the new rule came in; now, we really are up at 100 per cent. No company has been turned away because it was not paying the real living wage, and no company has withdrawn an application because it could not do so. If anything, circumstances have improved for company employees.

I am pleased to hear that, given that we advocated payment of the living wage for a long time.

Gordon MacDonald started talking about how much has been clawed back. Is your evaluation sensitive enough to be able to pick up emerging problems at companies, or is that the responsibility of the account manager? I am thinking about Kaiam Europe Ltd in Livingston. Would your evaluation have spotted that the company was experiencing difficulty?

Emerging problems happen in real time, whereas an evaluation is a retrospective exercise. The account manager and other team members who visit the company are, largely, responsible for understanding the company’s position.

In my team, we can tell from the grant-management process how the project is progressing, because a company puts in claims for the money that it has spent, so we can track whether it is meeting its milestones. The accounts are produced by certified accountants who audit the claims for us as they submit them. There are therefore two ways of getting information—from the account manager who goes in regularly to talk to the company about what is going on, and from the requests from the company to support its claims about how the projects are progressing, which gives us another way of understanding whether a company is starting to get into difficulties.

I know that evaluation does not stand alone—Wendy Hanson has just been talking about that. I want to ask about the murky world of econometric analysis in which you engage. Without going into detail, how can you be sure that the statistical stuff that you do is robust enough for the sample sizes that we are likely to have in Scotland?

“Economics” and “murky” are quite often brought together in a phrase.

There are probably two broad methodologies that we use for evaluations. One is to take companies that have been supported and a control group of companies in the economy that have not been supported. Through economic techniques, we look at how those two groups have performed over time. The Fraser of Allander institute is in the middle of some work on that for us, with the benefit that we do not get survey fatigue issues with companies because we do not have to go out to survey them.

However, that method has its downsides. To what extent can you get an adequate control group when you are comparing assisted companies with non-assisted companies? You can create a control group based on the size of the company and the sector that the company is in—what we call observable criteria. However, other criteria within companies are less observable—how good the management skills are, how ambitious the company is and so on—so it is less easy to get a control group for them.

The other method is to survey companies and go out to speak to them. The benefit of that is that we get qualitative evidence on what is working, but it also has its downsides. For example, if we had supported a company three or four years ago, the people who were involved in that project might not still be with the company, or might not remember exactly what the company was doing then. However, that is a way to get evidence and feedback from a company on how it thinks support has been beneficial.

There is no perfect way of doing it, but if we use both methodologies and they come out with broadly the same findings, that gives us a pretty good steer that the findings are robust. If we marry those with the real-time assessments and information that we get back about companies, the three elements give us a pretty good idea of how well projects are performing.

How does the evaluation process make positive changes to what we might wish to do in the future? I am sure that you agree that there is no point in conducting evaluations then always doing the same thing year in and year out. We know that we have record employment levels in Scotland. What is the trigger for thinking that we should change and adapt the model, and improve the process that we engage in, in order to try to achieve greater benefits, perhaps in parts of Scotland that have not been reached by such activities and interventions?

In considering what we are doing and how we might want to tweak things, we draw on a mix of evidence sources, including formal evaluations and the real-time intelligence that we get from our businesses and partners on what they are picking up. We can look at what is working and what the challenges are, then consider potential projects and the support that we can we put in. That all helps to raise ideas or issues that we might want to look at.

We also engage with our companies and our customers to get their views on how well the process is working to support companies: we ask whether we could do things more quickly, for example. That is fed into our work on the single business portal in Scotland, which will be a one-stop shop for businesses that are looking for advice. That work is based on the feedback that was collected as part of the enterprise and skills review.

There is a wide range of sources of evidence that we can draw on to see how well we are doing, the benefits and impacts, and whether there are things that we need to consider and tweak. It is probably the same for Highlands and Islands Enterprise.

Both organisations are committed to continuous improvement—that is embedded in how we work. As an economic development agency, HIE always starts from the point of development, so we are always looking for new opportunities. Through the Enterprise and Skills Strategic Board, we are working to see how we can improve our services for customers and ensure that our clients and customers are at the heart of what we do. We need to understand their needs. We also want to use digital technology to speed up, ease and personalise services for our customers. There are many opportunities. We are evolving and using customer surveying techniques, focus groups and digital technology to improve our services for the future.

Can you give examples in which you have changed your approach? I am looking at the table that was mentioned earlier and is in annex A of the Scottish Enterprise submission. It outlines all the RSA grants and all the sectors that benefited. I will make a shameless plug for my constituency in Ayrshire, which might seek a greater slice of such activity in the future. How would you make that change? If the agency is doing well in certain areas and the results are sustainable, on what basis do you decide to shift the emphasis to another part of Scotland that might need assistance? Have you done that, or is that just not what you do?

HIE has a very strong geographical focus. We have a range of measurements of what is happening across our area, and we look beyond our area to see whether there are mechanisms that we can learn from in order to make a greater impact. It is very important to build that into our evaluation. I mentioned female entrepreneurship, which is just one example. We also took that approach with account managed businesses when we carried out an evaluation back in 2015. The breadth of evidence can support different approaches.

The inclusive growth pilot is very important in showing that evidence. It can be more than an evaluation tool—it can also be a resource prioritisation tool. If we can get it to work effectively, it will show us something that is like a heat map of different types of disadvantage across our area, which we can then use as a tool to prioritise resources better.

Have you done that?

The pilot has just started. We have learned from evaluations when a programme or initiative works better in an area. For example, in community-led development, evaluation has found that the important ingredient could be a particular person. That evaluation looked at the role of the community development officer in a community-based organisation, who might take on a wider community role, and at the impact that that can have. That is wedded to having a person in geographic locations—if such organisations do not have such a person, they have a smaller impact. We use the findings to use that model somewhere that needs such help. We learn from and reflect very hard on evaluations.

This is my last question. Several members have talked about employment levels and the workforce, and we know that productivity is the key to adding real value and increasing gross domestic product in Scotland. How do we use programmes and interventions to increase productivity?

One of the focuses of RSA is to help companies to invest in capital. We know from a wide range of research and evidence that one of Scotland’s challenges in respect of productivity is our low capital investment rate, so we are helping companies to consider investing in capital and supporting them to do that. The evidence shows that capital investment is a driver of productivity, so that should feed through to increased productivity. The situation in respect of our research and development grants is similar: evidence shows that companies that undertake R and D tend to be more productive than those that do not, so R and D grants are also a driver of productivity.

We look at how all our types of support influence the drivers of productivity so that we ensure that we can make the link between what we are doing and the impact on productivity, which—as Mr Coffey said—is really important for driving high-quality jobs, increasing wages and so on.

11:45  

Investment in human capital is very important, particularly in the Highlands and Islands, where there has been market failure in provision of high-level leadership and management training. We invest significantly in that area, and companies have transformed their working practices and their processes—how they do things—because of investment in high-level leadership and in the Scottish manufacturing advisory service’s lean learning academy. There are a range of interventions that we can make to support better quality leadership and management in Highlands and Islands firms.

That is a good point. A number of companies that receive RSA also receive other types of support. The combination of support in leadership, workforce development and exporting makes the difference in driving productivity.

I apologise to the panel for not being here earlier in the meeting. I am sure that other members from the Highlands and Islands will accept that it is sometimes hard to get down from there.

I will follow on from Willie Coffey’s questions. The panel has talked about the geographical breakdown. How accurate is your evaluation of return on investment in areas such as Orkney, Shetland and Moray, given that they have different priorities? If we increase business in Orkney, there will be pressures, given the limited number of people there who are looking for work. Moray has a low-wage economy, so a different approach is needed.

There are probably two points that are relevant. We set different targets within the region. In our operating plan, we present the regional targets, but all the areas have different targets that reflect their economic context. You mentioned employment in Moray, so there might be a focus there on particular investment and measures, whereas the focus in the Western Isles might be more on fragile jobs. The targets are set internally, in different ways, to reflect economic circumstances. We must have cognisance of that when we do economic evaluation.

As I explained earlier—it might have been before Jamie Halcro Johnston arrived—it is important that we know the relative impact of what we do. Impacts can fluctuate quite drastically between our more urban areas and town centres and our islands, including the fragile parts of them. We are trying to create a tool to measure what we should do in a different way. We are not there yet, but we are building up the data and the stories. I hope that we will be able to enhance the pilot by the end of this financial year, so that we can build that information into the evaluation work. It is important that we have a different way of viewing numbers. They should not be viewed in isolation, because the geographical context must play a key role for us.

As well as the work that you do, there are other agencies that support economic development, including council departments and business gateways. How closely do you work with them? How helpful can they be in bringing back information for the evaluation? Does co-location and collaboration help with that? If so, is co-location your preferred option in most cases?

I work in Shetland, and I am co-located with the economic development department of Shetland Islands Council. That close working relationship is extremely valuable. The area teams in HIE have very close relationships with business gateway, because we have to support each another. It provides feedstock for the future, and we refer businesses to it.

Given that HIE is listed as a partner under schedule 1 to the Community Empowerment (Scotland) Act 2015, there is a legal requirement for us to collaborate and to facilitate and support community planning with our local partners. We take that responsibility very seriously across the area offices. Economic development is not just about investing in particular businesses and communities; it is also about transport and communications infrastructure. We need to work with other partners to ensure that infrastructure is in place to support businesses and community growth.

When I visited business gateway Shetland last year, it came across that co-location can be very important. If co-location were brought in across the Highlands and Islands Enterprise region, would there be improvements in the areas where it does not happen now?

If the potential exists for co-location, it can be a very positive thing, but I do not think that an absence of co-location is detrimental to the relationship. Co-location in the same premises is probably more efficient.

I appreciate that it is not always a HIE decision.

My final question is a follow-up to Dean Lockhart’s question about inclusive growth. Would it be easier to deliver and evaluate inclusive growth if there was a single definition of it?

Even if there was a single definition, it would be open to interpretation. What inclusive growth feels and looks like in the centre of Glasgow will be different from what it feels and looks like in the Highlands and Islands, where businesses and communities will have different priorities and face different challenges. Similarly, the opportunities that land at their doors will look and feel different.

In Scotland, we share the same concept and we understand what that is, but we need to look at the issue from a particular organisational or regional perspective. It is very important that HIE does that, given our social and community remit.

I agree. That is why it is important to have a basket of measures. As Lorna Gregson-MacLeod said, inclusive growth can mean slightly different things in different parts of Scotland. Having a basket of measures is probably a good way to pursue inclusive growth.

No one can disagree with the fact that Scottish Enterprise and Highlands and Islands Enterprise have done a good job, but as someone used to say to me, “That is now water under the bridge. What are you going to do tomorrow?”

I notice that there were quite a number of failed applications. You say that the figure is only 6.5 per cent, but am I correct in saying that that figure relates to applications from account managed companies, or does it relate to applications from companies that are not account managed?

I would have to give you that statistic separately, because I do not have it to hand. I can tell you that only 50 per cent of RSA awards go to account managed companies, so there is a chance that only 50 per cent of what we call rejections are of applications from account managed companies.

There is often a good reason why an application does not progress to an award, which will be to do with something that the company has told us. I do not think that whether a company is account managed would have an influence or impact on that. There is close working with the appraisal team so that we can understand what the company is trying to do. It is only if the company supplies information at the application stage that makes it ineligible, or if it cannot meet the criteria, that we would have to reject the application.

Since 2014, the total number of applications has fallen from 145 to 83—that figure is from 2018. What are you doing to increase the number of applications that are made? How are you encouraging new applications, new products and new innovations?

The RSA scheme is strictly bound under the state aid guidance that we use. We have only so much flexibility around what we can offer companies. The downturn was due to the inability of large companies to apply for funding—that is why the numbers shot down. Since then, we have worked extremely hard with our SME community to remind everybody that the scheme is still open for business. Although we still get applications from large companies, we get only one or two a year, so we are relying on our SME base to deliver the RSA measures.

We work very closely with different agencies and bodies to get across the message that the RSA scheme is still open for business. We look to our SMAS communities of practice and we present results to the Scottish local authorities economic development group. SLAED is very good at charging the business gateways with the task of stimulating activity. We spend a lot of time on the road promoting the schemes. We also do it through the usual electronic mechanisms, such as the web. Every time there is an RSA success story or an award is made, we put out a tweet to stimulate further activity and spread the message. That is starting to work. Although the number of offers is in decline, their value is increasing so the SMEs are generating more complex, interesting awards.

We get to work with companies early in the process. Often when we visit a company it tells us what it plans to do and we ask whether it has thought about training and other elements that we could weave in. We also ask about its long-term growth plans—whether it is thinking about just one project for the next two years or about what it will be doing in five years’ time. Getting involved in such conversations with companies early in the process improves their aspirations to bring us bigger projects on which we can deliver. Although the number of offers has arguably decreased, their complexity and value are starting to increase because of the groundwork that we are doing with companies.

I go back to Mr Coffey’s question about the slice of the cake that an area could have. Is there a regional target or could a funding breakdown by constituency be provided—or would that be too technical?

No, we can easily draw that information together and provide it after this meeting. For example, one option would be for us to break it down by local authority area.

We must also remember that some constituencies fall under two local authorities.

Yes, I was aware of that when I was preparing for the meeting and trying to work out where all the committee members sat. [Laughter.] I had a list of projects against various members’ names, and sometimes the same projects appeared in relation to different people. We will provide that information in the format that we think will best suit the committee.

So you were prepared for some of us to ask, “Where is my share of the cake?”

Absolutely.

Thank you.

For a number of years, I have been trying to persuade NHS Information Services Division and others to give members statistics and information on our own constituencies, but we rarely get them. They are provided in relation to local authorities, health boards or this or that, but rarely on the basis of our constituencies. For any aspect of this work, it would be very helpful for members to be told how their constituencies vary across the piece, if that is possible.

Okay. I will be happy to supply that after the meeting.

Thank you.

Dean Lockhart has a quick follow-up question.

My question is about demand and the decline in applications. If and when the Scottish national investment bank is up and running, significantly more funding might be available for enterprise development. How would such an increase match up with the declining demand in the marketplace that you have described?

A key differentiator between what Scottish Enterprise does and what the Scottish national investment bank will do is that we are a grants-led team. Therefore, we look at making unsecured investments in companies, whereas the bank’s team will largely look at making loans to them. The solutions for companies will depend on looking at them case by case.

We have challenging conversations with companies whenever they come and ask us about money. We look at whether a loan would be more appropriate for them than a grant. A company’s financial circumstances will determine whether it can meet the requirements of either of those options, and we can then provide the right type of financial assistance for it because we have two different offerings. I expect the future demand for grants to remain high because there are companies out there that, for whatever reason, just will not be able to secure loans.

As members have no further questions, I thank our witnesses very much for coming in today.

11:58 Meeting continued in private until 12:58.