Audit Committee, 02 Mar 2004

Meeting date: Tuesday, March 2, 2004

Official Report 244KB pdf

---

“Performance management in Historic Scotland”

Agenda item 5 relates to Audit Scotland's report "Performance management in Historic Scotland". I invite the Auditor General to brief the committee on the report.

My briefing will be brief. We have published a report on performance management in Historic Scotland because the matter has been in our work programme for some time. Our report was intended to examine how Historic Scotland approached measuring, managing and reporting its performance.

The general picture that emerges from the study is of an organisation that is achieving its targets and delivering services that meet its objectives in broad terms. In 2002-03, seven of the nine key targets were met or exceeded and the other two were narrowly missed. Those targets cover key features of Historic Scotland's performance in relation to conserving buildings and promoting interest in them. In the longer term, over the past seven years, the great majority of the targets—90 per cent—have been met or exceeded.

A key finding of the report is that, in recent years, not all the key areas of work were covered by performance targets. For example, targets relating to the grant scheme were discontinued three years ago and the target for visitor numbers was replaced with a target for market share compared with other visitor attractions. We have suggested that Historic Scotland should consider reintroducing performance targets for those matters and that more information could be provided on the cost and quality of services.

Another key finding is the considerable success that Historic Scotland has had in increasing its revenue, largely from admission charges and retail sales. Our report records that, in 10 years, generated income grew from £6 million to just over £19 million, which is a growth of 220 per cent. The percentage of the organisation's budget derived from income has doubled from 18 per cent in 1982 to 36 per cent in 2002. There is quite a change in the balance between income and Government grant. In effect, spending increases have been largely supported by increases in income. Historic Scotland's budget has risen from £40 million in 1993-94 to £53 million in 2002-03 and is planned to rise further over the next three years.

It is probably worth noting the volume of activity that Historic Scotland undertakes. For example, it handled some 25,000 listed building consent applications over 10 years as well as many thousands of other types of applications. In most cases, those are dealt with without any controversy or difficulty. However, it is fair to say, as the report does, that Historic Scotland's decisions can be controversial and occasionally attract complaints about fairness and consistency. Historic Scotland pointed out to us that none of those complaints has led to a successful application for judicial review or an unfavourable report by the ombudsman.

Generally speaking, the picture of the performance of the agency is positive. As members will be aware, in parallel with our study, the Scottish Executive has conducted a review of Historic Scotland, which concluded that Historic Scotland should remain an executive agency in the Scottish Executive Education Department and that it should continue to deliver all the functions that it currently delivers. The review identified a clear need for a cultural change in the organisation, although I should point out that our study did not consider that feature. The Minister for Tourism, Culture and Sport has announced that he is to ask the chief executive of Historic Scotland to implement an organisational change programme soon.

David Pia, who directed the study, is with me and we would be happy to answer any questions that you may have for us.

The committee will have the opportunity to decide what, if any, action we wish to take with respect to the Auditor General's report at a later stage in our deliberations, but members are welcome at this stage to seek clarification from the Auditor General and his team on any particular points arising from the report.

Auditor General, you identified the fact that you did not consider repair grants and their contribution to the social, economic and environmental objectives of ministers. Do you have any plans to revisit Historic Scotland and examine that matter? The issue of repair grants has caused me and other members some problems over the past five years and I have noticed an increasing number of public petitions about the level of grant and the way in which grants are approved or otherwise by Historic Scotland.

The short answer is that I have no plans to do further work on Historic Scotland for the foreseeable future. However, that does not mean that the matters that have been raised in the report will not be addressed. David Pia, along with the external auditor from Audit Scotland, Fiona Kordiak, attended the Historic Scotland audit committee the other day. They received a commitment there that all the issues that have been raised in the Audit Scotland report will be addressed and taken forward within Historic Scotland and will be overseen by Historic Scotland's audit committee.

It would be perfectly possible for this committee, on behalf of the Parliament, to invite Historic Scotland to report in future annual reports on any matters that may be of concern to members. Through the normal audit process, we will be keeping under review the progress made against the suggestions in the report. The option exists for members to receive reports-back if required.

The external auditor intends to examine the administration of the repair grants scheme over the next year. That is included in the auditor's plan.

I note in passing that, although this committee is often quick—by necessity—to identify failings and shortcomings in organisations, we have before us some very positive measures of performance and we might wish to explore that further in future.

I wish to ask about the mystery visitor programme. Paragraph 2.29 of the report states:

"the ‘mystery visitor' programme replaced the ‘percentage of satisfied visitors' targets as a measure of customer satisfaction and the quality of the service."

I am interested in that particularly as somebody who feels strongly that we need to find ways of assessing people's experiences of public services generally in a more qualitative way. Some of the traditional measures, such as percentages of satisfied visitors, have limitations. Could you comment on that point, Auditor General?

I will invite David Pia to come in on that in a moment, but I will offer a preliminary point first. The response that we received from Historic Scotland on the matter was that the organisation was performing extremely well against the simple measure of the percentage of visitors who are satisfied or very satisfied. Historic Scotland therefore decided to introduce a new measure, the mystery visitor survey, which it thought would be a more sensitive system. However, it is arguable that, to the general public, the percentage of satisfied visitors is a useful statistic to have available.

I am not sure that I can add much to that. The mystery visitor programme is commissioned to be run by an independent company. Historic Scotland argues that it finds that measure more helpful and reports a satisfaction level of 87 per cent according to that system, as compared with a level of 97 to 98 per cent, which was steadily reported with the customer satisfaction survey. We have drawn the attention of Historic Scotland to the issue of targets and how it measures visitor satisfaction and the organisation will review that system.

I presume that Historic Scotland does not need to choose between the two systems, as an organisation could use both systems, in theory. To my knowledge, it is relatively unusual, although perhaps not unique, for a public body to have pursued such a project in such a specific way, by making it one of its major targets. I wonder whether Audit Scotland has explored the potential of building such a system into the targets of other public bodies. Do you consider that the various organisations should liaise with and learn from one another?

In our reports of this kind, we draw attention to the systems that an organisation uses to measure customer satisfaction. That is central to the right approaches to performance management.

I would like some clarification of why Audit Scotland wants Historic Scotland to restore the information about the number and value of repair grants, as well as of other issues relating to its explicit targets.

We are concerned that although a substantial proportion—approximately 20 per cent—of Historic Scotland's annual expenditure of £11 million or £12 million relates to the grant scheme, none of its key performance targets relates to that scheme. However, as the report acknowledges, that is not a straightforward matter, because the number and value of grants do not tell the full story of what is done with the money and the outcomes of the expenditure. Nonetheless, we feel that Historic Scotland's failure to provide the information means that there is a gap in the suite of performance data that it publishes. We would like some more basic data about, for example, the numbers of applications to be reported. Those data might not be reported as a key performance target, but we would like more information about such activity to be reported in Historic Scotland's annual report.

The matter is clearly difficult, as you mentioned. In a sense, the performance is what the money delivers. Audit Scotland wants more information about the performance and the way in which it is administered.

We recommend that Historic Scotland should provide more information about grants and consider its means of identifying an appropriate target that relates to such a major sphere of activity.

It is useful to have had this work done. We note that the Executive has conducted a review as well. I welcome the minister's recognition of the need for a cultural change. Part of the value of the report is that it shows that the organisation is functioning well. Other reports have drawn attention to the need for some sort of cultural change, but that is not really within the remit of this committee. Audit Scotland recognises that, within the scope of its report, Historic Scotland is performing well.

We will discuss how to proceed at a later stage. I thank the Auditor General and his team for the briefing.