© National Audit Office 2023

Regulating to achieve environmental outcomes

Department for Environment, Food & Rural Affairs

Session 2022-23
21 April 2023
HC 1283

The National Audit Office (NAO) scrutinises public spending for Parliament and is independent of government and the civil service. We help Parliament hold government to account and we use our insights to help people who manage and govern public bodies improve public services.

The Comptroller and Auditor General (C&AG), Gareth Davies, is an Officer of the House of Commons and leads the NAO. We audit the financial accounts of departments and other public bodies. We also examine and report on the value for money of how public money has been spent.

In 2021, the NAO’s work led to a positive financial impact through reduced costs, improved service delivery, or other benefits to citizens, of £874 million.

We are the UK’s independent public spending watchdog.

We support Parliament in holding government to account and we help improve public services through our high-quality audits.

Regulating to achieve environmental outcomes

Department for Environment, Food & Rural Affairs

Report by the Comptroller and Auditor General

Ordered by the House of Commons
to be printed on 19 April 2023

This report has been prepared under Section 6 of the National Audit Act 1983 for presentation to the House of Commons in accordance with Section 9 of the Act

Gareth Davies
Comptroller and Auditor General
National Audit Office

17 April 2023

Value for money reports

Our value for money reports examine government expenditure in order to form a judgement on whether value for money has been achieved. We also make recommendations to public bodies on how to improve public services.

The material featured in this document is subject to National Audit Office (NAO) copyright. The material may be copied or reproduced for non-commercial purposes only, namely reproduction for research, private study or for limited internal circulation within an organisation for the purpose of review.

Copying for non-commercial purposes is subject to the material being accompanied by a sufficient acknowledgement, reproduced accurately, and not being used in a misleading context. To reproduce NAO copyright material for any other use, you must contact copyright@nao.org.uk. Please tell us who you are, the organisation you represent (if any) and how and why you wish to use our material. Please include your full contact details: name, address, telephone number and email.

Please note that the material featured in this document may not be reproduced for commercial gain without the NAO’s express and direct permission and that the NAO reserves its right to pursue copyright infringement proceedings against individuals or companies who reproduce material for commercial gain without our permission.

Links to external websites were valid at the time of publication of this report. The National Audit Office is not responsible for the future validity of the links.

011090 04/23 NAO

This report can be found on the National Audit Office website at www.nao.org.uk

If you need a version of this report in an alternative format for accessibility reasons, or any of the figures in a different format, contact the NAO at enquiries@nao.org.uk

The National Audit Office study team consisted of:

Annabelle Brown, Simran Nijjar, Sarah Pearcey, Eloise Peck under the direction of Charles Nancarrow.

For further information about the National Audit Office please contact:

National Audit Office
Press Office
157–197 Buckingham Palace Road
Victoria
London
SW1W 9SP

020 7798 7400

www.nao.org.uk

@NAOorguk

Key facts

Summary

Introduction

1 The government wants this to be the first generation to leave the natural environment of England in a better state than it inherited. In 2022 it published binding statutory targets in four priority areas: air quality; water; biodiversity; and resource efficiency and waste reduction, which came into force in January 2023. Achieving these targets will require action, not just from the government but also from businesses and individuals.

2 The Department for Environment, Food & Rural Affairs (Defra) has lead responsibility in government for environmental policy areas except net zero. In January 2023 it published the Environmental Improvement Plan, revising the 2018 25 Year Environment Plan in light of the statutory targets. The Environmental Improvement Plan (EIP) sets out the importance of both existing and planned regulatory interventions in contributing to environmental goals.

3 We use the term regulation to describe a set of rules and expected behaviours that people and organisations should follow, which will often involve bodies with regulatory powers enforcing and influencing compliance with those rules or behaviours. Defra already uses regulation extensively in achieving environmental outcomes, particularly through arm’s-length bodies such as the Environment Agency and Natural England, alongside other interventions.

Scope and purpose of this report

4 This report examines whether Defra and its arm’s-length bodies are using regulation effectively to achieve its environmental objectives. We reported in 2020 on how government was set up to deliver its long-term environmental goals.¹ This report also reviews progress against recommendations we made in our 2020 report, where they are particularly connected to Defra’s use of regulation. This report maps out where these bodies are already using regulation, often alongside spending or other government activity. It also assesses how regulation is working in practice in case studies of individual regulatory interventions. Our evaluation and recommendations are based on our good practice guidance on the principles of effective regulation.² The report does not assess the government’s overall performance against environmental goals, or the coherence of environmental strategies.

5 The report covers:

the extent to which the government has set a clear direction for environmental regulation and how well it understands how its regulatory interventions fit into its overall ambition (Part One);

how well Defra oversees its regulatory bodies in respect of delivering its environmental objectives, and evidence on current regulatory performance and capacity to deliver (Part Two); and

whether Defra has suitable arrangements for evaluating and measuring the performance of regulators and regulatory interventions in supporting Defra’s environmental objectives (Part Three).

Key findings

Defra’s environmental aims and use of regulation

6 Defra has put in place a legal framework for its ambitious environmental objectives but is still developing the indicators it needs to measure progress towards them. Defra’s 25 Year Environment Plan, published in 2018, set out ambitious environmental objectives. The Environment Act 2021 required statutory targets in four priority areas. It also created an independent body, the Office for Environmental Protection (OEP), which must monitor progress against the wider objectives and the statutory targets. Defra has not yet finalised the indicators to measure progress for more than half of its new targets (paragraphs 1.3 to 1.4, 1.9 to 1.11, Figure 2).

7 Defra is planning to use regulation and regulatory reform as a tool to achieve its environmental objectives, as set out in the January 2023 Environmental Improvement Plan. The Environmental Improvement Plan recognises that Defra requires action to be taken by individuals, businesses and the government to achieve its objectives. Regulation is included as a tool to deliver all ten of the goals set out, such as to protect habitats and prevent air pollution, alongside other measures including direct funding and private investment. It also includes commitments to improve specific regulatory frameworks, such as expanding environmental permits, reviewing farming and building regulations, and supporting regulators in ensuring compliance (paragraphs 1.5, 1.12 to 1.14).

8 Despite starting work in 2018, Defra is still at an early stage in understanding how existing regulations affect its plans. In 2011 Defra published a biodiversity strategy on how it would implement international and EU commitments up until 2020. It is yet to publish an update. Defra published detailed strategies on resources and waste in 2018 and air quality in 2019. It published a water strategy in April 2023. It is still developing its understanding of how existing regulation operates. For example, in 2021 Defra’s air quality team mapped the industries it regulated and considered the impact of net zero policies on air quality. In 2022 it started work to review the regulatory system for industry emissions to ensure alignment with government priorities for air quality. Work in other areas, particularly biodiversity and water quality, is less advanced. Without knowing what is in place, Defra will be unable to identify what changes may be required to deliver its targets (paragraphs 1.15 to 1.20).

9 Defra is not on course to achieve the goals it set in 2018, according to its own performance indicators and reporting from the OEP. Its new Environment Act targets, set in 2023, are broadly similar, so Defra will need to significantly improve its performance to achieve them. In many areas where there are performance data, Defra is failing to make the progress necessary to achieve its 2018 goals. The new targets are broadly similar, so Defra will need to improve its performance to achieve its new statutory targets. For example, in its 2018 goals Defra set a goal to take action to recover threatened, iconic or economically important species. However, there was a decline of 17.2% in priority species abundance between 2013 and 2018. Defra’s 2022 biodiversity targets include halting the decline of species abundance by 2030, and then ensuring that species abundance in 2042 is greater than in 2022. Defra told us that most of the measures that will affect future performance on species abundance, principally reforms to farm payments through the Environmental Land Management schemes, are yet to take effect. In January 2023, the OEP reported that the 2018 25 Year Environment Plan had so far failed to bring about the changes needed, at the pace and scale required, to meet the government’s stated ambitions (paragraphs 1.3 to 1.4 and 1.10 to 1.11, Figure 2).

10 Some of Defra’s proposed changes may need legislative change or other longer-term regulatory reform. It can take years before changes will be in force and contributing to achieving Defra’s objectives. For example, Defra expects its programme to update regulations for industrial emissions on the basis of Best Available Techniques to take around a year to agree changes. It will then take time to pass secondary legislation and will allow businesses four years to meet the standards. Some of the changes that Defra is considering making to its environmental framework would require legislative change. For example, significant reform of habitats regulations would need new primary legislation (paragraphs 1.21 to 1.23).

Defra’s oversight of regulatory delivery

11 Defra has recently made changes to its governance arrangements. It has more to do to align these arrangements to new environmental targets and plans. Defra’s governance structure for delivering its environmental goals involves boards reporting to an overarching Environment Committee. The structure has developed over time as priorities have changed, and the OEP had raised concerns about the complexity of the arrangements. In 2022, Defra made changes to simplify its board structures and in March 2023 it started to consider further changes it needed to make to oversee the new statutory targets it had set (paragraphs 2.2 to 2.6).

12 Defra is changing how it works with its regulators to improve its oversight of their performance and new arrangements are being put in place. Defra can set priorities directly for its arm’s-length bodies, and it does so to ensure that objectives are shared. In 2022 it began the process of developing new sponsorship arrangements for its arm’s-length bodies, but their operation is still at an early stage. Both Defra and regulators have identified areas where existing mechanisms for oversight do not provide either side with what they need and are taking action (paragraphs 2.7 to 2.10).

13 The Environment Agency and Natural England have existing workforce issues that have affected their ability to deliver their current workload. They will need further resources in the future to support the EIP but it is not yet clear how much. Both the Environment Agency and Natural England have been recruiting staff to fill existing vacancies planned under Spending Review 2021 and committed to again in the EIP. In February 2023 the Environment Agency had vacancies of around 600 full-time equivalent (FTE) posts, 5% of its planned staffing. Natural England has reduced its vacancy gap from over 250 in September 2022 to one in February 2023 but recognises that it has a significant risk around capability, which it plans to mitigate by increasing spending on learning and development in 2023-24. They are also facing demand to increase activity in some areas, such as monitoring and inspections to respond to failures in water quality. Our case study work has identified specific areas where they and Defra are falling behind with their work, for example backlogs in permit applications or delays in developing plans to administer new waste regulations. The activity set out in the EIP will likely need additional resources over the next spending review period for Defra and regulators to manage it effectively, and Defra told us they are currently estimating their requirements (paragraphs 2.11 to 2.15).

14 Defra is seeking to make changes to existing regulation to ensure that it fits future needs, and there are areas of good practice it could draw on. In response to the Retained EU Law (Revocation and Reform) Bill, Defra is reviewing its stock of retained EU law, which is the largest in government. While Defra’s default position is to retain this law, it is considering where legislation and processes can be amended to better fit the UK – for example, on habitats regulation. Our case study work has identified examples where approaches and ways of thinking could be adopted more widely to develop regulation that better fits Defra’s current ambitions. These include adopting a broad approach where regulation is designed to tackle multiple issues together, or regulation that adapts based on performance information and technological innovations (paragraphs 2.16 to 2.21).

Evaluation and performance measurement

15 Defra has limited data on the effectiveness of its regulation to inform decisions about future activities and where to prioritise resources. Defra receives performance reporting from the Environment Agency and Natural England via its sponsorship teams, but this data does not allow it to fully understand the effectiveness of their regulatory activities. It is challenging to isolate the impact of regulatory interventions. Defra’s logic models are incomplete and we could not always identify how its regulatory interventions contribute to its overall outcome measures. While progress is being made, without a clear understanding of what is and is not working, it is difficult for Defra to make decisions about future activities and where to prioritise resources (paragraphs 3.2 to 3.9).

16 Defra developed a department-wide evaluation strategy in February 2023 and is in the early stages of taking this forward. Defra has a ‘hub and spoke’ model where most evaluation work is carried out in policy teams, with a small central team that has recently developed a strategy. It does an evaluation audit that provides the central team with oversight of what evaluations are being done and where. It does not routinely share lessons from the evaluations with relevant teams across the department. However, Defra intends to consider how learning can be shared more widely and efficiently. This is still at the early planning stages (paragraphs 3.10, 3.12 to 3.18).

17 Defra had a backlog of 63 Post-Implementation Reviews (PIRs), as at March 2023. While PIRs are not the only way to determine what works, this limits its insight into how well regulation is working or the burden it is placing on business. Departments are expected to carry out PIRs on regulation which has a significant burden on business, to gain insight into whether the regulation is working as expected and whether it could be improved. There are delays in producing PIRs across government and the Regulatory Policy Committee has not received the number of PIRs expected given the number that should have been completed. Defra has identified that its own backlog of 63 reviews is due to pressures caused by EU Exit and the coronavirus pandemic. It plans to clear the backlog over 2023 and 2024. Until it has carried out these reviews, Defra lacks an important source of evidence for whether regulation is meeting its objectives and the impact it is having (paragraphs 3.11 and 3.18).

Conclusion

18 Defra put the ambitious targets to deliver its 2018 25 Year Environment Plan on a statutory footing in 2023, and, in 2021, set up the Office of Environmental Protection with a duty to monitor progress. The Environmental Improvement Plan, published in January 2023, sets out the changes Defra is seeking to meet its targets, and envisages an important role for regulation and regulatory reform. However, according to its own data, Defra is not on course to achieve its 2018 environmental goals. It is too soon to judge performance against the targets it has just announced.

19 Defra’s work to understand how its current regulation supports its overall environmental objectives is still at an early stage. Key bodies with regulatory functions are falling behind on aspects of their existing work, and the resource requirements for the activity set out in the Environmental Improvement Plan are still being estimated. Defra has started to strengthen its governance arrangements and develop a strategy for evaluation. But it does not have the information it needs on the effectiveness of regulation, and it has a backlog of regulatory Post-Implementation Reviews. These gaps limit its ability both to make evidence-based decisions about where to deploy its resources, and to understand wider impacts of its regulatory approach. These are serious weaknesses, and Defra must take swift and effective action and build on its recent work to address them in order to get the best value from its limited resources and give itself the best chance of achieving its objectives.

Recommendations

20 As Defra takes forward its Environmental Improvement Plan, it still has decisions to make about the design and operation of regulation to ensure that it is meeting its targets. To make good decisions, it will need to have a clear understanding of what works and why. We therefore recommend that Defra should do the following:

a By December 2024, as Defra complies with its statutory reporting requirements under the Environment Act, it should incorporate detailed operational plans for how it will achieve the goals of the EIP, including the role of regulation alongside other policy interventions. Defra should set out its understanding of where regulation is currently being used to deliver environmental goals and how it operates. The plans should include the skills, capacity and capabilities Defra and arm’s-length bodies will require. They should also provide clarity to stakeholders on expected changes to regulation and the timelines for any plans.

b By December 2024, ensure it is meeting deadlines for its workload of Post-Implementation Reviews. This includes clearing the current backlog of Post-Implementation Reviews, as well as carrying out those that fall due in that period.

c By December 2024, enhance mechanisms for sharing evaluation findings and good practice across the Defra group. As Defra implements its new evaluation strategy, it should clearly set out and enhance its approach to communicating internally how evaluation findings can be applied to strengthen the design and implementation of regulation.

d By December 2024, identify ways to ‘future-proof’ regulations, and possible opportunities to put this into practice. Acknowledging that any significant regulatory changes may require new legislation, Defra should identify existing good practice around flexible regulation that can adapt to meet changing targets and consider adopting this more widely.

e By December 2023, complete its changes to governance of the environmental targets, and set out its plan to review their effectiveness. This plan should include a timetable for the review.

f By December 2023, clarify what performance information it will expect from arm’s-length bodies to be able to assess their effectiveness. As new sponsorship arrangements between Defra and its arm’s-length bodies become more established, Defra should work with regulators to ensure that it has what it needs for oversight of their performance and to manage any issues.

1 Comptroller and Auditor General, Achieving government’s long-term environmental goals, Session 2019–2021, HC 958, November 2020.

2 National Audit Office, Principles of effective regulation, May 2021.

Part One

The Department for Environment, Food & Rural Affairs’ (Defra’s) environmental aims and use of regulation

1.1 This part of the report covers:

the background and scope of the report;

Defra’s ambitions and overall performance to date;

Defra’s plans for using regulation to meet its ambitions; and

Defra’s understanding of the current regulatory arrangements and timescales for regulatory reform.

Background

1.2 The government pledged in 2018 that this would be the first generation to leave the natural environment of England in a better state than it inherited. In November 2021, Parliament passed the Environment Act 2021. It required the government to set binding statutory targets in four priority areas: air quality; water; biodiversity; and resource efficiency and waste reduction. Achieving these targets will require action, not just from the government but also from businesses and individuals.

1.3 The Department for Environment, Food & Rural Affairs (Defra) has lead responsibility in government for all environmental policy areas except net zero. It has published specific sector strategies setting out goals, including for waste, air quality and water. In January it published its 2023 Environmental Improvement Plan, which sets out the policies and actions it will take to achieve statutory and interim targets. However, Defra told us that some of these measures, including reforms to farm payments through Environmental Land Management schemes (ELMs), are yet to take effect (for example on species abundance).

1.4 While the government has set out its broad goals in this area, it has not yet set out operational plans to achieve them. In 2020 we reported on the government’s arrangements to achieve its long-term environmental goals.³ We found that the government had not yet developed clear and comprehensive delivery plans for how it would deliver these objectives, or how it would coordinate the large range of activity needed to achieve its ambitious outcomes. We recommended that it clarify its environmental ambitions so that statutory targets would be part of a coherent suite of specific and measurable medium- and long-term objectives. We also recommended that it develops a delivery plan for how planned outcomes were to be achieved. This report reviews progress against these recommendations, where they are particularly connected to Defra’s use of regulation. Our work on local air quality and waste crime in 2022 also found that progress in both areas had been slower than expected.⁴

1.5 One of the key tools government can use to encourage action is regulation. We use the term regulation to describe a set of rules and expected behaviours that people and organisations should follow, that will often involve bodies with regulatory powers enforcing and influencing compliance with those rules or behaviours. This is a broad term which can encompass a wide range of activities and approaches, from strict and prescriptive rules and enforcement, such as bans and issuing permits, to principles-based approaches, such as environmental planning and reporting, to lighter-touch incentives through guidance and codes of practice.

1.6 The Environment Agency and Natural England are two of the largest arm’s-length bodies in the wider Defra group, with responsibilities to regulate the natural environment. In this report, we have examined Defra’s relationships with these two regulators in particular. We have used case studies to understand how regulation is working in practice. We looked at one case study in each of the government’s four priority areas.

Within air quality, we looked at the process of using Best Available Techniques to set requirements for processes and emissions in industries that are required to have environmental permits for their work. This is managed by the Environment Agency.

Within water, we looked at the Water Framework Directive which, alongside other interventions, sets out requirements for a range of regulators and industries to work together to take action to improve the quality of inland waters such as rivers and lakes. This is managed by the Environment Agency.

Within biodiversity, we looked at the Habitats Regulations Assessment (HRA), which requires competent authorities such as local government to review planned projects to assess their environmental impact and ensure that key habitats and species are protected. Natural England has to be consulted by the competent authority when undertaking an HRA.

Within waste, we looked at the development of Extended Producer Responsibility regulations, which will require manufacturers to identify and pay for the disposal of packaging materials they produce. Work is currently being carried out by Defra to prepare for this scheme.

Scope of this report

1.7 This report examines whether Defra and its arm’s-length bodies are using regulation effectively to achieve its environmental objectives. It maps out where the government is already using regulation, often alongside spending or other government activity. It also assesses how regulation is working in practice in a number of case studies of individual regulatory interventions. Our evaluation and recommendations are based on our good practice guidance on the principles of effective regulation. The report does not assess the government’s overall performance against environmental goals or the coherence of environmental strategies.

1.8 The report covers:

the extent to which the government has set a clear direction for environmental regulation and how well it understands the fit of its regulatory interventions into its overall ambition;

how well Defra oversees its regulatory bodies in respect of delivering its environmental objectives, and evidence on current regulatory performance and capacity to deliver; and

whether Defra has suitable arrangements for evaluating and measuring the performance of regulators and regulatory interventions in supporting Defra’s environmental objectives.

Defra’s environmental aims

1.9 The government published a 25 Year Environment Plan (the Plan) in 2018 as part of its broader ambition for this to be the first generation to leave the environment in a better state. The Plan established 10 overarching environmental goals, which included clean and plentiful water, clean air, and thriving plants and wildlife. It set out the government’s intention to report on progress annually, develop a set of metrics to assess progress against, and refresh the Plan periodically. Figure 1 provides an overview of the key events in the development of the government’s environmental policy between 2018 and 2023.

1.10 In November 2021, Parliament passed the Environment Act, which put the government’s environmental plans on a statutory basis. It required Defra to produce a policy statement on environmental principles, a long-term Environmental Improvement Plan, and legally binding targets in four priority areas, as well as specific targets for fine particulate matter and species abundance. In December 2022, the government published the statutory targets for air quality, water, biodiversity, and resource efficiency and waste reduction, which came into force in January 2023. As Figure 2 shows, many of these targets relate to existing commitments in the 25 Year Environment Plan, where Defra has already begun work to develop indicators via its Outcome Indicator Framework. However, for more than half of the 13 targets, there are no finalised performance indicators in the Outcome Indicator Framework. Defra has stated publicly that its progress reporting for the Environmental Improvement Plan will draw upon evidence from the Outcome Indicator Framework and other appropriate sources.

1.11 The Environment Act 2021 also put in place accountability arrangements via the Office for Environmental Protection (OEP). The OEP is required to monitor progress towards the government’s statutory targets and wider objectives. In January 2023, the OEP reported that, based on the data and information available, the 25 Year Environment Plan has so far failed to bring about the changes needed, at the pace and scale required, to meet the government’s stated ambitions for the environment in England.

Defra’s plans for using regulation to meet its ambitions

1.12 In January 2023, Defra published its Environmental Improvement Plan (EIP). Regulation is included as a tool to deliver all ten of the goals set out, alongside activity such as direct funding projects and plans to facilitate private investment. The Environmental Improvement Plan recognises that Defra requires action to be taken by individuals, businesses and government to achieve its objectives. The EIP also identifies the need to maintain and improve existing regulatory frameworks including specific commitments such as expanding environmental permit regulations, reviewing farming and building regulations, and supporting regulators in ensuring compliance. Government also published an environmental principles policy statement, which states that government policy should aim to prevent environmental harm before it has occurred, in preference to addressing it afterwards. This is often achieved through regulating, for example by banning harmful activity.

1.13 Some stakeholders have raised concerns about whether Defra will be able to meet its environmental objectives. The OEP responded publicly to Defra’s delay in publishing the statutory targets, highlighting the urgency and pace needed to drive environmental improvement. In November 2022, the Environmental Audit Committee wrote to Defra expressing concern that Defra had not provided strategic direction for regulators and industry to take action. It noted delays to other environmental policies, including the Extended Producer Responsibility Scheme and River Basin Management Plans. Previous National Audit Office work concluded that government needed to pick up the pace to achieve its objective of improving the natural environment within a generation.⁵

1.14 Stakeholders we interviewed expressed concern about Defra’s engagement with them on the Environmental Improvement Plan (EIP). Our principles of effective regulation guidance⁶ sets out the importance of regular, formal engagement with stakeholders to share thinking and forward plans. Stakeholders, including industry bodies and environmental groups, said that they had limited engagement on the development of the EIP and were awaiting further details on funding and resourcing for the commitments Defra had announced. Defra stated that the EIP would act as a tool to help private and third sector organisations understand how environmental policies and targets fit into a broader growth model, and how they can respond to it in line with their own business models. While stakeholders have welcomed the publication of the EIP, some industry stakeholders raised concerns that it does not clearly set out the impact on, and changes required from, different sectors of the economy.

Defra’s understanding of regulation

1.15 When setting up or considering changes to a regulatory system, government must have a strong understanding of the overall regulatory landscape to avoid costly, untimely or disruptive remedial action later on. We reviewed the structure of Defra’s regulatory landscape and its understanding of it.

1.16 Defra’s regulatory landscape is complex.

We identified around 30 bodies whose work supports Defra’s policy objectives (Figure 3). These include: Defra’s arm’s-length bodies, such as the Environment Agency and Natural England; and other government departments and their arm’s-length bodies, such as the Department for Transport and the Driver and Vehicle Standards Agency. Local government and planning authorities can also be involved. Defra needs clear governance and coordination mechanisms to be able to manage the whole process across this range of bodies.

Regulatory interventions often span policy areas. For example, interventions aimed at improving water quality, such as sustainable farming incentives and pollution incident reduction plans, also support biodiversity targets through protection of habitats and species. Defra therefore needs an integrated understanding of how its regulatory interventions in support of different policy objectives affect each policy area.

Regulation covers a wide range of industries, including property developers, manufacturers, agriculture and the water sector, and each industry is subject to multiple regulations. For example, farmers are subject to a range of standards, permits requirements, funding requirements and specific controls if their land includes protected sites. Therefore, Defra must consider how all environmental regulations, delivered across government, affect stakeholder groups and ensure that it understands stakeholders’ perspectives.

1.17 As outlined in Figure 4 regulatory interventions cover a wide range of intervention types (bans, financial incentives, reporting requirements), levels of compliance required (mandatory or voluntary), and levels of enforcement applied (no enforcement, improvement notices, financial penalties, etc.).

1.18 Interventions in different policy areas can operate in similar ways and knowledge and experience can be shared to improve their effectiveness. For example, Defra’s 2018 ban on microbeads helped inform its development of other, similar product bans such as plastic straws and single-use plastics. Similarly, Defra has shared knowledge and experience when developing voluntary codes of practice for industry, such as the Plastics Pact (which requires manufacturers and retailers to work to reduce plastic waste) and the Courtauld Commitment (which requires businesses across the UK food chain to work to reduce food waste, emissions and water stress). There may be opportunities for Defra to share its learning more widely across its range of planned and existing regulatory interventions. For example, sharing lessons learnt from implementing plastic bans to bans of horticultural peat (to preserve habitats for biodiversity) or household coal (to improve air quality).

1.19 We reviewed Defra’s understanding of the current regulatory landscape in its four priority areas. This should include an understanding of the way that different regulatory interventions interact including any gaps, and the contribution each piece already makes towards achieving the government’s goals. We found that Defra’s understanding of the regulatory landscape varies by policy area.

Waste and resources: Defra published a Resources and Waste Strategy in December 2018. This involved mapping commitments made within other government strategies such as the 25 Year Environment Plan and the Clean Growth Strategy. In 2020, alongside its first progress report, Defra published an evaluation plan that outlined how the impact of policies would be evaluated. It also commissioned an evaluation programme in 2022, to understand what has and has not been successful in implementing the strategy.

Air quality: Defra published a Clean Air Strategy in 2019 and has an established programme for measuring levels of air pollutants. In July 2021, Defra mapped the small industries and standards within its industrial emissions environmental permitting framework, so that it could prioritise work to meet targets on particulate emissions. In August 2021, Defra also examined the impact of net zero policy interventions on the delivery of air quality targets. Defra has done, or is planning to do some specific evaluations, for example on its domestic fuel regulations and local plans to manage breaches in nitrogen oxide levels. In 2022, it started to review the regulatory system for industry emissions, including industries outside environmental permit regulations, to ensure that the system aligns with government priorities such as new statutory targets and environmental principles.

Biodiversity: in 2011 Defra published a biodiversity strategy on how it would implement international and EU commitments up until 2020. It is yet to publish an update. It published a Nature Recovery Green Paper in March 2022, which outlines a range of proposals to reform the regulatory system. In July 2022 Defra developed its understanding of the risks to achieving its species abundance target by identifying gaps in evidence, and mapping regulatory interventions across all policy areas that contribute to achieving the target. Through this work, it produced a risk assessment and delivery plan for achieving the target.

Water: in December 2021, Defra recognised that it needed to develop a plan for water that integrated regulatory interventions across multiple policy areas to achieve its targets. In April 2023, Defra published a Plan for Water, covering water quality and supply. It also said, in December 2021, that it planned to commission research to understand how to target interventions to achieve its goals, but it has not yet set out a timetable for this.

1.20 Defra needs a clear understanding of how existing regulation contributes to environmental goals, so that it can identify gaps and plan regulatory changes. For example, on air quality, environmental permit regulations set emission standards for many industrial sectors including 70 guidance notes for small industry sectors. For larger industrial sites the impact of regulation is well reported. However, there are significant gaps in Defra’s understanding of smaller industrial sites. For example, smaller sites covered by permitting regulations do not have emission reporting requirements, and data are collected on an annual, aggregate level through the National Atmospheric Emissions Inventory. Therefore, Defra does not understand what impact environmental permits are having on local emission concentrations. It also does not have sufficient data to identify gaps where industries are not covered by permits, or where action is needed to tackle local emissions. It does not yet have a plan for obtaining this data.

Timescales for regulatory change

1.21 It can take several years to make changes to, or put in place, new regulation, particularly where this requires legislative change. Figure 5 sets out the main stages in changing regulation where a legislative change is needed.

1.22 Our case studies show how these timelines work in practice.

Defra is developing a new process for ‘Best Available Techniques’ (BAT) for industrial emissions following the UK’s exit from the EU. Of the 26 sectors it regulates, 10 have recent EU BAT in place. It started reviewing BAT for four of the remaining sectors in late 2022 and intends to publish its conclusions in the second half of 2023. Defra will then have to agree secondary legislation to update the regulations. It will give existing sites four years to implement BAT. Improvements in these sectors will begin when the secondary legislation comes into force, but existing sites will not have to implement improvements until 2027.

In its 2018 Resources and Waste Strategy, Defra announced that it intended to introduce Extended Producer Responsibility (EPR) regulations for packaging. It has since held two public consultations and published three impact assessments on the proposed policy. The Environment Act 2021 contained provisions for the introduction of EPR schemes. Despite initial plans to roll out EPR for packaging by 2023, Defra has since announced that the scheme will be phased in from 2024. Defra has identified business readiness as a risk to the scheme’s delivery.

1.23 Some of the changes that Defra is considering making to its environmental framework would require legislative change and may be constrained by pressures on Parliamentary time. For example, significant reform of habitats regulations, including establishing a single assessment process across all protected sites, would need new primary legislation. In July 2022, Defra’s biodiversity targets board noted that a potential risk to delivering on the targets was insufficient time available for developing or introducing new primary legislation, should this be required.

3 Comptroller and Auditor General, Achieving government’s long-term environmental goals, Session 2019–2021, HC 958, November 2020.

4 Comptroller and Auditor General, Investigation into government’s actions to combat waste crime in England, Session 2021-22, HC 1149, April 2022; Comptroller and Auditor General, Tackling local breaches of air quality, Session 2022-23, HC 66, June 2022.

5 Comptroller and Auditor General, Achieving government’s long-term environmental goals, Session 2019–2021, HC 958, National Audit Office, November 2020.

6 National Audit Office, Principles of effective regulation, May 2021.

Part Two

Oversight of regulatory delivery

2.1 This part covers:

the Department for Environment, Food & Rural Affairs’ (Defra’s) oversight of its regulators;

evidence on the performance and capacity of the main regulators that Defra relies on to meet its objectives; and

Defra’s and regulators’ work to update or reform regulation.

Defra’s oversight and governance of regulatory arrangements

Overarching governance

2.2 Defra has many different governance boards for its environmental work, but it is not always clear how they relate to each other. Figure 6 sets out the main boards Defra has identified as having oversight of its environmental work, including regulation. These boards are supported by further programme boards in specific areas.

2.3 In 2020, we reported that Defra had not specified which of the oversight boards linked to the 25 Year Environment Plan was responsible for setting action plans to require improvements, should it identify any underperformance. In January 2023, the Office for Environmental Protection (OEP) described Defra’s governance arrangements as “complex and unclear”, citing in particular the number of boards involved in guiding implementation and coordinating work on the ground, and limited evidence of their impact.

2.4 Defra set up a cross-government 25 Year Environment Plan Board partly in response to our 2020 report Achieving government’s long-term environmental goals. It is responsible for cross-government delivery of the plan. Attendees include: the Department for Energy Security & Net Zero; the Department for Levelling Up, Housing & Communities; and the Department for Transport.

2.5 In 2022, Defra changed the structure and operation of boards following a review by Defra’s Environment Committee, which identified that the governance arrangements:

required simplification, and contained too many boards, with repeat and nugatory commissions;

had grown on an ad-hoc basis and that there were limited controls or framework for governance, as well as lacking a clear sense of how the boards were linked; and

highlighted the need for continued improvement of its risk escalation and performance management.

Defra told us that it simplified its board structure and created clearer hierarchies by: aligning the boards and their functions to 25 Year Environment Plan outcomes; removing boards without decision-making functions; and merging those with similar roles. For example, Defra removed the Future Environment Committee which sat at the same level as the Environment Committee, and replaced the Domestic Nature Board with the Biodiversity Targets Board. It also set out which director had responsibility for each board.

2.6 Figure 6 shows Defra’s current board structure. Defra has identified that further changes to board structures and functions are needed as its plans develop. For example, the Environmental Improvement Plan (EIP) and Targets Delivery Board was set up specifically to oversee the development of the Environment Act targets. In March 2023, Defra began work to consider the governance needed for delivering the EIP, including reviewing required metrics and data. Defra stated in the EIP that its 25 Year Environment Plan Board would be the central cross-government authority to ensure that environmental principles are being applied and to monitor action towards Environment Act targets and the EIP. As it continues to develop its governance arrangements, Defra will need to take into account the balance between focused oversight of specific areas and the ability to provide integrated oversight to bring different areas of activity together.

Defra’s relationship with regulators

2.7 The biggest regulators in the Defra departmental group are the Environment Agency and Natural England. They operate separately from Defra, although Defra can set their priorities directly. For the 2022-23 financial year, the Secretary of State for Environment, Food and Rural Affairs asked the Environment Agency to prioritise flooding, farm and sewage inspections, and waste crime, while Natural England was asked to work on six areas including nature recovery and water quality. These priorities reflect Defra’s overall departmental objectives.

2.8 Defra is introducing new accountability arrangements for its arm’s-length bodies, including regulators. It is developing a programme of regular engagement between ministers and the chairs of the 10 most significant arm’s-length bodies. It is developing performance information to support engagement, particularly on how they are delivering their own and Defra’s objectives. Defra is reviewing its framework agreements with arm’s-length bodies and is currently focusing on the Environment Agency. All framework agreements with Defra should be formally reviewed every three years. Its current Environment Agency framework has not been reviewed since it was agreed in August 2017. Defra told us that an updated version is due to be published in summer 2023.

2.9 In 2022 Defra re-introduced sponsorship teams within the core department for its arm’s-length bodies, in response to Public Accounts Committee recommendations and Cabinet Office guidance, but it has not yet finalised how they will work. The sponsorship teams for the Environment Agency and Natural England sit within the Environment, Rural and Marine Group but work across Defra. The sponsorship teams we spoke to do not all have individual terms of reference yet. However, as part of the process, Defra’s central public bodies team is working with sponsorship teams to use the Cabinet Office’s Sponsorship Code of Good Practice to assess and improve the maturity of its sponsorship relationships. Sponsorship teams told us that they saw their role as including the coordination of performance information, agreeing priorities with arm’s-length bodies and support for governance.

2.10 As Figure 7 demonstrates, officials from arm’s-length bodies have been introduced onto governance boards throughout Defra. Defra has also introduced more discussions between senior sponsors and regulators, and ministers are engaging more regularly with the senior leadership of arm’s-length bodies. Defra told us that this change forms part of its work to update its accountability approach. However, both Defra and regulators have identified areas where existing mechanisms for oversight do not provide either side with what they need.

Some Defra policy teams expressed concern that they do not have enough oversight of individual regulatory interventions. Sponsorship teams recognise that they need to review arm’s-length bodies’ performance reporting to ensure that what they have aligns to Defra’s objectives.

Some officials at the regulators we spoke to felt that concerns they raised, for example that their funding is insufficient to deliver their responsibilities, did not get resolved. The sponsorship teams told us they have worked with policy teams to set out routes to escalate issues with performance, but this is still in the early stages.

Regulatory performance and capacity

2.11 Defra and its arm’s-length bodies will require increased staff and resources to deliver the scale of activity set out in the Environmental Improvement Plan (EIP) over the long term. While the EIP mentions the importance of Defra’s arm’s-length bodies, and the need to embed skills, capacity and capability across government, it does not set out what this might involve for Defra or its arm’s-length bodies. Regulators have set out to Parliament that they envisage needing increased resources in the next spending review, but are working through the detail with Defra. Defra told us that it and regulators are currently estimating their requirements.

2.12 Defra’s two largest regulators have grown in recent years to reflect additional responsibilities and are facing workforce issues. They have been recruiting staff to fill existing vacancies planned under Spending Review 2021 and committed to again in the EIP. As at February 2023, the Environment Agency had 12,224 full-time equivalent (FTE) staff, around 600 or 5% below the level of 12,815 FTE it has funding for. The Environment Agency told us that, since April 2022, it has recruited over 2,000 new starters and that these members of staff are being brought up to the required level of capability. In its reporting to Defra, it has highlighted the impact of recruitment on the overall capability of the organisation, citing a ‘capability gap’ as it recruits and trains new staff. In September 2022, Natural England was below its year-end target of 2,980 FTE by 268 FTE – a gap of almost 9%. By February 2023 its staff resource was 2,979 FTE. Natural England recognises that it has a significant risk around capability that it plans to mitigate by increasing spending on learning and development for 2023-24.

2.13 Defra’s and regulators’ regulatory performance has recently come under scrutiny and this is leading to increased work. In the water sector, the Environmental Audit Committee found failures in the regulatory arrangements for monitoring, governance and enforcement on water quality. It said that a step change in regulatory action alongside water company investment and cross-catchment collaboration with farmers and drainage authorities was urgently required. The OEP is investigating the regulation of combined sewer overflows. Defra told us that, as the Environment Agency responds through increased monitoring and inspections of water quality in response to these issues, it is likely to detect more issues that require intervention and further work, which will ultimately benefit the environment. Defra recognises that work is required in multiple areas - for example, our investigation into waste crime⁷ found that the Environment Agency did not have the data it needed to identify and assess the full extent of waste crime, and therefore to effectively prioritise its resources. The available evidence indicated that the incidence and cost of dealing with waste crime across England was increasing. Defra considers that increased monitoring and inspection across its regulators and other delivery bodies will ultimately lead to greater compliance levels and a longer-term reduction in workload.

2.14 In each of our case studies, performance is behind where it needs to be on at least some aspects of the statutory target that the regulation, alongside other interventions, supports (Figure 8). For example, the government’s target is for 75% of water bodies across England to achieve good ecological status by 2027. However, only 16% of water bodies have so far achieved this status, with little change since 2009. Similarly, 75% of protected sites must be in a favourable condition by 2042. As of 2021, only 38% of Sites of Special Scientific Interest in England have met this target.

2.15 In three of our four case studies, the regulators and Defra were dealing with clear operational challenges.

Extended Producer Responsibility: Defra’s Extended Producer Responsibility regulations are expected to be operational by April 2024. There have been delays in developing legislation, finalising the programme delivery plan, and deciding on the functions of the Scheme Administrator. In June 2022, Defra reported that a key risk to delivering its collection and packaging reforms within committed timeframes was insufficient capacity, with 32 full-time equivalent vacancies across project delivery, finance and digital. As of March 2023 there were 21 additional required posts for the reforms.

Best Available Techniques in Environmental Permitting: the Environment Agency’s permitting service licenses activity, including for the industrial sectors that operate under Best Available Techniques regulation. It has been under considerable capacity and capability pressures due to a lack of staff and a backlog of permit applications has built up. Despite this backlog, Defra is considering increasing the demand on the service by bringing more sectors into the Best Available Techniques regime. Ammonia emissions have been increasing since 2013, and 80% of emissions derive from the agricultural industry. In January 2023, Defra announced that it would consult this year on extending environmental permitting to cover dairy and intensive beef farms, after first announcing in 2019 that it planned to do this by 2025. The Environment Agency has begun preliminary work to develop its evidence bases and identify how best to put the extension into practice.

Water Framework Directive: the Environment Agency published the third cycle of River Basin Management Plans in December 2022, a year later than the statutory deadline for doing so. Defra attributed the delay to the Environment Agency diverting resources to its COVID-19 work. In January 2023, the Environment Agency identified that its lack of resources was a significant risk to its ability to implement the plans and meet legal requirements by the target date of 2027. It is taking steps to mitigate this risk.

Updating regulation to reflect current priorities

2.16 Defra’s body of regulation is based on legislation that varies in age. For example, the first protected sites were designated under the National Parks and Access to Countryside Act 1949, and legal protections for wild birds were implemented in 1981 through the Wildlife and Countryside Act. The regulation reflects the objectives and regulatory practices at the time, including targets set by the EU. Defra is reviewing its retained EU law. In the Environmental Improvement Plan, it announced that it was also considering updating regulation in areas such as farming, water quality, woodland creation and building standards. However, Defra does not have a structured approach for conducting these reviews or a timescale for completing them.

Reviewing retained EU law

2.17 Much of the legislation underpinning the UK’s environmental regulations originated in the EU, including on air quality and nature conservation standards. The Retained EU Law (Revocation and Reform) Bill introduced to Parliament in September 2022 will require Ministers in each department to preserve, restate, modify or replace its retained EU law before it is revoked at the end of 2023.

2.18 Defra is currently in the process of identifying its retained EU law and deciding what to do with each piece of legislation. The government’s retained EU law dashboard currently shows that Defra has at least 1,780 pieces of retained EU law, the most of any government department. Defra has revised this figure upwards several times. Industry and environmental group stakeholders have raised concerns about Defra’s capacity to manage the scale of the review, the impact on other aspects of its environmental work programme, and the lack of opportunity for consultation and scrutiny on proposed changes.

2.19 Given the time and resources needed to review and amend this volume of EU law, Defra’s default position will be to preserve most of its retained EU law in domestic law for the December 2023 deadline. Defra told us it is currently estimating the resources it will need to complete the exercise. In some areas, regulators are supporting Defra to review pieces of legislation by providing insight into aspects of the legislation that could be changed.

Reviewing other regulatory frameworks

2.20 In two of our four case studies we found that Defra is considering how it can update existing regulation. However, work is at an early stage.

In 2022, Defra began considering ways to improve the Habitats Regulations Assessment tool, which is used by competent authorities such as local planning authorities to maintain the ecological status of sites protected under European legislation. This work is part of Defra’s wider wildlife reform programme. This programme aims to change the whole system of protected sites and species regulations from nature protection to nature restoration. Defra estimates that the system currently consists of 33 individual pieces of legislation.

In November 2022, Defra indicated that it may want to make changes to improve the operation of the Water Framework Directive. The Environment Agency has commented publicly on the challenge of achieving good water quality status for rivers due to the Directive’s assessment and reporting requirements. No formal commitment to review the Water Framework Directive has yet been made.

Good regulatory practice

2.21 Through our case studies, we identified specific examples where Defra is planning or developing more efficient or effective ways of regulating. While environmental regulation is complex, there is scope for Defra to extend these approaches and ways of thinking across its regulation work more widely. The areas we have identified include the following.

Developing regulation that considers an integrated system and not just individual elements. For example, the work to develop Best Available Techniques examines emissions to water and land as well as air. River Basin Management Plans, a key tool in implementing the Water Framework Directive, bring together information on all the water bodies in a catchment area. This approach can be applied in different ways - by considering the impact of a regulated body on a range of environmental objectives, or by considering an environmental system as a whole.

Regulating to promote best practice as it develops. Best Available Techniques set standards for industry by working with them to identify emerging techniques that are effective at minimising the impact on the environment. It can then use the example of those industries that have already adopted technology to set the standard for the rest. Defra is considering applying this system to smaller industries to ensure that standards are updated and reduce pollution from these sources. Defra could also extend this approach to other systems that rely on a large number of stakeholders of varying levels of engagement and performance.

7 Comptroller and Auditor General, Investigation into government’s actions to combat waste crime in England, Session 2021-22, HC 1149, April 2022.

Part Three

Evaluation and performance measurement

3.1 This part covers:

the Department for Environment, Farming & Rural Affairs’ (Defra’s) performance information and reporting on regulation; and

how the Defra group evaluates its regulation.

Regulatory performance information and reporting

3.2 Defra reports on progress against its environmental objectives through its Outcome Indicator Framework. As Figure 1 and Figure 8 show, Defra has more work to do to develop some of the indicators. The Office for Environmental Protection’s (OEP’s) 2023 progress report highlighted the need for simple measures that are promptly reported and aligned to the government’s environmental targets and objectives.

3.3 Defra recognises that, although the data provide an indication of environmental trends across different areas, it does not establish links between observed trends and drivers of change, such as policy or regulatory interventions.

3.4 Our case study work illustrates why it can be difficult to link regulatory activity to the outcomes that Defra seeks to achieve.

Outcomes are generally delivered though the actions of third parties and it can be hard to measure the regulator’s impact. For example, to achieve biodiversity goals, Natural England sets standards and monitoring for habitats. It requires landowners to take action to preserve or improve the condition of their land.

More than one regulatory action may influence a particular outcome and it can be difficult or impossible to disaggregate the impact of different outcome measures. For example, the Extended Producer Responsibility regulations are intended to reduce waste by incentivising reducing and recycling packaging. It sits alongside other interventions also designed to reduce waste (such as improving household recycling). Defra is considering how it can address this risk within the future reporting of waste reduction schemes.

Performance reporting within the Defra group

3.5 Defra needs a performance monitoring framework for regulators that provides sufficient and clear information to assess their overall performance and make the best use of resources. In our 2019 study, Environmental metrics: government’s approach to monitoring the state of the natural environment, we reported on the challenge of linking specific interventions intended to influence industry to outcome data.

3.6 We reviewed performance information that the Environment Agency and Natural England report to Defra. We examined whether it supported Defra’s environmental outcomes, how often it was reported, and whether anyone in Defra was responsible for oversight and escalation of any issues that were identified.

3.7 The Natural England sponsorship team within Defra receives the regulator’s performance indicators every three months, as well as information on areas of corporate risk. In 2022-23, Natural England reduced the number of performance indicators it was reporting. It introduced an indicator on biodiversity in response to Defra policy team’s feedback, but noted the challenge of having performance indicators for all its funding due to the broad spectrum of work it carries out. Defra told us that it was in the process of reviewing how Natural England’s performance indicators aligned with Defra’s goals, and that it had not been able to report on this since January 2022.

3.8 Alongside its internal performance reporting, the Environment Agency provides a quarterly performance report to its sponsorship team. This is organised against the annual priorities given to them by the Secretary of State. However, the Environment Agency has set out its own three strategic goals and five-year aims, and the example performance report we saw did not report progress against these. The sponsorship teams do not currently share this performance information with the Executive Committee but it is shared with senior sponsors.

3.9 The link between regulatory activity and outcomes in our case studies was not always clear. In some cases, such as environmental permitting, it is clear to see the link between the regulatory intervention and air quality. However, in other areas we could not make the link to outcomes. Our 2019 report identified potential solutions to this challenge. These include developing clear logic models and theories of change to link inputs and outputs to outcomes. However, we have not seen evidence that Defra has routinely developed these models and none of the internal performance reporting we have seen provided information on how specific regulation interventions contributed to outcomes.

Evaluation

3.10 In addition to monitoring performance, departments should evaluate the impact of regulatory interventions to identify what works and to improve future design and implementation. The government’s Magenta Book guidance on evaluation outlines the need for robust and proportionate evaluation.⁸

3.11 Departments must also review any regulatory measures with expected annual net business impacts of over £5 million by carrying out Post-Implementation Reviews (PIRs). These examine: whether regulation has met the intended objectives; the direct and indirect impacts; whether regulation is still needed; and any improvements needed.

Defra’s evaluation strategy

3.12 Defra has a ‘hub and spoke’ model with a small central evaluation team, and most evaluation work is done within individual policy areas.

The Strategic Evaluation team sits in the Central Analysis and Insights division. It is a small team made up of under 10 social researchers and is responsible for maintaining professional standards of evaluation across the department.

The Better Regulation team, made up of four analysts and policy advisors, works separately to support policy teams deliver PIRs.

Most analysis staff sit within Defra’s wider teams, working at a range of grades and specialisms. For example, in September 2022, there were around 60 analysts in the Environmental Quality team and around 50 in the Environmental Strategy directorate. Analysts, particularly social researchers, in these wider teams are responsible for evaluations alongside other research and analysis.

Policy teams are responsible for conducting PIRs. These are sometimes carried out alongside evaluations or built into evaluation plans.

3.13 Within the Defra group individual policy areas (including policy and analytical teams) are responsible for building evaluation into the design, implementation and review of their interventions. For example, the business case for Extended Producer Responsibility regulations sets out in broad terms the proposed scope of evaluation work, and the evaluation strategy explains how information will be collected through design and implementation. As set out in HM Treasury’s Green Book, when reviewing the development of business cases, Defra checks that previous monitoring and evaluation evidence, where it exists, has fed into the cost and benefit analysis.

3.14 The central team provides training and support across the Defra group. It manages an Evaluation Community of Practice Group, which shares learning and best practice on the development and design of evaluations.

The central team conducts regular evaluation audits that set out what evaluations are being conducted by teams across the group, including Post-Implementation Reviews (where these are part of policy evaluations). The audit aims to understand what evaluation work is being carried out and where in Defra and to identify where evaluation work may need support from the central team. The audit does not bring together findings from different evaluations or routinely share these across Defra teams. In November 2022, an overview of the evaluation work being undertaken was provided to senior management at Defra’s Executive Committee.

HM Treasury’s 2020 Spending Review announced an Evaluation Taskforce to address strategic evidence gaps within government’s evaluation work. In response to this, in February 2023, Defra produced an evaluation strategy that sets out Defra’s vision and overarching goals for evaluation over the next three years. This prioritises evaluation activity on areas agreed with HM Treasury and the Evaluation Taskforce, based on criteria set out in the Magenta Book. These priority areas include Defra’s Resources and Waste Strategy and the Future Farming Countryside Programme including Environmental Land Management Schemes. It is in the early stages of taking this strategy forward.

Evaluation of regulatory interventions

3.15 In 2021 Defra commissioned evaluation work for its 25 Year Environment Plan (2018) including an initial process evaluation and an Evaluation Framework that will be applied to future reviews. This planned work is expected to finish in March 2023.

3.16 Defra has also committed to future evaluations of regulation that supports its environmental objectives. For example, it has commissioned the development of a theory of change and evaluation framework for its new biodiversity targets. Similarly, within its 2023 Environmental Improvement Plan Defra has committed to reviewing its farming regulations to make sure that they are fit for purpose and effective at preventing water pollution.

3.17 We found that Defra has done some evaluation in our case study areas, but there were limits in its approach and coverage. For example, Defra collated views and recommendations from stakeholders involved in the Habitats Regulations Assessment process on how to ensure that the regulation supports the government’s nature recovery ambitions. A working group examined how regulation is delivered in practice, and suggested process changes and legislative changes to the wider regulatory framework around protected sites. It did not evaluate the impact of the regulations on the ecological status of protected sites, which limits Defra’s understanding of how interventions contribute to environmental outcomes.

3.18 The Regulatory Policy Committee is concerned about failures across government to meet statutory requirements for evaluation work, and encourages departments to undertake robust evaluation by making recommendations on how to improve the monitoring and evaluation process in its opinions. It has not received the number of PIRs expected given the number that should have been completed. Failure to conduct PIRs means that departments are unable to make informed decisions over the future of those regulations. Due to pressures caused by EU Exit and the coronavirus pandemic, as of May 2022 Defra had a backlog of 62 outstanding PIRs out of a total of 91 that were required between 2016 and 2022. In March 2023, the backlog was 63 PIRs. Over the past year, the Better Regulation team has conducted planning and prioritisation work and developed a delivery plan to clear the backlog by 2024, alongside its wider schedule of statutory reviews. As part of its evaluation strategy, Defra intends to consider how to link PIRs to its wider evaluation work so that learning can be shared more widely and efficiently.

8 HM Treasury, The Magenta Book: Central Government guidance on evaluation, March 2020.

Appendix One

Our audit approach

Our evidence base

1 This report presents our independent conclusions on whether the Department for Environment, Farming & Rural Affairs (Defra) is using regulation effectively to achieve its environmental objectives. Our conclusions were reached following an analysis of evidence collected primarily between September 2022 and January 2023.

2 The evaluative criteria that we used to assess value for money were based on our 2021 Principles of effective regulation⁹ and included: the extent to which government has set a clear strategic direction for environmental regulation and how well it understands the fit of its regulatory interventions into its overall ambition; how well Defra oversees and governs its regulatory bodies in respect of delivering its environmental objectives; and the effectiveness of the arrangements for evaluating and measuring the performance of regulators in supporting Defra’s environmental objectives.

3 In forming our conclusions, we drew on a range of study methods and a variety of evidence sources, as described in the paragraphs below. We collated and analysed the evidence we obtained, using our evaluative criteria as a framework. We looked across different sources of evidence to support each of our findings. We triangulated any issues, brought to our attention through our interactions with government departments or other stakeholders, against other sources. Where necessary, we asked questions of the relevant parties and/or asked for more information to ensure that we focused on the highest-priority areas.

Methods

Interviews

4 We conducted 30 virtual interviews and meetings with representatives from Defra, the Environment Agency, Natural England, and wider stakeholders to inform our audit.

5 Meetings with Defra, the Environment Agency and Natural England. We interviewed officials from Defra, the Environment Agency and Natural England selected to participate because of their job roles and their relevance to the audit.

To discuss the strategic direction for environmental regulation and the government’s environmental objectives, we spoke to officials across the three bodies including the following teams at Defra: Environment Act; Environmental Regulations; and Environmental Improvement Plan Targets.

To discuss oversight and governance arrangements with arm’s-length bodies (ALBs) we met with teams including: Defra Natural England sponsorship; Defra Environment Agency sponsorship; Defra Environment Rural and Marine Directorate; Environment Agency Environmental Regulation; and Natural England Legislative Reform.

To discuss evaluation and performance measurement, we met with Defra teams including Evaluation and Better Regulation.

To discuss our case study areas, we met with both Defra and ALB teams including: Defra Air Quality policy; Defra Waste and Resources policy; Defra Biodiversity policy; Defra Water policy; Environment Agency Water Framework Directive; Environment Agency Best Available Techniques; and Natural England Habitats Regulations Assessment.

During fieldwork, we also held regular meetings with Defra to triangulate our understanding of other evidence sources.

6 Scoping discussions with experts and academics. While we were designing the study we consulted other public bodies involved in environmental protection to provide a high-level understanding of the system of regulation as well as to identify key areas of concern. We spoke to representatives from:

the Department for Business, Energy & Industrial Strategy (now the Department for Energy Security & Net Zero) to discuss the use of regulation on the government’s Net Zero Strategy;

the Office for Environmental Protection to discuss its role in holding government to account on environmental objectives; and

the Climate Change Committee to discuss the use of regulation in emissions reduction and climate change adaptation.

7 Semi-structured interviews with stakeholders and experts. We met with other stakeholders to discuss: the clarity of the government’s environmental objectives; engagement with Defra and the regulators; and the role of regulation in achieving environmental objectives. We spoke to representatives from the following organisations:

Aldersgate Group

The Broadway Initiative

ClientEarth

Friends of the Earth

Green Alliance

Local Government Association

National Farmers Union

Rewilding Britain

RSPB

Wildfish

Wildlife and Countryside Link

The Wildlife Trusts.

8 We structured interview data against themes as well as identifying additional themes emerging from the data. We analysed the data, reviewing it against the themes identified in our evaluative framework, as well as taking into consideration themes emerging from the data. We used the analysis to:

inform further lines of enquiry that were followed up with Defra; and

triangulate evidence from other sources (including our document review and evidence from wider stakeholders).

Document and literature review

9 We reviewed a range of published and unpublished documents to develop our understanding of: the evidence supporting Defra’s strategies; the use of regulatory interventions; governance arrangements across the Defra group; and how performance measurement and evaluation are used.

We reviewed previous National Audit Office reports on environmental protection, to identify previous recommendations, findings, and consistent themes. We also reviewed our Principles of effective regulation to provide a basis for our evaluation and recommendations.

We reviewed documentation in the public domain including the Environment Act targets, Environmental Improvement Plan 2023, Outcome Indicator Framework, consultation documents and responses, government strategies, and public reporting on progress including specifically on regulation.

We reviewed documentation provided by Defra, the Environment Agency and Natural England that is not available in the public domain. This included strategic and risk documentation, board minutes, and evidence and findings from internal audit reports and internal quality assurance work.

10 We reviewed each document against our overarching audit questions.

11 Defra was not able to identify and share Post-Implementation Reviews on our case study areas. This limited our ability to comment on the quality of their evaluation or any findings. We did not review in depth the Plan for Water, which Defra published on 4 April 2023, because this was after we had concluded our fieldwork.

Identifying existing regulations

12 In the absence of a comprehensive list of environmental regulations from government, we mapped out the landscape to understand the range of interventions used by Defra and its regulators, the number of bodies involved, the age of regulation, the cross over between policy areas, the range of industries regulated and the regulated entities.

13 To gather our evidence, we conducted a literature review of publicly available documentation, reviewed client documents and discussed regulations with stakeholders. We compiled the information into a list of interventions in an Excel file to facilitate comprehensive and consistent analysis. To quality assure our mapping work, we sought review from the NAO Defra Value for Money team and Defra policy teams.

Case Studies

14 We carried out four case studies of current and future regulatory interventions to provide insight into the operation of regulation to support key targets. The aim of the case studies was to:

understand what the regulation is, how it works and what it intends to achieve;

understand how it fits into the government’s overall objectives and other work in the same area;

understand how it links to statutory environmental targets;

understand what required reforms have been considered and what their impact will be;

review performance information and governance arrangements;

review evaluation information; and

understand risks and how they are managed.

15 We selected case studies to capture high-interest interventions delivered by Defra’s two biggest regulators. It is not a representative sample - the case studies are not generalisable and we have not drawn wider conclusions about the effectiveness of regulation from them.

16 The primary sampling criteria were:

priority area;

regulator type;

intervention type; and

client and stakeholder interest.

Overall, we achieved good diversity across the primary sampling criteria. The achieved sample is set out in Figure 8.

17 Each case study consisted of interviews with the Defra policy team and corresponding delivery team at the relevant ALB, as well as analysis of performance information data.

18 The fieldwork period of this study ran concurrently with other NAO work on waste reform, which included our case study topic, Extended Producer Responsibilities, within its scope. Where appropriate, we jointly attended meetings and shared relevant documents.

19 Our analysis of case study data was used to:

inform further lines of enquiry that were followed up with Defra;

triangulate findings from interviews with Defra staff and wider stakeholders; and

report on examples of best practice or areas of improvement across our study scope.

9 National Audit Office, Principles of effective regulation, May 2021.

Design and Production by NAO Communications Team
DP Ref: 011090-001