Developing and Evaluating an AI capability

i.AI is building tools to tackle the costly and time-consuming process of analysing government consultations. With the UK government running 700-800 consultations every year, costing around £20 million, taking months and often relying on contractors, the potential impact of innovation in this space is high. We've developed a Large Language Model (LLM)-based solution to streamline analysis, improve policy-making, and deliver better value for taxpayer money.

Our AI-driven method addresses consultation analysis in two ways: extracting common topics across all responses and classifying which topics are present in individual submissions. These steps are the most manual, costly and prone to human error. By automating them we hope to enable analysts and researchers to quickly gauge public sentiment, identify potential oversights and focus on using their insight to improve policy. This blog focuses on topic extraction. In particular, it will explain how we developed and evaluated this new capability, and what lessons we have learnt along the way.

Developing a Robust Approach to Topic Extraction

How do we know if the topics our model (or a human) extracts from a set of consultation responses truly reflect opinions in the responses? In other words, how do we shift from a subjective “vibes based” analysis of topic lists to an objective and quantifiable evaluation. This is the key issue we need to address if we want to build confidence around the reliability and effectiveness of our LLM topic generation application. We broke this problem down into 5 distinct steps:

1. Problem Definition

At its core, this is a topic modelling problem. Topic modelling is the identification and grouping of similar ideas across a range of texts, it is an active field of research. We decided that LLMs’ ability to process large amounts of free text data makes them ideal topic modellers. Our new approach chains together a series of LLM calls to produce a succinct and human readable list of topics. These topics are built with an awareness of the broader context of a consultation.

To evaluate our LLM-based topic modelling algorithm, we first needed to define the characteristics of a good topic model, which meant agreeing upon what made a good “topic”. Getting close to our users to understand their viewpoints and test ideas with them was crucial, and user research was invaluable in setting up these relationships and helping us learn from them. We defined topics as distinct viewpoints in the responses that explained why a person agreed or disagreed with the proposal. Our belief was that these topics would provide greater insight to policy makers looking to address concerns about proposed legislation.

Spending the time agreeing on these complex ideas before setting out on the project was essential. It reduced the risk of parallel streams of work diverging and gave us something concrete to evaluate against.

Lesson 1: Try and define what a “good” output looks like early, and test this with your users. You are likely to learn a lot, and this helps define how you will know if you have succeeded.

2. Evaluation Data Collection

One problem with topic modelling is the definition of a good topic is highly subjective, in other words - it's an unsupervised learning task with no ground truth labels. Trying to pull out topics from thousands of responses is an undeniably difficult task, and so it is no surprise there is large variation in the types of topics that have historically been identified in different consultations, run by different departments, and that human error creeps into the process.

To combat this, we used LLMs to generate synthetic consultation data where we knew exactly what topics were present in responses and how frequently they appeared. Our synthetic data generation involved these processes:

1. Consultation Generation: Generate a plausible synthetic consultation. This includes coming up with synthetic questions and plausible topics that could appear in answers to these questions.
2. Topic and Sentiment Sampling: Configurable sampling from different topics and agreement/disagreement positions, controlling the content of a synthetic response. This process allows us to quantify exactly how often each synthetic topic would appear in our evaluation dataset.
3. Persona Generation: Add additional colour to a given response by generating a fake backstory and emotional profile. This includes how likely a respondent is to make grammatical mistakes, and whether they're responding as an individual or on behalf of a fabricated organisation.

Lesson 2: If you don’t have ground truth data, then generating synthetic data could be a solution for you.

3. Metric Selection

Metric selection is closely related to problem definition. Once you have decided what good looks like, you need to decide how to measure it. Different performance metrics will reward different model behaviours, as a result they need to be chosen carefully. Defining optimal model behaviour should not be restricted to engineers, non-technical team members can provide a huge amount of value to this process. A useful byproduct of opening up discussions on metric selection is that it reinforces the shared conceptual framework on what you are aiming to achieve.

We decided that the goal of our topic generation algorithm was to produce the smallest number of topics that accurately captured all of the ground truth topics. We were able to do this by using an LLM as a judge, and creating our own custom evals that give the LLM our data and ask it some questions. Specifically, we tracked this using the following three metrics:

1. Precision: how well-grounded generated topics are. This metric would punish hallucinations, where the model produced topics that are not related to responses.
2. Recall: how many of the ground truth topics are well captured. This metric punishes a situation where ground truth topics were not picked up by the generation process.
3. Topic Count: Monitoring the number of topics produced.

Lesson 3: LLMs enable a wide set of options for generating metrics, particularly for free text outputs. Experiment to find out what works for your use case.

4. Prompt Development

To make the most of the teams’ diverse expertise, we conducted an internal hackathon, allowing all team members to contribute to prompt writing. A key consequence of the widespread uptake of LLMs is that a key skill required - writing prompts - has a low technical barrier to entry. While certain technical ideas from prompt engineering, such as including few-shot examples, undoubtedly improve performance, often the largest gains come from incorporating domain knowledge into a prompt. This process allowed us to test a range of prompting strategies and allowed non-technical team members to gain first-hand experience with prompting.

Lesson 4: You can use hackathons to bring a diverse set of views into the prompt generation process

5. Evaluation and Integration

The final step of this workflow is evaluating different prompts and integrating any changes that improve performance. Practically, we used our metrics to evaluate a range of prompts produced during our hackathon, to see how the specifics of a synthetic consultation (how long it was, how varied the topic frequencies were) impacted performance. This evaluation ultimately allowed us to identify a new approach to prompting that significantly improved our topic generation.

Our methodology has also made it trivial to continually test out new strategies. Quantifying the performance of our current strategy means that we can quickly benchmark new prompts, collecting robust evidence to justify whether or not we should include them into the pipeline. This benchmarking also gives us increased visibility on how external changes (i.e. new model releases) will impact our application.

Lesson 5: Quantifying the effectiveness of an approach facilitates rapid iteration and improvement of an algorithm

Conclusion

Our AI-driven approach to government consultation analysis has shown promising results, watch this space for a more detailed evaluation of our tool. Our goal is to enhance evidence-based policy making by automating repetitive tasks. We believe that giving experts high quality insights into consultation responses and more time to analyse them can bring about a meaningful improvement in the policy making process.

Moreover, working on this has taught us 5 key lessons, that we believe are transferable to other AI projects:

1. All team members should align on the goals of a project before work commences.
2. If you don’t have ground truth data, consider generating synthetic data.
3. LLMs enable a much wider set of options for generating metrics, particularly for free text outputs. Experiment to find out what works for your use case.
4. Use hackathons to bring a diverse set of views into the prompt generation process
5. Quantifying the effectiveness of an approach facilitates rapid iteration and improvement of an algorithm

N.B. The i.AI Consult tool is not actively being used to analyse any live consultations. It is designed to provide policy makers with a deeper insight into consultations not automate decision-making.