Building Trust in AI: Introducing i.AI’s Assurance Principles

AI presents transformative opportunities for our economy and society. The rapid development of AI capabilities in recent years, particularly generative AI like Large Language Models (LLMs) such as ChatGPT, has sparked significant excitement about the potential applications and benefits of AI systems. The economic benefits are compelling, with external estimates suggesting the UK stands to gain £40 billion annually in public-sector productivity improvements by embracing AI, amounting to £200 billion over five years.

At the Incubator for AI (i.AI), we are set up to harness the opportunity of all types of AI to improve lives, drive growth, and deliver better public services. Developing AI products in the public interest motivates us as a team, and it is important that we do this responsibly to build trust. This requires a multidisciplinary approach to embedding human values and ethical considerations throughout the AI development lifecycle. 

Our assurance principles

At i.AI, we want to use our assurance principles to deliver AI for public good by developing robust products that build public trust. These principles are aligned with the Generative AI Framework for HMG, the AI Regulation White Paper, and DSIT's Introduction to AI Assurance.

1. Safety, Security, and Robustness: We believe that AI systems must not only function appropriately but also be secure against exploitation, safeguard users' data privacy, and be meticulously designed to minimise errors. Key considerations include:

• How extensively do we incorporate human oversight in the system?
• What safety measures are in place to mitigate potential harms?

1. Appropriately Transparent and Explainable: For AI to be genuinely embraced, there needs to be a level of explainability with regards to its outputs. Users need to trust the tool they're using, and that confidence comes from transparency. For this, we need to ask ourselves questions such as:

• Can we open source the code?
• If an AI makes a decision, can we unpack and explain the process clearly to someone not specialised in AI?

1. Fairness: Our aim is to create AI systems that are just and equitable. A significant part of this is ensuring that these systems don't perpetuate or introduce bias. This involves critical analysis of:

• The potential for the system to introduce bias.
• The quality and representativeness of the data we use.

1. Improvability and Openness to Challenge: As a team, we are adaptive and constantly learning from feedback to evolve our AI tools over time. Questions that guide this principle include:

• How can users report issues, and how swiftly can we address them?
• Have we engaged with the appropriate stakeholders and departments to ensure a well-rounded development?

1. Accountable with Clear Governance: We need to hold ourselves to account to deliver products that are aligned with these principles and our governance allows us to do that. In this regard, our key questions include:

• Are our governance processes robust and effective?
• Is the line of accountability clear and transparent to all stakeholders? 

By rigorously addressing these categories with pointed questions, we ensure that our AI systems aren't just advanced in terms of technology, but are also ethical, equitable, and ready to benefit wider society. We also need to ensure that our actions are proportionate; we are testing a lot of ideas quickly, many of which will fail before ever interacting with a real user, so it is important we don’t put undue burden on our early stage products.

How we tailor our assurance

AI is a rapidly evolving field, so it is important that we remain agile in our approach to product assurance. Also, each product is different, utilising different technologies in very different contexts, with different levels of risk, so it is important we tailor the assurance process to the product. There are three main factors that we consider in our approach as a team:

1. Development stage

When it comes to developing AI systems, the way we ensure a product's quality and effectiveness hugely varies depending on what stage it is in. Think of it like nurturing a plant; every growth stage requires a different type of care. For example, we would invest more time in user needs research at the scoping and Alpha stage to effectively shape the product’s direction. This is our chance to truly tailor the AI, setting its development off in the right direction by grounding it in real user insights. We might then consider a randomised controlled trial and red teaming exercise later on at the Beta stage to enable more rigorous testing and evaluation. This allows us to check that the AI behaves just as we expect under a variety of conditions. It’s a bit like putting it through a stress test — making sure it's up to the challenge across different scenarios and can handle whatever is thrown its way.

By carefully navigating these steps and adapting our approach at each stage, we not only meet the technical and ethical standards but also build robust, user-friendly AI systems that are ready for the real world. 

2. The user and nature of the product

At i.AI, we understand that the assurance of a product's safety doesn't just stop at code correctness or system reliability, it extends into how the product will be used, who will be using it, and the impact it aims to create. For products that face the public, such as tools used directly by consumers, we enforce much stricter safeguards compared to systems designed for specialised use within our expert teams.

All our products include a “human-in-the-loop” approach. This means that instead of creating fully autonomous AI tools, we design our systems to assist and enhance the effectiveness of everyday workers. An excellent example of this is Caddy, our AI-powered co-pilot designed for customer service functions. 

When thinking about how we govern and assure our AI products, we don’t stick to a rigid set of categories. Instead, we encourage exploring a variety of questions to really understand the nature of the AI tool we are developing. This approach helps in tailoring our assurance processes because we acknowledge that not all tools are created equal—each serves different purposes and faces different kinds of challenges.

Here are some of the key questions that guide us in our governance and assurance journey:

• What scale of impact could the outcomes have on user needs and public interest?
• Does the tool include conversational capabilities?
• How explainable are the decisions made by the AI?

These sorts of questions enable us to understand the nature of a tool and adapt our approach to assurance accordingly. By not adhering strictly to a one-size-fits-all approach, each AI tool is assured with nuances that consider its unique context and intended use.

3. External market factors

As new AI models emerge, our assurance methods may also need to adapt to accommodate advancements in technology and shifts in how AI is applied across various domains. This involves staying informed about the latest developments in AI algorithms, machine learning techniques, and data processing methodologies. By maintaining a flexible approach with our assurance processes, we can ensure they are robust, effective, and aligned with current best practices.

Changes in market dynamics, such as the introduction of new regulations, evolving user expectations, and increased competition, necessitate dynamic assurance processes that can respond swiftly and efficiently. Similarly, shifts in supply chain flows, including variations in supplier reliability, logistics, and geopolitical factors, demand adaptable assurance strategies to maintain integrity and performance standards. By implementing flexible and dynamic assurance processes, we can better navigate these complexities and ensure the successful deployment and operation of AI products.

What now?

At i.AI we are striving to build AI products and systems that deliver value for the public. To achieve this we have developed strong assurance principles, and we are now baking those principles into our wider processes. We will continue to iterate on this and make it easier for our engineers to assure their work, for example by releasing toolkits for each development stage of a product. This is part of a wider strand of work to develop our product strategy, which is helping us empower our product teams to solve real problems for real people.