Posted: 22nd October 2024

Over £1m in funding from the Advanced Research & Invention Agency (ARIA) will see the launch later this year of a raft of new research projects addressing AI safety. ARIA, a government-sponsored R&D funding agency, sponsors institutions to help unlock scientific and technological breakthroughs that benefit society.

Three new programmes, led by Professors Sam Staton, Alessandro Abate and Nobuko Yoshida, will bring significant developments to help safeguard the rapidly advancing field of AI:

Employing categorical probability towards safe AI  

Research by Professor Sam Staton aims to advance AI safety using categorical probability, which reveals how different structures are related to one another by organising and analysing statistical models and their composition.

SAINT (Safe AI ageNTs – Formal Certification of AI Agents in the world)  

Professor Alessandro Abate’s research will also tackle safe AI through what the project team terms ‘neural synthesis’. This builds on the established area of certificate synthesis, which analyses the dynamical behaviour of autonomous AI systems.

Probabilistic Protocol Specification for Distributed Autonomous Processes 

Research headed by Professor Nobuko Yoshida utilises a new theory of probabilistic session types (ProST), which aims to ensure safe AI systems by counteracting the uncertainty inherent in the behaviour of autonomous components.

These research programmes are in response to ARIA’s Safeguarded AI programme, whose goal is to usher in a new era of AI safety, unlocking the full economic and social benefits of advanced AI systems, while minimising risks. By combining scientific world models and mathematical proofs, these research projects will contribute to ARIA’s aim of building an AI gateway that will enable the secure channelling of AI’s vast potential.   

The ARIA investment includes £682k for Employing categorical probability towards safe AI, £234k for SAINT, and £85k for Probabilistic Protocol Specification for Distributed Autonomous Processes.

Find out more about ARIA’s Safeguarded AI programme at https://www.aria.org.uk/safeguarded-ai-creators/  

Editor's note: This article was updated on 15 Jan 2025 to reflect the correct funding figure for Employing categorical probability towards safe AI: from £480k to £682k.