Source: Local Government Lawyer

Alexi Markham and Jocelyn S Paulley analyze what public sector and in-house teams generally require during the procurement stage of AI-enabled services. They emphasize that procuring AI is not merely acquiring an “AI-powered” tool but involves clearly defining the specific use case, setting appropriate requirements, fairly allocating risks, and establishing governance structures to ensure systems operate safely and reliably after deployment. Public sector AI use cases fall into four categories: supporting decision-making, research and monitoring, delivering services, and improving efficiency, with examples such as algorithms identifying companies likely to succeed in exporting by analyzing Companies House data, habitat mapping via satellite imagery for environmental policy support, and machine learning for sensitivity reviews before transferring records to the National Archives.

The discussion highlights that AI risk encompasses multiple dimensions including the system’s function, how it is used, the data it consumes, and the implications when outputs are inaccurate or difficult to explain. The UK regulatory approach to AI is context-based and proportionate, relying on existing sector-specific laws. Procurement of AI products raises issues involving transparency, fairness, privacy, bias, intellectual property, confidentiality, cyber security, liability, and compliance with evolving laws. Effective AI procurement benefits from early collaboration among legal, procurement, technical, and operational teams, paying particular attention to testing and governance post-contract award. Contracts should address transparency and explainability, accuracy and mitigation of hallucinations, bias and data quality, data rights and IP considerations, cyber security measures including supply chain exposure, and liability aligned with control over data, fine-tuning, and model updates.