Artificial intelligence (AI) makes a mistake. Who pays for the damage?

More and more organisations use AI to assess applications, analyse data and support decision-making. But what happens when an AI system produces the wrong outcome and causes damage as a result?

Take an HR software provider that uses AI to rank job applicants. Because of an error in the model, the system rejects suitable candidates. The client misses out on talent and may also face claims from rejected applicants. This raises an important question: does the responsibility lie with the software provider, the developer of the AI model, the client, or the employee who relied on the outcome?

Multiple parties, shared responsibility
AI often involves several parties. An external supplier develops the model, a software company integrates it into a platform, the client provides data and an employee uses the output. Who is liable depends, among other things, on the cause of the error. Was the model unreliable? Was the application configured incorrectly or used for a different purpose? Or was there insufficient human oversight? Contractual arrangements also determine who is responsible for what. Legislation plays an important role as well. The European AI Act sets requirements for organisations that develop and deploy AI. Which obligations apply depends, among other things, on the type of application and the organisation’s role.

Not every AI error is a cyber incident
An AI system can function technically as intended and still produce an incorrect or discriminatory outcome. In that case, there is no hack, data breach or system outage. For software and SaaS providers, AI-related risks may therefore be more closely linked to professional or product liability than to cyber risk. The cause, type of damage, role of the organisation and policy terms all play a part in answering the key question: which insurance policy, if any, provides coverage?

Make clear agreements in advance
When using AI, clearly define:

• What the application may and may not be used for

• Who is responsible for data and system settings

• When human oversight is required

• Who investigates and corrects errors

• How liability and costs are allocated

Think about the claim before implementation
Liability questions often only arise after something has gone wrong. Have you already established who is responsible if AI leads to an incorrect decision?
Our advice is to identify where AI is used within your organisation, which parties are involved and what the potential financial consequences of an error could be. Include AI in your risk assessment and then check whether your contracts and insurance policies match those risks.

Meijers’ specialists are happy to help. For more information, please contact Sandy Buurman-Drenth, Senior Account Manager Tech.