Incident Reporting

The process of documenting and reporting AI system failures, errors, unexpected outputs, or harms. At the organizational level, incident reporting means maintaining a log of AI failures and near-misses to identify patterns and improve governance. At the regulatory level, the EU AI Act requires providers of high-risk AI systems to report 'serious incidents' — those causing death, serious injury, significant property damage, or grave fundamental rights violations — to national authorities. Building an incident log before it is legally required is one of the highest-value AI governance investments a small team can make.

A team's AI customer service bot gives a refund amount that contradicts the company's policy. The customer service manager logs it in the incident log: date, tool, what happened, how it was corrected, and whether the model configuration needs updating.