AI systems fail differently from traditional software. A web server crashes visibly; a model that starts producing subtly wrong predictions can run for weeks before anyone notices. AI incident response must account for these silent failures, the probabilistic nature of model outputs, and the difficulty of determining root cause when the system is a learned function rather than explicit logic.

What Constitutes an AI Incident

Define AI-specific incident categories beyond standard service outages:

  • Model producing harmful, offensive, or dangerous outputs
  • Significant accuracy degradation detected through monitoring or user reports
  • Data breach through model memorization or prompt injection
  • Bias or fairness violation affecting a protected group
  • Model being used outside its intended scope with adverse consequences
  • Complete model failure (errors, timeouts, empty responses)
  • Cost anomaly suggesting system malfunction

Detection

AI incidents are detected through multiple channels:

Automated monitoring. Quality score degradation, drift alerts, error rate spikes, latency anomalies, and cost anomalies. These catch quantifiable problems quickly.

User reports. Complaints, support tickets, and feedback signals. Users often detect qualitative problems that metrics miss, especially for nuanced failures like tone, relevance, or context appropriateness.

Internal discovery. Team members notice issues during development, testing, or routine system review. Create low-friction channels for reporting concerns.

External reports. Researchers, journalists, or the public identify issues. Have a clear process for receiving and triaging external reports.

Triage

When an incident is detected, quickly assess:

Severity. Is anyone being harmed right now? Is there legal or regulatory exposure? Is the issue affecting all users or a subset? Severity determines response urgency and escalation level.

Scope. How many users are affected? Is the issue limited to specific inputs, user segments, or geographies? Is it getting worse?

Root cause hypothesis. Is this a model issue (bad predictions), a data issue (corrupt or missing input data), an infrastructure issue (service degradation), or a pipeline issue (stale model, failed deployment)?

Immediate Response

Severity 1 (active harm). Take the system offline or fall back to a safe default immediately. Communicate with affected users. Escalate to leadership. Notify legal if there is regulatory exposure.

Severity 2 (degraded quality). Enable additional guardrails, switch to a more conservative model configuration, or route affected traffic to a fallback system. Increase monitoring sensitivity.

Severity 3 (minor degradation). Increase monitoring, begin investigation, and schedule a fix within normal development cycles.

Investigation

Gather evidence. Collect traces, logs, model inputs and outputs, monitoring data, and recent deployment history. For AI systems, include: which model version is serving, when it was last updated, whether data pipelines ran successfully, and whether input distributions have changed.

Reproduce the issue. Attempt to reproduce the failure in a controlled environment. For model quality issues, gather a set of examples that demonstrate the problem and test them against previous model versions.

Identify root cause. Common root causes for AI incidents include: data pipeline failures delivering stale or corrupt data, model drift due to changing input distributions, deployment errors loading the wrong model version, prompt template changes with unintended consequences, and upstream API changes affecting tool use or retrieval.

Resolution and Recovery

Fix the root cause, validate the fix against the examples gathered during investigation, deploy the fix through your standard pipeline (do not bypass safety checks in the rush to fix), and verify resolution in production through monitoring.

Post-Incident Review

Conduct a blameless post-incident review within a week. Document the timeline, root cause, impact, response effectiveness, and action items. Focus on systemic improvements: better monitoring that would catch this sooner, guardrails that would limit blast radius, and process changes that would prevent recurrence. Add the failure mode to your evaluation suite and red team scenarios.