AI risk assessment is the process of systematically identifying what can go wrong with an AI system and determining whether the risks are acceptable. Unlike traditional software risk assessment, AI systems introduce probabilistic behavior, data-dependent failure modes, and emergent capabilities that require specialized evaluation approaches.

When to Conduct Assessments

Before development begins. A lightweight assessment at the design stage prevents investing in systems whose risks outweigh their benefits. Focus on use case appropriateness, affected populations, and regulatory exposure.

Before deployment. A comprehensive assessment evaluating the built system against identified risks. This is where you validate that mitigations work and document residual risks.

After significant changes. Model retraining, new data sources, expanded use cases, or changes to the operating environment warrant reassessment.

Periodically during operation. Risks evolve as the world changes. Annual reassessments for high-risk systems catch emerging risks from concept drift, new attack vectors, or changing regulatory requirements.

Risk Identification

Organize risk identification around these categories:

Performance risks. The model produces incorrect outputs. Consider: What is the base error rate? Are errors uniformly distributed or concentrated in specific subgroups? What is the impact of an individual error? What is the cumulative impact of systematic errors?

Bias and fairness risks. The model treats different groups differently in ways that are unjust or illegal. Examine performance across demographic groups, protected characteristics, and intersectional subgroups. Consider both direct discrimination and proxy effects.

Security risks. The model is vulnerable to adversarial attacks, prompt injection, data poisoning, model extraction, or membership inference. Consider both the technical feasibility of attacks and the motivation of potential attackers.

Privacy risks. The model memorizes or leaks training data. The model’s outputs reveal information about individuals in the training set. The model enables surveillance or profiling not intended by its design.

Reliability risks. The model fails under load, produces inconsistent outputs, or degrades silently. Consider edge cases, out-of-distribution inputs, and failure cascading through dependent systems.

Misuse risks. The model is used for purposes it was not designed for. Users over-rely on model outputs, apply the model to out-of-scope problems, or use it to circumvent controls.

Reputational risks. The model generates offensive content, makes embarrassing errors publicly, or is associated with harms reported in media.

Risk Evaluation

For each identified risk, evaluate:

Likelihood. How probable is this risk materializing? Use evidence from evaluation results, red team testing, analogous systems, and domain expertise. Be honest about uncertainty.

Impact. If the risk materializes, what is the severity? Consider harm to individuals, financial loss, regulatory penalties, reputational damage, and operational disruption.

Detectability. How quickly would you know if this risk materialized? Risks that are both likely and hard to detect deserve the most attention.

Risk score. Combine likelihood, impact, and detectability into a risk rating. A simple matrix (high/medium/low) is usually more useful than complex quantitative scoring, which implies false precision.

Risk Mitigation

For each significant risk, define mitigations:

Technical mitigations. Guardrails, content filters, confidence thresholds, input validation, output verification, rate limiting, and fallback mechanisms.

Process mitigations. Human review of high-impact decisions, escalation procedures, regular audits, and monitoring dashboards.

Organizational mitigations. Training for users and operators, clear documentation of system limitations, incident response procedures, and feedback channels.

Document the expected effectiveness of each mitigation and the residual risk after mitigation. Accept, transfer, or further mitigate based on your organization’s risk appetite.

Documentation and Review

Produce a risk assessment report that includes: system description, risk inventory with evaluations, mitigation plan, residual risks, and recommended monitoring. This document serves as the basis for deployment decisions, regulatory compliance evidence, and ongoing risk management. Review and update it according to the schedule defined in your governance framework.