Accuracy on a held-out test set is where model evaluation starts, not where it ends. A model with 95% accuracy that fails catastrophically on a critical subgroup, breaks under adversarial inputs, or takes ten times longer than the latency budget is not ready for production. Comprehensive evaluation examines a model from multiple angles to build confidence that it will perform reliably in the real world.

Performance Metrics

Choose metrics that align with your business objective, not just the most common ones for your task type.

Classification. Accuracy is misleading for imbalanced datasets. Use precision and recall (and their harmonic mean, F1) for each class. Use area under the ROC curve (AUC-ROC) for ranking quality and area under the precision-recall curve (AUC-PR) for imbalanced settings. For multi-class problems, report both macro-averaged and micro-averaged metrics.

Regression. Mean absolute error (MAE) is interpretable. Mean squared error (MSE) penalizes large errors more heavily. Root mean squared error (RMSE) is in the same units as the target. Use the metric that best captures what matters for your use case.

Ranking. Normalized discounted cumulative gain (NDCG), mean reciprocal rank (MRR), and precision at K measure different aspects of ranking quality. For search and recommendation, these matter more than classification metrics.

Generation. BLEU, ROUGE, and BERTScore measure similarity to references. For open-ended generation, use human evaluation or LLM-as-judge frameworks to assess fluency, relevance, faithfulness, and helpfulness.

Slice-Based Analysis

Aggregate metrics hide important details. Break down performance by:

  • Demographic groups relevant to your application (age, gender, geography, language)
  • Input characteristics (short vs. long inputs, simple vs. complex queries, common vs. rare categories)
  • Data source (if training data comes from multiple sources)
  • Time period (to detect temporal performance variation)

A model that performs well overall but poorly on a specific subgroup is not acceptable for production deployment. Define minimum performance thresholds per slice, not just overall.

Fairness Evaluation

Demographic parity. Are positive prediction rates similar across groups?

Equal opportunity. Are true positive rates similar across groups?

Predictive parity. Is precision similar across groups?

These metrics can conflict. Choose the fairness criterion that aligns with your use case’s ethical requirements and legal obligations. Document your choice and rationale.

Robustness Testing

Adversarial inputs. Test with inputs designed to fool the model: typos, paraphrases, irrelevant additions, and format changes. A model that fails on minor input perturbations is fragile.

Out-of-distribution inputs. Test with data from outside the training distribution. How does the model behave? Does it fail gracefully (return low confidence) or fail silently (return high-confidence wrong answers)?

Stress testing. Test at production load levels to verify latency and throughput requirements are met. Test with concurrent requests to verify thread safety and resource management.

Calibration

A model that says it is 90% confident should be correct 90% of the time. Calibration measures this alignment between predicted confidence and actual accuracy. Well-calibrated models enable better decision-making downstream. Use reliability diagrams and expected calibration error (ECE) to assess calibration. Apply temperature scaling or Platt scaling if calibration is poor.

Business Impact Evaluation

Offline business metrics. Translate model performance into business terms: expected revenue impact, cost savings, error handling costs, and user experience metrics. A model with slightly lower accuracy but much lower latency might deliver more business value.

A/B testing. The definitive evaluation is production A/B testing against the current system (or no system). Define success metrics before the test, run for sufficient duration to reach statistical significance, and analyze both primary metrics and guardrail metrics.

Documenting Evaluation Results

Record all evaluation results in a model card that accompanies the model through its lifecycle. Include the evaluation methodology, dataset descriptions, metric values (overall and per slice), known limitations, and intended use boundaries. This documentation supports governance reviews, deployment decisions, and ongoing monitoring.