AI-Adapted Test Pyramid
The testing pyramid adapted for AI systems: unit tests for deterministic logic, integration tests with mocked models, evaluation tests with …
Quality
Amazon Lookout for Vision - Visual Anomaly Detection
A comprehensive reference for Amazon Lookout for Vision: automated visual inspection, defect detection, and deployment patterns for …
CMMI - Capability Maturity Model Integration
A process improvement framework that helps organizations improve performance across projects, divisions, and the enterprise.
Code Review Practices for ML Codebases
Practical guide to code review for ML projects, covering what to look for in training code, data pipelines, serving code, and experiment …
End-to-End Testing
What end-to-end testing is, how browser automation validates full-stack AI applications, and why E2E tests are essential but expensive.
Evaluator-Optimizer Pattern
Automated evaluation loops where one model generates output and another evaluates it, driving iterative improvement until quality thresholds …
Flaky Test
What flaky tests are, why they are especially common in AI systems, and strategies for managing non-deterministic test failures.
Integration Testing
What integration testing is, how it verifies component interactions, and where test boundaries belong in AI systems.
LLM Evaluation Methods - Measuring Language Model Quality
A comprehensive guide to evaluating large language models, covering automated metrics (BLEU, ROUGE, BERTScore), LLM-as-judge, human …
Mocking
Test doubles for AI systems: mocks, stubs, fakes, and spies explained, with guidance on when to use each for testing AI applications.
RAG Evaluation
Methods and metrics for measuring the quality of Retrieval Augmented Generation systems, covering retrieval accuracy, generation …
Reflection Pattern - Self-Critique and Iterative Refinement for LLMs
Using self-reflection loops where an LLM evaluates and improves its own output, catching errors and improving quality without human …
Snapshot Testing
What snapshot testing is, how it captures and compares output snapshots for regression detection, and its application in AI systems.
Test Fixture
What test fixtures are, how they provide predefined data and state for reproducible tests, and fixture patterns for AI systems.
Test-Driven Development
The TDD red-green-refactor cycle and how it applies to AI application development where outputs are non-deterministic.
Unit Testing
What unit testing is, how isolation and test doubles work, and assertion patterns relevant to AI application development.
Unit Testing AI Applications
How to unit test AI codebases effectively: testing prompt templates, output parsers, data validation, chunking functions, and embedding …
Property-Based Testing
What property-based testing is, why it is ideal for AI systems that cannot be tested with exact-output assertions, and the tools available …