Architecture decisions in AI systems are harder to reverse than in traditional software. Choosing a batch inference pipeline over real-time serving, selecting a feature store, or deciding between fine-tuning and RAG all have long-lasting consequences. Architecture Decision Records (ADRs) provide a lightweight method to document these decisions so future teams understand not just what was decided, but why.

What Is an ADR

An ADR is a short document that captures a single architecture decision. The format is deliberately simple to encourage adoption. Michael Nygard’s original template contains four sections:

Title - A short noun phrase. Example: “Use Redis for feature serving layer.”

Status - Proposed, Accepted, Deprecated, or Superseded. Include the date of each status change.

Context - The forces at play: business requirements, technical constraints, team skills, timeline pressure. For AI systems, this often includes data characteristics, latency requirements, and model complexity.

Decision - The choice made, stated directly. “We will use Redis as the feature serving layer for real-time inference, with batch features materialized from the feature store on a 15-minute schedule.”

Consequences - What becomes easier and harder as a result. Be honest about trade-offs. “Redis adds operational overhead (cluster management, memory sizing) but provides sub-millisecond feature retrieval that our 50ms latency SLA requires.”

ADRs for AI-Specific Decisions

AI systems introduce decision categories that traditional software does not have:

Model architecture decisions - Why a transformer was chosen over an LSTM, or why an ensemble was preferred over a single model. Document the experiments that informed the decision, the accuracy/latency/cost trade-offs, and the datasets used for evaluation.

Training infrastructure decisions - GPU type, distributed training strategy, spot vs on-demand instances. These decisions have direct cost implications. An ADR that explains why the team chose 4xA100 over 8xA10G, with benchmark data, prevents future teams from re-running the same experiments.

Data pipeline decisions - Batch vs streaming ingestion, data lake format, feature engineering approach. These are foundational decisions that affect every downstream component. Document the data volumes, freshness requirements, and cost constraints that drove the choice.

Serving architecture decisions - Real-time vs batch inference, model format (ONNX, TensorRT, native), autoscaling strategy. Include latency benchmarks, throughput tests, and cost projections.

Architecture Evaluation with ATAM

The Architecture Tradeoff Analysis Method (ATAM) provides a structured way to evaluate architecture decisions before committing to them. For AI systems, the process adapts as follows:

Step 1: Present the business drivers. What are the quality attribute requirements? For AI systems, these typically include inference latency, throughput, model freshness (how quickly retrained models reach production), and cost per prediction.

Step 2: Present the architecture. Walk through the system using architecture views: deployment diagrams, data flow diagrams, and component diagrams that show the training pipeline, feature store, model registry, and serving infrastructure.

Step 3: Identify architectural approaches. For each quality attribute, identify the architectural tactics used. For latency: caching, model distillation, quantization. For availability: redundant serving endpoints, fallback models, graceful degradation.

Step 4: Generate a quality attribute utility tree. Prioritize scenarios by importance and difficulty. “Model serving latency under 100ms at p99 during peak traffic” might be high importance, high difficulty. “Retrain and deploy a model within 4 hours of data drift detection” might be high importance, medium difficulty.

Step 5: Analyze architectural approaches. For each high-priority scenario, trace through the architecture and identify sensitivity points (where a small change has large impact), tradeoff points (where achieving one quality attribute hurts another), and risks (where the architecture may not meet the scenario).

Maintaining ADRs

ADRs are immutable records. When a decision is reversed, write a new ADR that supersedes the old one rather than editing the original. This preserves the decision history.

Store ADRs in the repository alongside the code they govern. A docs/adr/ directory with sequentially numbered files (e.g., 0001-use-redis-feature-serving.md) works well. Teams can use tools like adr-tools or log4brains to manage the ADR lifecycle.

Review ADRs during architecture reviews and post-incident retrospectives. A production incident often reveals that an assumption documented in an ADR no longer holds, triggering a new decision. The discipline of connecting incidents back to architecture decisions builds institutional knowledge that survives team turnover.