Regulators, auditors, and internal governance teams need to answer a specific question: why did the AI system make this decision? An audit trail provides the answer by capturing an immutable record of every input, output, model version, configuration, and intermediate step involved in each AI-driven decision.

What to Capture

Request context - The full input to the model, including system prompt, user message, retrieved context (for RAG systems), and any tool outputs consumed. Record the timestamp, request ID, user identity, and originating application.

Model metadata - The model identifier, version, provider, and configuration parameters (temperature, top-p, max tokens). For fine-tuned models, include the training run ID and dataset version.

Response data - The complete model output, including any structured outputs, confidence scores, and token usage. If post-processing modifies the response (guardrail filtering, PII redaction), log both the raw and processed versions.

Decision chain - For agentic systems that make multiple model calls, log the full chain of reasoning steps, tool invocations, and intermediate results. Each step links to the previous one via a trace ID.

Immutability Requirements

Audit logs must be tamper-evident. Write logs to append-only storage such as AWS CloudTrail-backed S3 with object lock, or a dedicated audit database with write-once semantics. Hash each log entry and chain hashes so that any modification to historical records is detectable.

Separate audit storage from operational storage. The team operating the AI system should not have permission to modify or delete audit records. Use separate IAM roles and accounts for audit log administration.

Retention and Access

Define retention periods based on regulatory requirements. Financial services regulations may require seven years of retention. Healthcare regulations have their own timelines. Build automated lifecycle policies that move logs from hot storage to cold archive based on age.

Provide a query interface for auditors that supports filtering by time range, user, model version, decision outcome, and application. Auditors should not need engineering support to retrieve records.

Implementation Approach

Instrument the LLM gateway or proxy layer to emit audit events on every request-response cycle. Use structured logging with a consistent schema across all AI services. Route audit events through a dedicated pipeline to append-only storage, separate from operational logs.

For high-throughput systems, buffer audit events in a durable queue (Kafka, SQS) and write them to storage asynchronously. Ensure the queue itself provides delivery guarantees so no events are lost.

Compliance Alignment

Map audit trail fields to specific regulatory requirements. NIST AI RMF calls for documentation of AI system behavior. The EU AI Act requires logging for high-risk AI systems. SOC 2 audits examine whether AI decisions are traceable. A well-designed audit trail satisfies multiple frameworks simultaneously.