Design a comprehensive audit trail system that tracks every AI decision, input, and output for accountability and debugging.
## CONTEXT When an AI system makes a decision that affects a customer, patient, or employee, regulators and legal teams will ask: what input was received, what model version processed it, what logic was applied, and why was this specific output generated? Organizations without comprehensive AI audit trails face crippling regulatory fines, inability to defend against bias lawsuits, and no way to diagnose production issues. A well-designed audit trail transforms AI operations from a black box into a fully traceable system where every decision can be reconstructed and explained. ## ROLE You are a platform architect with 14 years of experience designing observability and audit infrastructure for mission-critical systems, including 7 years focused on AI-specific traceability requirements. You built the AI audit platform at a major healthcare company that logs every clinical AI decision for 30 million patient interactions annually, meeting FDA and HIPAA requirements while maintaining sub-5ms logging overhead. Your audit systems have survived regulatory examinations across financial services, healthcare, and government, and your design philosophy prioritizes completeness and immutability while keeping storage costs manageable through intelligent tiering. ## RESPONSE GUIDELINES - Define the audit log schema with specific field names, data types, and indexing strategies - Design for immutability — audit records must be tamper-proof once written - Balance completeness with performance — logging must not add meaningful latency to the AI response path - Include specific storage cost estimates based on the expected volume and retention requirements - Do NOT log only successes — failures, errors, and filtered outputs are equally important for audit purposes - Do NOT design audit storage without retention policies — unbounded log growth creates cost and liability problems ## TASK CRITERIA 1. **Event Catalog Definition** — Enumerate every auditable event in [INSERT AI PLATFORM]: user request received, authentication and authorization check, prompt assembly (system prompt + user input + context), model API call (with model version), tool/function invocations, retrieval augmentation queries, output filtering decisions, final response delivery, and user feedback received. Define the criticality level of each event. 2. **Audit Log Schema Design** — Create the complete log schema with fields: event_id (UUID), timestamp (UTC microseconds), request_id (correlation ID), session_id, user_id, event_type, model_id and version, input_content_hash (SHA-256 for privacy), output_content_hash, full_input (encrypted), full_output (encrypted), latency_ms, token_count, filter_decisions, tool_calls, confidence_scores, and custom metadata. 3. **Immutability & Integrity** — Design tamper-proof logging: append-only storage, cryptographic hash chaining between sequential records, digital signatures on log entries, and integrity verification procedures that can detect any modification or deletion of audit records. 4. **Storage Tiering Strategy** — Define storage tiers based on [INSERT COMPLIANCE REQUIREMENTS]: hot tier (0-30 days, fully queryable, estimated cost per GB), warm tier (30 days to 1 year, searchable with higher latency, cost per GB), and cold tier (1-7+ years, archival with retrieval SLA, cost per GB). Calculate expected storage volume based on [INSERT DAILY VOLUME]. 5. **Privacy & Data Protection** — Specify how audit logs handle sensitive data: PII encryption at rest and in transit, content hashing for audit without storing raw PII where possible, data subject access request procedures, right-to-erasure handling that maintains audit integrity, and jurisdiction-specific storage requirements. 6. **Access Control Model** — Define role-based access: auditors (read-only, full access), compliance team (read-only, filtered views), engineering (read-only for debugging, time-limited), management (aggregate reports only), and external regulators (supervised access with export controls). Specify authentication requirements for each role. 7. **Query Optimization** — List the top 10 queries auditors and engineers will run and design indexes to support them: find all decisions for a specific user, trace a single request end-to-end, find all instances where a specific filter triggered, compare model version outputs, search for anomalous latency patterns, and aggregate decision distributions over time periods. 8. **Real-Time Audit Streaming** — Design a real-time audit event stream for monitoring dashboards and alerting: publish events to a message queue, define consumers for real-time compliance monitoring, anomaly detection on audit patterns, and integration with SIEM systems for security monitoring. 9. **Retention & Lifecycle Management** — Define retention policies per event type aligned with [INSERT COMPLIANCE REQUIREMENTS]: minimum retention periods, automatic tier migration schedules, expiration and deletion procedures, legal hold capabilities for litigation preservation, and annual retention policy review process. 10. **Audit Trail Testing & Validation** — Design the testing approach: verify completeness (no events missed), verify accuracy (logged data matches actual events), verify immutability (tamper detection works), load testing at 2x expected volume, and recovery testing (can reconstruct any decision from audit records). ## INFORMATION ABOUT ME - My AI platform: [INSERT AI PLATFORM — e.g., custom LLM application, Azure OpenAI Service, AWS Bedrock, self-hosted model] - My compliance requirements: [INSERT COMPLIANCE REQUIREMENTS — e.g., HIPAA, SOX, GDPR, EU AI Act, FDA 21 CFR Part 11] - My expected daily volume: [INSERT DAILY VOLUME — e.g., 100K AI decisions/day, 1M events/day, 10M log entries/day] - My current logging infrastructure: [INSERT INFRASTRUCTURE — e.g., ELK stack, Datadog, Splunk, CloudWatch, custom] - My retention requirements: [INSERT RETENTION — e.g., 7 years for financial, 10 years for healthcare, 3 years minimum] ## RESPONSE FORMAT - Begin with an audit architecture overview showing event flow from capture to storage to retrieval in 5-7 bullet points - Use labeled sections for each component with schema definitions and configuration specifications - Include the complete audit log schema as a table with field names, data types, and indexing flags - Provide a storage cost projection table based on the expected daily volume across all tiers - End with an implementation roadmap divided into Phase 1 (core logging and storage), Phase 2 (immutability and access controls), and Phase 3 (analytics and compliance reporting)
Or press ⌘C to copy
Replace these placeholders with your own content before using the prompt.
[INSERT AI PLATFORM][INSERT COMPLIANCE REQUIREMENTS][INSERT DAILY VOLUME]