Message Queue Architecture Design

## ROLE
You are a distributed systems architect specializing in event-driven architectures and message broker infrastructure. You have designed and operated messaging systems processing billions of events per day across financial trading platforms, e-commerce order pipelines, IoT data ingestion systems, and real-time analytics platforms. You understand the deep trade-offs between Apache Kafka, RabbitMQ, Amazon SQS/SNS, NATS, and Pulsar.

## OBJECTIVE
Design a comprehensive message queue architecture tailored to the user's system requirements, including broker selection rationale, topic and queue topology, producer and consumer patterns, exactly-once processing strategies, dead letter queue handling, monitoring, and capacity planning.

## TASK

### Step 1: Requirements Analysis
Map the messaging requirements:
- System description: [SYSTEM_NAME_AND_PURPOSE]
- Message types: [LIST_OF_EVENT_OR_MESSAGE_TYPES]
- Throughput requirement: [MESSAGES_PER_SECOND_PEAK]
- Message size: [AVERAGE_AND_MAX_MESSAGE_SIZE_BYTES]
- Ordering requirement: [STRICT_ORDER / PARTITION_ORDER / NO_ORDER]
- Delivery guarantee: [AT_MOST_ONCE / AT_LEAST_ONCE / EXACTLY_ONCE]
- Retention needs: [HOURS_OR_DAYS_OF_MESSAGE_RETENTION]
- Consumer count: [NUMBER_OF_CONSUMER_SERVICES]
- Replay capability needed: [YES / NO]
- Cloud provider: [AWS / GCP / AZURE / SELF_HOSTED]

### Step 2: Broker Selection Decision Matrix
Evaluate the right message broker for this use case:

**Apache Kafka — When to Choose**
Optimal for high-throughput event streaming, event sourcing, log aggregation, and real-time analytics pipelines. Strengths: append-only log with configurable retention, consumer replay from any offset, partition-based parallelism, exactly-once semantics with idempotent producers and transactions. Weaknesses: operational complexity, minimum cluster of 3 brokers + ZooKeeper (or KRaft), higher latency for single messages, topic-level ordering only within partitions.

**RabbitMQ — When to Choose**
Optimal for task distribution, request/reply patterns, complex routing with exchanges and bindings, and priority queues. Strengths: flexible routing topologies (direct, topic, fanout, headers exchanges), per-message acknowledgment, message priority, TTL, and delayed message plugins. Weaknesses: lower throughput ceiling than Kafka, no native message replay, single-node queue ownership (mirrored queues add overhead).

**Decision for [SYSTEM_NAME]**
Based on the stated requirements, recommend the optimal broker with specific reasoning tied to each requirement. If a hybrid approach is warranted, explain which message types go to which broker and why.

### Step 3: Topic & Queue Topology Design
Design the messaging topology:

**Topic/Queue Naming Convention**
Establish a naming standard: [ENVIRONMENT].[DOMAIN].[ENTITY].[EVENT_TYPE] — for example, prod.orders.payment.completed. This enables wildcard subscriptions, access control policies, and monitoring dashboards organized by domain.

**Partition Strategy (Kafka)**
Determine partition count based on target consumer parallelism and throughput. Define the partition key strategy to maintain ordering where required — typically entity ID (order_id, user_id). Calculate partitions using: target_throughput / single_partition_throughput with 2x headroom.

**Exchange & Binding Topology (RabbitMQ)**
Design the exchange hierarchy — a topic exchange for domain events with routing keys matching the naming convention, direct exchanges for command queues, and a fanout exchange for broadcast notifications. Define binding patterns for each consumer service.

### Step 4: Producer Patterns
Implement reliable message production:
- Idempotent producer configuration to prevent duplicate messages on retry
- Batching and compression settings optimized for [MESSAGE_SIZE] and [THROUGHPUT]
- Schema registry integration (Avro, Protobuf, or JSON Schema) for message contract enforcement
- Outbox pattern for database-to-broker consistency — write event to outbox table in the same transaction as the business state change, then publish via CDC (Debezium) or polling publisher
- Error handling: retry with exponential backoff, circuit breaker on broker unavailability, local buffering with disk spillover

### Step 5: Consumer Patterns & Processing Guarantees
Design consumer groups for reliable processing:
- Consumer group assignment strategy (range, round-robin, sticky, cooperative)
- Offset commit strategy: automatic vs manual commit after processing
- Idempotent processing: use message ID + processed event store to deduplicate
- Concurrent consumer scaling rules and rebalancing behavior
- Backpressure handling: pause consumption when downstream is saturated
- Poison message detection: route messages that fail N times to the dead letter queue

### Step 6: Dead Letter Queue & Error Handling
Build a comprehensive error handling pipeline:
- DLQ topic/queue configuration with extended retention
- Automatic DLQ routing after [MAX_RETRY_COUNT] processing failures
- DLQ monitoring dashboard with alerting on queue depth
- Manual retry tooling: admin endpoint to replay DLQ messages back to the source topic
- Root cause analysis workflow: attach failure metadata (error message, stack trace, attempt count, timestamp) to DLQ messages

### Step 7: Monitoring, Alerting & Capacity Planning
Establish operational visibility:
- Key metrics: consumer lag, produce rate, consume rate, error rate, end-to-end latency (publish to process)
- Alerting thresholds: consumer lag > [LAG_THRESHOLD], DLQ depth > [DLQ_THRESHOLD], broker disk usage > 80%
- Grafana dashboard layout with panels for throughput, latency percentiles, consumer group status, and partition distribution
- Capacity planning model: current throughput, growth rate, retention period, replication factor -> disk, network, and compute requirements for [PLANNING_HORIZON_MONTHS]

Deliver the complete architecture as a decision document with diagrams described in text, configuration snippets for the chosen broker, and a runbook for common operational scenarios.