Market Basket Analysis

You are a retail analytics specialist conducting market basket analysis. Based on the following context, build a complete association mining and recommendation system.

Business Type: [GROCERY/E-COMMERCE/RETAIL/RESTAURANT/B2B DISTRIBUTION]
Transaction Data: [DESCRIBE YOUR DATA - order IDs, product IDs, quantities, timestamps]
Number of Transactions: [APPROXIMATE TRANSACTION COUNT]
Product Catalog Size: [NUMBER OF UNIQUE PRODUCTS OR SKUS]
Analysis Goal: [CROSS-SELLING/BUNDLING/STORE LAYOUT/RECOMMENDATION ENGINE]
Current Approach: [HOW CROSS-SELLING IS HANDLED TODAY - MANUAL/RULES/NONE]

Build the market basket analysis across these six sections:

1. DATA PREPARATION AND TRANSACTION FORMATTING
Prepare the transaction data for association rule mining. Transform the raw data into the required format: a binary matrix where rows are transactions and columns are items, or a list of itemsets. Handle data quality issues specific to basket analysis including returns that should remove items from the original basket, quantity considerations where buying five of an item should still count as one item present, and extremely frequent items like shopping bags that add noise without insight. Filter transactions by relevant criteria such as minimum basket size and time period. Apply product hierarchy grouping to reduce dimensionality when the SKU count is too high for meaningful patterns, aggregating from individual products to subcategories where appropriate.

2. ASSOCIATION RULE MINING WITH APRIORI
Implement the Apriori algorithm to discover frequent itemsets and generate association rules. Explain and compute the three key metrics: support which measures how often items appear together, confidence which measures the conditional probability of the consequent given the antecedent, and lift which measures the strength of association beyond what is expected by chance. Set appropriate thresholds for minimum support, minimum confidence, and minimum lift based on the dataset characteristics. Provide Python code using mlxtend for the complete Apriori pipeline. Generate rules of varying sizes from two-item to multi-item associations. Address the computational considerations for large catalogs and show how to manage runtime.

3. ADVANCED ASSOCIATION TECHNIQUES
Go beyond basic Apriori with more sophisticated approaches. Implement the FP-Growth algorithm which is significantly faster for large datasets and compare its results to Apriori. Apply temporal association mining to discover rules that hold during specific time periods, days of week, or seasons. Build sequential pattern mining to find purchase sequences where customers who buy A tend to buy B within a certain time window. Implement category-level association analysis that reveals department-to-department shopping patterns. For e-commerce, add clickstream analysis to discover browse-but-not-buy associations alongside purchase associations.

4. RESULTS FILTERING AND BUSINESS INTERPRETATION
Filter the raw association rules to identify the most actionable and interesting ones. Remove trivial rules where the association is obvious or already well-known. Apply interestingness measures beyond lift including conviction, leverage, and Zhang metric to identify genuinely surprising associations. Organize rules by business application: rules useful for cross-selling at checkout, rules suggesting product bundling opportunities, rules informing store layout adjacencies, and rules driving personalized email recommendations. Create visualizations including network graphs of product associations, heatmaps showing category-level affinity, and ranked tables of the top rules by lift and actionability.

5. RECOMMENDATION ENGINE IMPLEMENTATION
Translate association rules into a working recommendation system. Build a rule-based recommendation engine that given a customer's current basket suggests additional items ranked by lift and confidence. Implement a collaborative filtering layer that personalizes recommendations based on the customer's purchase history and the baskets of similar customers. Create a hybrid approach that blends association rules with popularity and margin considerations to optimize for revenue impact rather than pure association strength. Provide Python code for a recommendation API that accepts a basket and returns ranked suggestions with explanations such as customers who bought X and Y also bought Z.

6. BUSINESS APPLICATION AND IMPACT MEASUREMENT
Define specific business actions based on the analysis results. Create product bundling proposals with estimated uptake rates and margin impact. Design store layout recommendations showing which departments or products should be placed near each other. Build targeted cross-selling campaigns with specific product pairs and recommended offer structures. Define an A/B testing framework to measure the revenue lift from implementing association-based recommendations versus the current approach. Create a monitoring dashboard that tracks recommendation acceptance rates, incremental revenue from cross-selling, and average basket size changes over time.