✔ PASS Warehouse & Statistical task_100_Q100_market_basket_association_rules Scaled: 1M+ Records Ordered: Yes

Market Basket Association Rules: Support, Confidence, and Lift in Pure SQL

Candidate Latency (1M Rows)
113.75 ms
(1.0x slower than golden)
Golden Latency (1M Rows)
112.36 ms
DuckDB in-memory engine
Underlying Data Volume
1,000,000 rows
22.9 MiB across 1 table
Evaluated Model
gemini-3.8-flash-low
Gen Latency: 12.1s
✔ Correctness Verified on 1M+ Records
Candidate SQL output matches ground truth golden query across all multiset rows and numeric tolerances (tol=1e-3). Zero data skew or anti-patterns detected on full dataset scale.

1. Analytical Significance & What It Means

Scenario & Frontier Trap
Business Scenario & Problem

Data miners discover product affinities ('Customers who buy Item A also buy Item B') using Support, Confidence, and Lift metrics computed directly from transaction tables.

⚡ The Frontier Trap (Why LLMs Fail)

Generating cross-products across all transactions creates memory spills on large catalogs.

Prompt / User Request Given to Model
Data miners discover product affinities ('Customers who buy Item A also buy Item B') using Support, Confidence, and Lift metrics computed directly from transaction tables.

Execute the required analytical SQL calculation in DuckDB.
Expected output columns: item_a, item_b, pair_baskets, support, confidence, lift.
Results should be ordered as specified.

2. Underlying Tables, Row Counts & Data Volume

1 Tables (1,000,000 rows • 22.9 MiB)

Production-scale fixtures loaded into the isolated DuckDB sandbox with at least 1,000,000 records per table:

📊 order_baskets 1,000,000 rows 22.9 MiB Data Volume
order_id (INTEGER) item (VARCHAR)
Data Preview (showing sample rows from 1,000,000 total rows):
# order_id item
1 1 Bread
2 1 Butter
3 1 Milk
4 2 Bread
5 2 Butter
6 3 Bread
7 3 Milk
8 4 Beer
9 4 Diapers
10 4 Bread
11 5 Butter
12 6 Milk
13 7 Beer
14 8 Diapers
15 9 Bread

3. Graphical Query Execution Plan & Cost Bottlenecks

Physical Operator Tree (DuckDB 2.0 Profiler)

Visual flowchart of the physical execution pipeline. Operators with high relative costs (≥ 20% of query runtime or heavy latency) are visually highlighted:

Pipeline Latency Distribution (Total Plan Time: 329.26 ms) Nodes consuming ≥ 20% of query runtime are highlighted with glowing borders
🔄 CTE 0.0%
Latency: 0.01 ms
Rows In/Out: 0
Data Vol: 0 B
📐 PROJECTION 0.0%
Latency: 0.01 ms
Rows In/Out: 5
Data Vol: 120 B
📁 Projections: ['__internal_decomp...
📊 HASH_GROUP_BY 2.9%
Latency: 9.42 ms
Rows In/Out: 5
Data Vol: 80 B
📐 PROJECTION 0.0%
Latency: 0.06 ms
Rows In/Out: 1,000,000
Data Vol: 7.63 MB
📁 Projections: #0
📐 PROJECTION 1.7%
Latency: 5.46 ms
Rows In/Out: 1,000,000
Data Vol: 7.63 MB
📁 Projections: __internal_compress...
📐 PROJECTION 0.3%
Latency: 1.14 ms
Rows In/Out: 1,000,000
Data Vol: 15.26 MB
📁 Projections: __internal_decompre...
📊 HASH_GROUP_BY 🔥 29.5% Cost
Latency: 97.21 ms
Rows In/Out: 1,000,000
Data Vol: 11.44 MB
⚠️ Heavy Execution Bottleneck (29.5% of query runtime)
📐 PROJECTION 0.0%
Latency: 0.07 ms
Rows In/Out: 1,000,000
Data Vol: 11.44 MB
📁 Projections: ['item', 'order_id'...
📐 PROJECTION 1.0%
Latency: 3.22 ms
Rows In/Out: 1,000,000
Data Vol: 11.44 MB
📁 Projections: ['__internal_compre...
🔍 TABLE_SCAN 3.6%
Latency: 12.00 ms
Rows In/Out: 1,000,000
Data Vol: 19.07 MB
📁 memory.main.order_baskets
📐 PROJECTION 0.0%
Latency: 0.00 ms
Rows In/Out: 4
Data Vol: 256 B
📁 Projections: ['__internal_decomp...
⏳ ORDER_BY 0.0%
Latency: 0.08 ms
Rows In/Out: 4
Data Vol: 192 B
📐 PROJECTION 0.0%
Latency: 0.01 ms
Rows In/Out: 4
Data Vol: 192 B
📁 Projections: ['__internal_compre...
📐 PROJECTION 0.0%
Latency: 0.02 ms
Rows In/Out: 4
Data Vol: 256 B
📁 Projections: ['item_a', 'item_b'...
📐 PROJECTION 0.0%
Latency: 0.00 ms
Rows In/Out: 4
Data Vol: 352 B
📁 Projections: ['item_a', 'item_b'...
📐 PROJECTION 0.0%
Latency: 0.01 ms
Rows In/Out: 4
Data Vol: 320 B
📁 Projections: ['item_a', 'item_b'...
⚡ HASH_JOIN 0.1%
Latency: 0.26 ms
Rows In/Out: 4
Data Vol: 256 B
⚡ HASH_JOIN 0.1%
Latency: 0.18 ms
Rows In/Out: 4
Data Vol: 224 B
🔍 CTE_SCAN 0.0%
Latency: 0.00 ms
Rows In/Out: 5
Data Vol: 120 B
⚙️ CROSS_PRODUCT 0.0%
Latency: 0.03 ms
Rows In/Out: 4
Data Vol: 192 B
🎯 FILTER 0.0%
Latency: 0.04 ms
Rows In/Out: 4
Data Vol: 160 B
📐 PROJECTION 0.0%
Latency: 0.01 ms
Rows In/Out: 12
Data Vol: 480 B
📁 Projections: ['__internal_decomp...
📊 HASH_GROUP_BY 0.2%
Latency: 0.52 ms
Rows In/Out: 12
Data Vol: 288 B
📐 PROJECTION 0.0%
Latency: 0.00 ms
Rows In/Out: 16
Data Vol: 256 B
📁 Projections: ['#0', '#1']
📐 PROJECTION 0.0%
Latency: 0.00 ms
Rows In/Out: 16
Data Vol: 256 B
📁 Projections: ['__internal_compre...
📐 PROJECTION 0.0%
Latency: 0.01 ms
Rows In/Out: 16
Data Vol: 512 B
📁 Projections: ['__internal_decomp...
📊 HASH_GROUP_BY 0.2%
Latency: 0.56 ms
Rows In/Out: 16
Data Vol: 320 B
📐 PROJECTION 0.0%
Latency: 0.00 ms
Rows In/Out: 16
Data Vol: 320 B
📁 Projections: ['item', 'item', 'o...
📐 PROJECTION 0.0%
Latency: 0.01 ms
Rows In/Out: 16
Data Vol: 320 B
📁 Projections: ['#0', '__internal_...
⚡ HASH_JOIN 🔥 28.8% Cost
Latency: 94.80 ms
Rows In/Out: 16
Data Vol: 576 B
⚠️ Heavy Execution Bottleneck (28.8% of query runtime)
🔍 TABLE_SCAN 4.5%
Latency: 14.78 ms
Rows In/Out: 1,000,000
Data Vol: 19.07 MB
📁 memory.main.order_baskets
🔍 TABLE_SCAN 3.6%
Latency: 11.77 ms
Rows In/Out: 1,000,000
Data Vol: 19.07 MB
📁 memory.main.order_baskets
📊 UNGROUPED_AGGREGATE 0.0%
Latency: 0.06 ms
Rows In/Out: 1
Data Vol: 8 B
📊 HASH_GROUP_BY 🔥 23.3% Cost
Latency: 76.57 ms
Rows In/Out: 999,994
Data Vol: 3.81 MB
⚠️ Heavy Execution Bottleneck (23.3% of query runtime)
📐 PROJECTION 0.0%
Latency: 0.11 ms
Rows In/Out: 1,000,000
Data Vol: 3.81 MB
📁 Projections: order_id
🔍 TABLE_SCAN 0.2%
Latency: 0.81 ms
Rows In/Out: 1,000,000
Data Vol: 3.81 MB
📁 memory.main.order_baskets
🔍 CTE_SCAN 0.0%
Latency: 0.01 ms
Rows In/Out: 5
Data Vol: 120 B
Operator Cost % Latency Processed Rows Data Volume Details / Table Status
HASH_GROUP_BY 29.5% 97.21 ms 1,000,000 11.44 MB - 🔥 BOTTLENECK
HASH_JOIN 28.8% 94.80 ms 16 0.00 MB - 🔥 BOTTLENECK
HASH_GROUP_BY 23.3% 76.57 ms 999,994 3.81 MB - 🔥 BOTTLENECK
TABLE_SCAN 4.5% 14.78 ms 1,000,000 19.07 MB memory.main.order_baskets Normal
TABLE_SCAN 3.6% 12.00 ms 1,000,000 19.07 MB memory.main.order_baskets Normal
TABLE_SCAN 3.6% 11.77 ms 1,000,000 19.07 MB memory.main.order_baskets Normal
HASH_GROUP_BY 2.9% 9.42 ms 5 0.00 MB - Normal
PROJECTION 1.7% 5.46 ms 1,000,000 7.63 MB Projections: __internal_compress_string_ubigint( Normal
PROJECTION 1.0% 3.22 ms 1,000,000 11.44 MB Projections: ['__internal_compress_string_ubigin Normal
PROJECTION 0.3% 1.14 ms 1,000,000 15.26 MB Projections: __internal_decompress_string(#0) Normal
TABLE_SCAN 0.2% 0.81 ms 1,000,000 3.81 MB memory.main.order_baskets Normal
HASH_GROUP_BY 0.2% 0.56 ms 16 0.00 MB - Normal
HASH_GROUP_BY 0.2% 0.52 ms 12 0.00 MB - Normal
HASH_JOIN 0.1% 0.26 ms 4 0.00 MB - Normal
HASH_JOIN 0.1% 0.18 ms 4 0.00 MB - Normal
PROJECTION 0.0% 0.11 ms 1,000,000 3.81 MB Projections: order_id Normal
ORDER_BY 0.0% 0.08 ms 4 0.00 MB - Normal
PROJECTION 0.0% 0.07 ms 1,000,000 11.44 MB Projections: ['item', 'order_id'] Normal
UNGROUPED_AGGREGATE 0.0% 0.06 ms 1 0.00 MB - Normal
PROJECTION 0.0% 0.06 ms 1,000,000 7.63 MB Projections: #0 Normal
FILTER 0.0% 0.04 ms 4 0.00 MB - Normal
CROSS_PRODUCT 0.0% 0.03 ms 4 0.00 MB - Normal
PROJECTION 0.0% 0.02 ms 4 0.00 MB Projections: ['item_a', 'item_b', 'pair_baskets' Normal
CTE_SCAN 0.0% 0.01 ms 5 0.00 MB - Normal
CTE 0.0% 0.01 ms 0 0.00 MB - Normal
PROJECTION 0.0% 0.01 ms 16 0.00 MB Projections: ['__internal_decompress_string(#0)' Normal
PROJECTION 0.0% 0.01 ms 5 0.00 MB Projections: ['__internal_decompress_string(#0)' Normal
PROJECTION 0.0% 0.01 ms 16 0.00 MB Projections: ['#0', '__internal_compress_string_ Normal
PROJECTION 0.0% 0.01 ms 12 0.00 MB Projections: ['__internal_decompress_string(#0)' Normal
PROJECTION 0.0% 0.01 ms 4 0.00 MB Projections: ['__internal_compress_string_ubigin Normal
PROJECTION 0.0% 0.01 ms 4 0.00 MB Projections: ['item_a', 'item_b', 'pair_baskets' Normal
CTE_SCAN 0.0% 0.00 ms 5 0.00 MB - Normal
PROJECTION 0.0% 0.00 ms 16 0.00 MB Projections: ['__internal_compress_string_ubigin Normal
PROJECTION 0.0% 0.00 ms 4 0.00 MB Projections: ['item_a', 'item_b', 'pair_baskets' Normal
PROJECTION 0.0% 0.00 ms 4 0.00 MB Projections: ['__internal_decompress_string(#0)' Normal
PROJECTION 0.0% 0.00 ms 16 0.00 MB Projections: ['item', 'item', 'order_id'] Normal
PROJECTION 0.0% 0.00 ms 16 0.00 MB Projections: ['#0', '#1'] Normal

4. Queries Executed (Candidate vs Golden Ground Truth)

Beautified & Syntax Highlighted
Candidate SQL Executed by gemini-3.8-flash-low (Runtime on 1M Rows: 113.75 ms)
WITH basket_totals AS
  (SELECT count(DISTINCT order_id) AS total_baskets
   FROM order_baskets),
     item_counts AS
  (SELECT item,
          count(DISTINCT order_id) AS item_baskets
   FROM order_baskets
   GROUP BY item),
     pair_counts AS
  (SELECT a.item AS item_a,
          b.item AS item_b,
          count(DISTINCT a.order_id) AS pair_baskets
   FROM order_baskets a
   JOIN order_baskets b ON a.order_id = b.order_id
   AND a.item != b.item
   GROUP BY a.item,
            b.item)
SELECT p.item_a,
       p.item_b,
       p.pair_baskets,
       round(p.pair_baskets * 1.0 / t.total_baskets, 3) AS support,
       round(p.pair_baskets * 1.0 / ca.item_baskets, 3) AS confidence,
       round((p.pair_baskets * 1.0 * t.total_baskets) / (ca.item_baskets * cb.item_baskets), 3) AS lift
FROM pair_counts p
JOIN item_counts ca ON p.item_a = ca.item
JOIN item_counts cb ON p.item_b = cb.item
CROSS JOIN basket_totals t
WHERE p.pair_baskets >= 2
ORDER BY lift DESC,
         p.item_a,
         p.item_b;
Golden Reference Ground Truth SQL (Runtime on 1M Rows: 112.36 ms)
-- Compute Support, Confidence, and Lift for Item A -> Item B
 WITH basket_totals AS
  (SELECT count(DISTINCT order_id) AS total_baskets
   FROM order_baskets),
      item_counts AS
  (SELECT item,
          count(DISTINCT order_id) AS item_baskets
   FROM order_baskets
   GROUP BY item),
      pair_counts AS
  (SELECT a.item AS item_a,
          b.item AS item_b,
          count(DISTINCT a.order_id) AS pair_baskets
   FROM order_baskets a
   JOIN order_baskets b ON a.order_id = b.order_id
   AND a.item != b.item
   GROUP BY a.item,
            b.item)
SELECT p.item_a,
       p.item_b,
       p.pair_baskets, -- Support: P(A and B)
 round(p.pair_baskets * 1.0 / t.total_baskets, 3) AS support, -- Confidence: P(B | A) = pair / count_a
 round(p.pair_baskets * 1.0 / ca.item_baskets, 3) AS confidence, -- Lift: Confidence / P(B) = (pair * total) / (count_a * count_b)
 round((p.pair_baskets * 1.0 * t.total_baskets) / (ca.item_baskets * cb.item_baskets), 3) AS lift
FROM pair_counts p
JOIN item_counts ca ON p.item_a = ca.item
JOIN item_counts cb ON p.item_b = cb.item
CROSS JOIN basket_totals t
WHERE p.pair_baskets >= 2
ORDER BY lift DESC,
         p.item_a,
         p.item_b;
DuckDB Fixture Setup DDL & Data Population
CREATE TABLE order_baskets (order_id INT, item VARCHAR);


INSERT INTO order_baskets
VALUES (1, 'Bread'),
       (1, 'Butter'),
       (1, 'Milk'),
       (2, 'Bread'),
       (2, 'Butter'),
       (3, 'Bread'),
       (3, 'Milk'),
       (4, 'Beer'),
       (4, 'Diapers');

5. Query Results & Expected Ground Truth

Tabular Comparison
Actual Candidate Output (4 rows × 6 cols)
# item_a item_b pair_baskets support confidence lift
1 Bread Butter 2 0 0 0
2 Bread Milk 2 0 0 0
3 Butter Bread 2 0 0 0
4 Milk Bread 2 0 0 0
Expected Golden Output (4 rows × 6 cols)
# item_a item_b pair_baskets support confidence lift
1 Bread Butter 2 0 0 0
2 Bread Milk 2 0 0 0
3 Butter Bread 2 0 0 0
4 Milk Bread 2 0 0 0
← Prev: Window Frame Ties Exclusi... ⊞ Back to Benchmark Dashboard Next: Kaplan-Meier Non-Parametr... →