Partitioning vs Bucketing

Partitioning and bucketing organise data in different ways. Partitioning groups rows by a column value and commonly stores those groups in separate directories. Bucketing assigns rows to a fixed set of buckets using a hash of the bucket column.

Consider a sales table partitioned by order_date. Records for each date are stored under that date’s partition. A query that filters on a specific date can skip other partitions, reducing the amount of data it needs to scan. Choose partition columns carefully: too many tiny partitions can increase overhead.

With bucketing, a key such as customer_id determines the bucket assignment. The number of buckets is chosen in advance. Compatible bucket layouts can help repeated joins by reducing the redistribution of rows. A bucket is a logical grouping; the exact number and layout of physical files depends on how the data is written.

Use query patterns to guide the choice. Date-based filters often benefit from partition pruning, while repeated joins on a stable key may benefit from bucketing. These techniques can be combined where the table format and execution engine support them.