Arnab Karmakar created IMPALA-15476:
---------------------------------------
Summary: Push down UUID partition predicates to Iceberg for
identity and bucket partitions
Key: IMPALA-15476
URL: https://issues.apache.org/jira/browse/IMPALA-15476
Project: IMPALA
Issue Type: Sub-task
Components: Frontend
Reporter: Arnab Karmakar
UUID value predicates are never pushed to Iceberg in scans. The Iceberg Java
library compares UUIDs with the signed {{{}UUID.compareTo(){}}}, while Parquet
writes UUID min/max bounds in unsigned byte order. Iceberg's
{{InclusiveMetricsEvaluator}} can therefore skip data files that contain
matching rows, for {{{}={}}}, {{IN}} and range predicates. As a result, scans
of UUID-partitioned tables get no partition pruning.
Two narrower pushdowns avoid the problem:
# {*}Identity partitions{*}. If a UUID column is identity partitioned in every
partition spec, each data file holds a single value, so its lower and upper
bounds are equal. {{=}} and {{IN}} can then be pushed: equality checks don't
depend on byte order, and Iceberg evaluates partition values exactly.
# {*}Bucket partitions{*}. For {{u = X}} or {{u IN (...)}} on a column with a
{{bucket(N, u)}} partition field, compute the bucket number in the FE with
Iceberg's transform. Push only {{bucket(N, u) = <hash>}} and keep the original
UUID predicate in the scan. Bucket transforms don't preserve order, so
Iceberg's metrics check never skips files for them, and bucket partition
summaries are INTs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
