Zoltán Borók-Nagy created IMPALA-15709:
------------------------------------------
Summary: Push down Iceberg predicates on sub-microsecond TIMESTAMP
literals by rounding per operator
Key: IMPALA-15709
URL: https://issues.apache.org/jira/browse/IMPALA-15709
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Reporter: Zoltán Borók-Nagy
IMPALA-15636 stopped pushing down predicates to Iceberg if a TIMESTAMP literal
has sub-microsecond precision, because rounding the literal to the nearest
microsecond changed the result of comparisons.
Iceberg timestamps have microsecond precision, so these predicates could be
rewritten to equivalent ones by rounding depending on the operator:
* {{ts < x}} and {{ts >= x}}: round x up, e.g. {{ts < '2024-01-01
00:00:00.0000004'}} becomes {{ts < '2024-01-01 00:00:00.000001'}}
* {{ts <= x}} and {{ts > x}}: round x down
* {{ts = x}} is always false and {{ts != x}} is always true; IN and NOT IN can
drop such values
Imported data files can contain nanosecond values (see IMPALA-15639), so the
rewrite must not be used for files whose bounds are not exact.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
