[
https://issues.apache.org/jira/browse/IMPALA-15636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy resolved IMPALA-15636.
----------------------------------------
Resolution: Fixed
> Iceberg predicate push down rounds timestamp literals to microseconds
> ---------------------------------------------------------------------
>
> Key: IMPALA-15636
> URL: https://issues.apache.org/jira/browse/IMPALA-15636
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: correctness, impala-iceberg
> Fix For: Impala 5.0.0
>
>
> Iceberg timestamps have microsecond precision, and IcebergPredicateConverter
> rounds TIMESTAMP literals to the nearest microsecond. This changes the result
> of comparisons: e.g. {{ts < '2024-01-01 00:00:00.0000004'}} becomes {{ts <
> '2024-01-01 00:00:00'}}, so Iceberg skips files where {{ts = '2024-01-01
> 00:00:00'}}.
> {code:sql}
> create table t (ts timestamp) partitioned by spec (day(ts)) stored as iceberg;
> insert into t values ('2024-01-01 00:00:00');
> select count(*) from t
> where ts > '2023-01-01' and ts < '2024-01-01 00:00:00.0000004';
> -- returns 0, expected 1
> create table h (ts timestamp) stored as iceberg
> tblproperties ('impala.iceberg.push_down_hint'='ts');
> insert into h values ('2024-01-01 00:00:00');
> select count(*) from h where ts > '2023-12-31 23:59:59.9999996';
> -- returns 0, expected 1
> {code}
> Found while working on IMPALA-15629. Master only pushes down predicates to
> Iceberg if one of them is on a partition column, or on a column listed in the
> {{impala.iceberg.push_down_hint}} table property, so the repros use these.
> IMPALA-15629 would push down predicates on any column.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
