Zoltán Borók-Nagy created IMPALA-15636:
------------------------------------------
Summary: Iceberg predicate push down rounds timestamp literals to
microseconds
Key: IMPALA-15636
URL: https://issues.apache.org/jira/browse/IMPALA-15636
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Zoltán Borók-Nagy
Assignee: Zoltán Borók-Nagy
Iceberg timestamps have microsecond precision, and IcebergPredicateConverter
rounds TIMESTAMP literals to the nearest microsecond. This changes the result
of comparisons: e.g. {{ts < '2024-01-01 00:00:00.0000004'}} becomes {{ts <
'2024-01-01 00:00:00'}}, so Iceberg skips files where {{ts = '2024-01-01
00:00:00'}}.
{code:sql}
create table t (ts timestamp) partitioned by spec (day(ts)) stored as iceberg;
insert into t values ('2024-01-01 00:00:00');
select count(*) from t
where ts > '2023-01-01' and ts < '2024-01-01 00:00:00.0000004';
-- returns 0, expected 1
create table h (ts timestamp) stored as iceberg
tblproperties ('impala.iceberg.push_down_hint'='ts');
insert into h values ('2024-01-01 00:00:00');
select count(*) from h where ts > '2023-12-31 23:59:59.9999996';
-- returns 0, expected 1
{code}
Found while working on IMPALA-15629. Master only pushes down predicates to
Iceberg if one of them is on a partition column, or on a column listed in the
{{impala.iceberg.push_down_hint}} table property, so the repros use these.
IMPALA-15629 would push down predicates on any column.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
