[
https://issues.apache.org/jira/browse/IMPALA-15643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy resolved IMPALA-15643.
----------------------------------------
Resolution: Fixed
> Negated Iceberg predicates on partition columns match NULL partitions
> ---------------------------------------------------------------------
>
> Key: IMPALA-15643
> URL: https://issues.apache.org/jira/browse/IMPALA-15643
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Critical
> Labels: correctness, impala-iceberg
> Fix For: Impala 5.0.0
>
>
> Iceberg evaluates {{!=}}, {{NOT IN}} and {{NOT}} as true for a NULL partition
> value, while in Impala they are NULL for a NULL value, so the row doesn't
> match. The scanner doesn't evaluate predicates again that Iceberg fully
> evaluated on the partition values, so queries return the rows of the NULL
> partition, and ALTER TABLE DROP PARTITION drops it. This also happens for
> partition transforms, e.g. {{truncate()}}. {{NOT LIKE}} on a string partition
> column with a NULL partition fails with a NullPointerException in Iceberg.
> {code:sql}
> create table t (p int, x int) partitioned by spec (p) stored as iceberg;
> insert into t values (NULL, 10), (1, 10), (5, 10);
> select p from t where p != 5; -- returns 1 and NULL, expected 1
> select p from t where p not in (5, 7); -- returns 1 and NULL, expected 1
> select p from t where not (p = 5); -- returns 1 and NULL, expected 1
> select p from t where not (p = 5 or p = 7); -- returns 1 and NULL,
> expected 1
> select p from t where not (p = 5 and x = 10); -- returns 1 and NULL,
> expected 1
> alter table t drop partition (p != 5);
> -- drops 2 partitions, expected 1 (p=1)
> create table s (p string, x int) partitioned by spec (p) stored as iceberg;
> insert into s values (NULL, 10), ('a', 10), ('b', 10);
> select p from s where p not like 'a%';
> -- NullPointerException: Cannot invoke "String.startsWith(String)" because the
> -- return value of
> "org.apache.iceberg.expressions.BoundReference.eval(...)" is null
> create table d (p int, x int) partitioned by spec (truncate(10, p)) stored as
> iceberg;
> insert into d values (NULL, 10), (1, 10), (25, 10);
> select p from d where p != 25; -- returns 1 and NULL, expected 1
> {code}
> Other comparisons ({{<}}, {{<=}}, {{>}}, {{=}}, {{IN}}, {{LIKE}}, {{IS
> NULL}}) are not affected, because Iceberg evaluates them as false for NULL
> values, just like Impala doesn't match NULL values with them.
> Found while working on IMPALA-15629.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
