Zoltán Borók-Nagy created IMPALA-15643:
------------------------------------------
Summary: Negated Iceberg predicates on partition columns match
NULL partitions
Key: IMPALA-15643
URL: https://issues.apache.org/jira/browse/IMPALA-15643
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Zoltán Borók-Nagy
Assignee: Zoltán Borók-Nagy
Iceberg evaluates {{!=}}, {{NOT IN}} and {{NOT}} as true for a NULL partition
value, while in Impala they are NULL for a NULL value, so the row doesn't
match. The scanner doesn't evaluate predicates again that Iceberg fully
evaluated on the partition values, so queries return the rows of the NULL
partition, and ALTER TABLE DROP PARTITION drops it. This also happens for
partition transforms, e.g. {{truncate()}}. {{NOT LIKE}} on a string partition
column with a NULL partition fails with a NullPointerException in Iceberg.
{code:sql}
create table t (p int, x int) partitioned by spec (p) stored as iceberg;
insert into t values (NULL, 10), (1, 10), (5, 10);
select p from t where p != 5; -- returns 1 and NULL, expected 1
select p from t where p not in (5, 7); -- returns 1 and NULL, expected 1
select p from t where not (p = 5); -- returns 1 and NULL, expected 1
select p from t where not (p = 5 or p = 7); -- returns 1 and NULL,
expected 1
select p from t where not (p = 5 and x = 10); -- returns 1 and NULL,
expected 1
alter table t drop partition (p != 5);
-- drops 2 partitions, expected 1 (p=1)
create table s (p string, x int) partitioned by spec (p) stored as iceberg;
insert into s values (NULL, 10), ('a', 10), ('b', 10);
select p from s where p not like 'a%';
-- NullPointerException: Cannot invoke "String.startsWith(String)" because the
-- return value of "org.apache.iceberg.expressions.BoundReference.eval(...)"
is null
create table d (p int, x int) partitioned by spec (truncate(10, p)) stored as
iceberg;
insert into d values (NULL, 10), (1, 10), (25, 10);
select p from d where p != 25; -- returns 1 and NULL, expected 1
{code}
Other comparisons ({{<}}, {{<=}}, {{>}}, {{=}}, {{IN}}, {{LIKE}}, {{IS NULL}})
are not affected, because Iceberg evaluates them as false for NULL values, just
like Impala doesn't match NULL values with them.
Found while working on IMPALA-15629.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
