Zoltán Borók-Nagy created IMPALA-15634:
------------------------------------------
Summary: Iceberg LIKE predicate push down returns wrong results
Key: IMPALA-15634
URL: https://issues.apache.org/jira/browse/IMPALA-15634
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Zoltán Borók-Nagy
Assignee: Arnab Karmakar
IcebergPredicateConverter has several problems with LIKE predicates:
* A pattern like {{'ab_'}} is converted to {{startsWith('ab')}} and marked as
fully converted. So the scanner doesn't evaluate it on identity partition
columns, and SHOW FILES and ALTER TABLE DROP PARTITION use it as an exact
filter.
* NOT is applied to relaxed predicates: {{NOT (s LIKE 'a%b')}} becomes {{NOT
startsWith(s, 'a')}}, which skips files with values like {{'axc'}}.
* Backslashes are not interpreted the way the backend interprets them. The
backend matches constant and simple prefix patterns verbatim, see
{{LikePredicate::OptimizeConstantPatternMatch()}}. In its regex path, a
backslash escapes any character, and a trailing lone backslash is dropped. The
converter only unescapes a backslash that is followed by %, _ or another
backslash.
{code:sql}
create table p (i int, s string) partitioned by spec (s) stored as iceberg;
insert into p values (1, 'abcd'), (2, 'abc');
select i from p where s like 'ab_';
-- returns 1 and 2, expected 2
alter table p drop partition (s like 'ab_');
-- drops both partitions, expected only s='abc'
create table t (s string) stored as iceberg
tblproperties ('impala.iceberg.push_down_hint'='s');
insert into t values ('axc');
insert into t values ('abcd');
insert into t values ('dab');
insert into t values ('dcd');
select count(*) from t where s not like 'a%b'; -- 2, expected 4
select count(*) from t where s not like 'ab_'; -- 3, expected 4
create table e (s string) stored as iceberg
tblproperties ('impala.iceberg.push_down_hint'='s');
insert into e values ('abxc');
insert into e values (concat('a', chr(92), chr(92), 'bz'));
insert into e values ('a%b');
select count(*) from e where s like 'a\\b%c'; -- 0, expected 1 (1st row)
select count(*) from e where s like 'a\\\\b%'; -- 0, expected 1 (2nd row)
select count(*) from e where s like 'a\\%b\\'; -- 0, expected 1 (3rd row)
{code}
Found while working on IMPALA-15629. Master only pushes down predicates to
Iceberg if one of them is on a partition column, or on a column listed in the
{{impala.iceberg.push_down_hint}} table property, so the repros use these.
IMPALA-15629 would push down predicates on any column.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
