[
https://issues.apache.org/jira/browse/IMPALA-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17791796#comment-17791796
]
ASF subversion and git services commented on IMPALA-12580:
----------------------------------------------------------
Commit 0853ceec4a7c3bfbc51cd24bb95d77f314104360 in impala's branch
refs/heads/master from Peter Rozsa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0853ceec4 ]
IMPALA-12580: Fix Iceberg predicate filtering skippedExpression calculation
This patch changes the collection method of skipped expressions from
toSet to toList. toSet uses HashSet, which calls hashCode for
expression nodes, which is not always applicable because expressions
recreated by constant propagation have no id (null) and the default
hashCode implementation for Expr requires it.
Change-Id: I692d3b186e5e73caf9c66ada4afbe36e49641952
Reviewed-on: http://gerrit.cloudera.org:8080/20735
Reviewed-by: Gabor Kaszab <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Iceberg predicate filtering fails with UnsupportedOperationException
> --------------------------------------------------------------------
>
> Key: IMPALA-12580
> URL: https://issues.apache.org/jira/browse/IMPALA-12580
> Project: IMPALA
> Issue Type: Improvement
> Components: fe
> Affects Versions: Impala 4.3.0
> Reporter: Peter Rozsa
> Assignee: Peter Rozsa
> Priority: Major
> Labels: iceberg
>
> If a SELECT contains a predicate that is a candidate for propagation like "a
> = 10 and b = a", the optimizer replaces these predicates with a new one,
> where the id of the predicate is null.
> There's a check in the Expr's hashCode implementation, if the id of the
> expression is null, we throw an UnsupportedOperationException.
> The problem surfaced after IMPALA-12089, to fixthis problem, proper cloning
> and id generation should be used for every expression.
> ([https://github.com/apache/impala/blob/6e90dafcf4b8528061c5ae20933e112c355da834/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L2519)]
> To avoid this problem for Iceberg expression, a quick fix can be applied by
> replacing the collection of expressions from toSet() to toList()
>
> Minimal reproducible example:
> {code:java}
> create table i1(a int, b int, c int) partitioned by spec(a, b) stored as
> iceberg;
> insert into i1(a, b, c) values (1, 1, 1);
> select * from i1 where a = 1 and b = a and c = 1;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
