[
https://issues.apache.org/jira/browse/IMPALA-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884458#comment-17884458
]
ASF subversion and git services commented on IMPALA-13302:
----------------------------------------------------------
Commit da9400d63ce62e37ea531e7fe6564bf4ac2e0e45 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=da9400d63 ]
IMPALA-13302: Restore registering all conjuncts
Reverts an optimization in IMPALA-12164 that skipped registering
remaining conjuncts if they were expected to be removed by a rewrite
rule, then call markConjunctsAssigned on them.
In some cases the rewrite rule is not applied in the first pass, only
during reAnalyze. During the first pass, the Expr would have
registerConjunct called and an ID assigned because there were no
constant false conjuncts.
During reAnalyze, there would be a constant false conjunct after
rewrite, so the Expr would skip registerConjunct but then have
markConjunctsAssigned with its existing ID. Later a new Expr gets the
same ID from registerConjunct and skips markConjunctsAssigned because
it's believed to already be assigned, skipping slot materialization.
Incomplete rule rewriting is handled separately as IMPALA-13344.
Logs tuple id for eqJoinConjunct and cleans up logging calls with
parameter substitution. Also returns more slot context on Precondition.
Adds ExprRewriteRules tests that previously hit the new Precondition in
markConjunctAssigned, and rewrite PlannerTest.
Change-Id: I5959a3b3e18302e00b1d37e5f50410ebdb224cb0
Reviewed-on: http://gerrit.cloudera.org:8080/21684
Reviewed-by: Riza Suminto <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Some ExprRewriteRule results are not analyzed, leading to unmaterialized
> slots from reAnalyze
> ---------------------------------------------------------------------------------------------
>
> Key: IMPALA-13302
> URL: https://issues.apache.org/jira/browse/IMPALA-13302
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.3.0
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Critical
> Fix For: Impala 4.5.0
>
>
> IMPALA-12164 skipped registering conjuncts that the analyzer expects to
> remove because an earlier conjunct evaluates to constant False. However some
> ExprRewriteRules don't analyze the predicates they produce, which can lead to
> those conjuncts not actually being removed until a reAnalyze phase.
> reAnalyze uses a new Analyzer (with new GlobalState); it restarts counting
> Expr IDs from 0. That can lead to re-using the same Expr ID and marking it as
> assigned. Then when a new Expr gets the same ID, it will skip materializing
> slots, which can cause problems later (like if that Expr is part of a hash
> join).
> Some example queries:
> 1. still a problem
> {code}
> WITH v AS (SELECT 1 FROM functional.alltypestiny t1
> JOIN functional.alltypestiny t2 ON t1.id = t2.id)
> SELECT 1
> FROM functional.alltypestiny t1
> WHERE ((t1.id = 1 and false) or (t1.id = 1 and false))
> AND t1.id = 1 AND t1.id = 1
> UNION ALL
> SELECT 1
> FROM functional.alltypestiny t1
> WHERE ((t1.id = 1 and false) or (t1.id = 1 and false))
> AND t1.id = 1 AND t1.id = 1
> UNION ALL SELECT 1 FROM v
> UNION ALL SELECT 1 FROM v;
> {code}
> 2. already fixed via IMPALA-13203
> {code}
> WITH v as (SELECT 1 FROM functional.alltypes t1
> JOIN functional.alltypes t2 ON t1.id = t2.id)
> SELECT 1 FROM functional.alltypes t1
> WHERE t1.id = 1 AND t1.id = 1 AND t1.id = 1 AND false
> UNION ALL
> SELECT 1 FROM functional.alltypes t1
> WHERE t1.id = 1 AND false
> UNION ALL SELECT 1 FROM v
> UNION ALL SELECT 1 FROM v;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
