[
https://issues.apache.org/jira/browse/IMPALA-9620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080164#comment-17080164
]
ASF subversion and git services commented on IMPALA-9620:
---------------------------------------------------------
Commit 293dc2ec92d0cadf1d3803a22e66e762d0ff6cf1 in impala's branch
refs/heads/master from Aman Sinha
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=293dc2e ]
IMPALA-9620: Ensure group-by and cnf exprs are analyzed
This change initializes the SelectStmt's groupingExprs_
with the analyzed version. It also analyzes the new predicates
created by the Conjunctive Normal Form rewrite rule such that
potential consumers of this rewrite don't encounter problems.
Before this change, the SelectStmt.analyzeGroupingExprs() made
a deep copy of the original grouping exprs, then analyzed the
copy but left the original intact. This causes problems because
a rewrite rule (invoked by SelectStmt.rewriteExprs()) may try to
process the original grouping exprs and encounter INVALID_TYPE
(types are only assigned after analyze). This was the root cause
of the problem described in the JIRA. Although this was a pre-
existing behavior, it gets exposed when enable_cnf_rewrites=true.
Note that the deep-copied analyzed grouping exprs are supplied
to MultiAggregateInfo and since many operations are using
this data structure, we don't see widespread issues.
This patch fixes it and as a conservative measure, does the
analyze of new predicates in the CNF rule. (note: there are likely
other rewrite rules where explicit analyze should be done but
that is outside the scope for this issue).
Testing:
- Added new unit tests with predicates in SELECT and GROUP BY
- Ran 'mvn test' for the FE
Change-Id: I6da4a17c6e648f466ce118c4646520ff68f9878e
Reviewed-on: http://gerrit.cloudera.org:8080/15693
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Predicates in the SELECT and GROUP-BY cause failure with CNF rewrite enabled
> ----------------------------------------------------------------------------
>
> Key: IMPALA-9620
> URL: https://issues.apache.org/jira/browse/IMPALA-9620
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Aman Sinha
> Assignee: Aman Sinha
> Priority: Major
>
> Predicates can appear in the SELECT and GROUP BY list as part of IF(), CASE()
> clauses.
> When enable_cnf_rewrites is set to true, such queries encounter failure
> during planning as shown below. Queries run successfully when the flag is
> disabled.
> Note that the predicate does not have to be disjunctive predicate for this
> failure to occur..even other types of predicates repro the issue.
> {noformat}
> set enable_cnf_rewrites = true;
> select l_quantity, if(l_quantity < 5 or l_quantity > 45, 'invalid', 'valid')
> from lineitem group by l_quantity, if(l_quantity < 5 or l_quantity > 45,
> 'invalid', 'valid') limit 5
> ERROR: IllegalStateException: null
> {noformat}
> Stack trace:
> {noformat}
> I0407 17:40:40.306650 31240 jni-util.cc:288]
> 2741e90d2edac592:c625a35f00000000] java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at org.apache.impala.analysis.SlotRef.getIdsHelper(SlotRef.java:229)
> at org.apache.impala.analysis.Expr.getIdsHelper(Expr.java:1286)
> at org.apache.impala.analysis.Expr.getIdsHelper(Expr.java:1286)
> at org.apache.impala.analysis.Expr.getIds(Expr.java:1279)
> at
> org.apache.impala.rewrite.ConvertToCNFRule.convertToCNF(ConvertToCNFRule.java:111)
> at
> org.apache.impala.rewrite.ConvertToCNFRule.apply(ConvertToCNFRule.java:86)
> at
> org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:85)
> at
> org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:83)
> at
> org.apache.impala.rewrite.ExprRewriter.applyRuleRepeatedly(ExprRewriter.java:71)
> at
> org.apache.impala.rewrite.ExprRewriter.rewrite(ExprRewriter.java:55)
> at
> org.apache.impala.analysis.SelectStmt.rewriteCheckOrdinalResult(SelectStmt.java:1043)
> at
> org.apache.impala.analysis.SelectStmt.rewriteExprs(SelectStmt.java:1068)
> at
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:472)
> at
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:415)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1530)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1497)
> {noformat}
> Other variations of the same pattern that also encounter similar failure:
> {noformat}
> explain select case when not (l_quantity = 5) then 0 else 1 end from lineitem
> group by case when not (l_quantity = 5) then 0 else 1 end
> ERROR: IllegalStateException: null
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
