IMPALA-4042: Preserve root types when substituting grouping exprs In case of count(distinct), FunctionCallExpr.analyze() changes type for "NULL" into "BOOLEAN" to make sure that BE doesn't see any "NULL_TYPE" exprs. In the meantime, Expr substitution, happening in Expr.substituteImpl() reverts this change back to original type, "NULL_TYPE".
This causes an issue when AggregateInfo.checkConsistency() performs precondition check where slot types from AggregateInfo.outputTupleDesc_ should be matched with the types from AggregateInfo.groupingExpr_. The slot type shows "BOOLEAN" while type from groupingExpr_ is "NULL_TYPE", which makes the precondition fail and throws an exception. To resolve the issue, preserveRootType is set to true when Expr.substituteList() gets called in AggregateInfo.substitute() Change-Id: Icf3b4511234e473e5b9548fbf3e97f333c9980f1 (cherry picked from commit b17785b4890bedd1c825140ce3c48cd7d9734295) Reviewed-on: http://gerrit.cloudera.org:8080/4600 Reviewed-by: Alex Behm <[email protected]> Tested-by: Internal Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/112ff68e Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/112ff68e Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/112ff68e Branch: refs/heads/master Commit: 112ff68eddde5e29bacf31170741ee7746c0cbb8 Parents: e659337 Author: Yonghyun Hwang <[email protected]> Authored: Fri Sep 30 16:12:46 2016 -0700 Committer: Internal Jenkins <[email protected]> Committed: Wed Oct 5 03:04:17 2016 +0000 ---------------------------------------------------------------------- .../apache/impala/analysis/AggregateInfo.java | 4 +++- .../queries/PlannerTest/distinct.test | 22 ++++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/112ff68e/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java ---------------------------------------------------------------------- diff --git a/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java b/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java index 29b80ef..8590872 100644 --- a/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java +++ b/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java @@ -335,7 +335,9 @@ public class AggregateInfo extends AggregateInfoBase { */ public void substitute(ExprSubstitutionMap smap, Analyzer analyzer) throws InternalException { - groupingExprs_ = Expr.substituteList(groupingExprs_, smap, analyzer, false); + + // Preserve the root type for NULL literals. + groupingExprs_ = Expr.substituteList(groupingExprs_, smap, analyzer, true); LOG.trace("AggInfo: grouping_exprs=" + Expr.debugString(groupingExprs_)); // The smap in this case should not substitute the aggs themselves, only http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/112ff68e/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test ---------------------------------------------------------------------- diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test b/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test index 15f1d59..e5f3bce 100644 --- a/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test +++ b/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test @@ -473,3 +473,25 @@ select * from (select count(distinct int_col) cd from functional.alltypes group 00:SCAN HDFS [functional.alltypes] partitions=24/24 files=24 size=478.45KB ==== +# IMPALA-4042: count(distinct NULL) fails on a view +select count(distinct null) from functional.alltypes_view +---- DISTRIBUTEDPLAN +06:AGGREGATE [FINALIZE] +| output: count:merge(NULL) +| +05:EXCHANGE [UNPARTITIONED] +| +02:AGGREGATE +| output: count(NULL) +| +04:AGGREGATE +| group by: NULL +| +03:EXCHANGE [HASH(NULL)] +| +01:AGGREGATE [STREAMING] +| group by: NULL +| +00:SCAN HDFS [functional.alltypes] + partitions=24/24 files=24 size=478.45KB +====
