[
https://issues.apache.org/jira/browse/IMPALA-7785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16669357#comment-16669357
]
Paul Rogers edited comment on IMPALA-7785 at 10/30/18 10:52 PM:
----------------------------------------------------------------
Thanks [~Philip]. Switched to master; problem does not occur. Problem is a bit
more subtle. Updated the title and description. Here is a query that should
fail on master. (Have not tried it on master, but it fails on my branch and
does not touch any of the code I changed.)
{code:sql}
SELECT id = 0 && true
FROM functional.alltypes
GROUP BY id = 0 && true
{code}
Yet, the following works:
{code:sql}
SELECT true && id = 0
FROM functional.alltypes
GROUP BY id = 0
{code}
The reason it works is that we manually rewrote the {{GROUP BY}} clause to
match how the rewrite engine rewrote the {{SELECT}} clause.
The customer workaround is to write both expressions in a form that the rewrite
engine won't alter. (That does not help us for IMPALA-7655.)
was (Author: paul.rogers):
Thanks [~Philip]. Switched to master; problem does not occur. Closing this
ticket.
> GROUP BY clause not analyzed prior to rewrite step
> --------------------------------------------------
>
> Key: IMPALA-7785
> URL: https://issues.apache.org/jira/browse/IMPALA-7785
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.0
> Reporter: Paul Rogers
> Priority: Minor
>
> The FE fails to analyze a {{GROUP BY}} clause prior to invoking the rewrite
> rules, causing the rules to fail to do any rewrites.
> For the {{SELECT}} list, the analyzer processes each expression and marks it
> as analyzed.
> The rewrite rules, however, tend to skip unanalyzed nodes. (And, according to
> IMPALA-7754, often are not re-analyzed after a rewrite.)
> Consider this simple query:
> {code:sql}
> SELECT case when string_col is not null then string_col else 'foo' end
>
> FROM functional.alltypestiny
> GROUP BY case when string_col is not null then string_col else 'foo' end
>
> {code}
> This query works. Now, using the new feature in IMPALA-7655 with a query that
> will be rewritten to the above:
> {code:sql}
> SELECT coalesce(string_col, 'foo')
> FROM functional.alltypes
> GROUP BY coalesce(string_col, 'foo')
> {code}
> The above is rewritten using the new conditional function rewrite rules.
> Result:
> {noformat}
> org.apache.impala.common.AnalysisException:
> select list expression not produced by aggregation output
> (missing from GROUP BY clause?):
> CASE WHEN string_col IS NOT NULL THEN string_col ELSE 'foo' END
> {noformat}
> The reason is the check used in multiple rewrite rules:
> {code:java}
> public Expr apply(Expr expr, Analyzer analyzer) throws AnalysisException {
>
> if (!expr.isAnalyzed()) return expr;
>
> {code}
> Step though the code. The {{coalesce()}} expression in the {{SELECT}} clause
> is analyzed, the one in the {{GROUP BY}} is not. This creates a problem
> because SQL semantics require the identical expression in both clause for
> them to match. (It also means no other rewrite rules, at least not those with
> this check, are invoked, leading to an unintended code path.)
> This query makes it a bit clearer:
> {code:sql}
> SELECT 1 + 2
> FROM functional.alltypestiny
> GROUP BY 1 + 2
> {code}
> This works. But, if we use test code to inspect the "rewritten" {{GROUP BY}},
> we find that it is still at "1 + 2" while the {{SELECT}} expression has been
> rewritten to "3".
> Seems that, when working with rewrites, we must be very careful because, as
> the code currently is written, we rewrite some clauses but not others. Then,
> we have to know when it is safe to have the SELECT clause differ from the
> GROUP BY clause. (Looks like it is OK for constants to differ, but not for
> functions...)
> VERY confusing, would be better to just fix the darn thing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
