[
https://issues.apache.org/jira/browse/IMPALA-11040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17459740#comment-17459740
]
ASF subversion and git services commented on IMPALA-11040:
----------------------------------------------------------
Commit ad29ce70b3f4cfab4105be52ef664129060fa04e in impala's branch
refs/heads/master from skyyws
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ad29ce7 ]
IMPALA-11040: Remove unnecessary reset() method in class 'UnionStmt'
When query contains multiple nested union stmt, more than twenty or
thirty, and needs 'reAnalyze', such as rewrite expr. Query would
execute slowly, even failed due to 'reset' method called in class
'UnionStmt' and 'SetOperationStmt'.
'SetOperationStmt' is added in IMPALA-9943 and IMPALA-4974. Multiple
nested union stmt will lead to 'reset' called numbers to grow
exponentially. Since 'operands_' will be reset in two class' reset()
method, and handle with their children recursively. Too many nested
union stmt will cause deep nesting.
UnionStmt.reset() content is exactly same as SetOperationStmt.reset().
This patch removed this method in 'UnionStmt'. After this, the
original query would execute quickly.
An example already add in file 'union.test', without this patch, the
example query would execute slowly, or even fail.
Testing:
- Added new test case in 'union.test'
Change-Id: I408a396d40d9622f2ae6c459f49cbfcc19affd14
Reviewed-on: http://gerrit.cloudera.org:8080/18061
Reviewed-by: Qifan Chen <[email protected]>
Reviewed-by: Zoltan Borok-Nagy <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Query get stuck when containing multiple nested union
> -----------------------------------------------------
>
> Key: IMPALA-11040
> URL: https://issues.apache.org/jira/browse/IMPALA-11040
> Project: IMPALA
> Issue Type: Question
> Components: Frontend
> Affects Versions: Impala 4.0.0
> Reporter: Sheng Wang
> Assignee: Sheng Wang
> Priority: Major
> Fix For: Impala 4.1.0
>
>
> When query contains multiple union, and encounter a expr rewrite. Then this
> query will do 'reAnalyze', and get stuck in 'reset' method. Here is the
> example to reproduce this bug:
> {code:java}
> SELECT `t1`.`dt`
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
> (SELECT *
> FROM
>
> (SELECT *
> FROM
>
> (SELECT *
>
> FROM
>
> (SELECT *
>
> FROM
>
> (SELECT *
>
> FROM
>
> (SELECT *
>
> FROM
>
> (SELECT *
>
> FROM
>
> (SELECT *
>
> FROM
>
> (SELECT '20211020' AS `dt`
>
> UNION ALL SELECT '20211021' AS `dt`) AS `t`
>
> UNION ALL SELECT '20211022' AS `dt`) AS `t`
>
> UNION ALL SELECT '20211023' AS `dt`) AS `t`
>
> UNION ALL SELECT '20211024' AS `dt`) AS `t`
>
> UNION ALL SELECT '20211025' AS `dt`) AS `t`
>
> UNION ALL SELECT '20211026' AS `dt`) AS `t`
>
> UNION ALL SELECT '20211027' AS `dt`) AS `t`
> UNION
> ALL SELECT '20211028' AS `dt`) AS `t`
> UNION
> ALL SELECT '20211029' AS `dt`) AS `t`
> UNION ALL
> SELECT '20211030' AS `dt`) AS `t`
> UNION ALL
> SELECT '20211031' AS `dt`) AS `t`
> UNION ALL SELECT
> '20211101' AS `dt`) AS `t`
> UNION ALL SELECT
> '20211102' AS `dt`) AS `t`
> UNION ALL SELECT
> '20211103' AS `dt`) AS `t`
> UNION ALL SELECT
> '20211104' AS `dt`) AS `t`
> UNION ALL SELECT '20211105'
> AS `dt`) AS `t`
> UNION ALL SELECT '20211106' AS
> `dt`) AS `t`
> UNION ALL SELECT '20211107' AS
> `dt`) AS `t`
> UNION ALL SELECT '20211108' AS `dt`)
> AS `t`
> UNION ALL SELECT '20211109' AS `dt`) AS
> `t`
> UNION ALL SELECT '20211110' AS `dt`) AS `t`
> UNION ALL SELECT '20211111' AS `dt`) AS `t`
> UNION ALL SELECT '20211112' AS `dt`) AS `t`
> UNION ALL SELECT '20211113' AS `dt`) AS `t`
> UNION ALL SELECT '20211114' AS `dt`) AS `t`
> UNION ALL SELECT '20211115' AS `dt`) AS `t`
> UNION ALL SELECT '20211116' AS `dt`) AS `t`
> UNION ALL SELECT '20211117' AS `dt`) AS `t`
> UNION ALL SELECT '20211118' AS `dt`) AS `t`
> UNION ALL SELECT '20211119' AS `dt`) AS `t`
> UNION ALL SELECT '20211120' AS `dt`) AS `t1`
> CROSS JOIN
> (SELECT 1000+1000) AS `t2`
> GROUP BY `t1`.`dt`;
> {code}
> This query may take long time to execute, such as ten minutes. If we reduce
> 'UNION', the query will execute quickly.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
