[
https://issues.apache.org/jira/browse/IMPALA-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872593#comment-17872593
]
ASF subversion and git services commented on IMPALA-4551:
---------------------------------------------------------
Commit c8253cee9175219ab04ceff1400b701ec4c368e8 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c8253cee9 ]
IMPALA-13280: Speed up test_under_statement_expression_limit
This test spent over 7 minutes doing constant folding in Frontend
Planner and over 20 minutes doing CodeGen in Coordinator Backend. It was
holding resources from other concurrent test queries during exhaustive
test run.
This test was added by IMPALA-4551, which mainly test expression limit
during query planning. This patch disable codegen and tweak the
expression a bit to cut codegen and constant folding time.
Testing:
- Manually run test_under_statement_expression_limit in my local
machine. After patch, the test can complete in around 4 seconds.
Change-Id: If91b973fef95a89cc24a9172fdf24e2f11a19e6d
Reviewed-on: http://gerrit.cloudera.org:8080/21652
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Yida Wu <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Set limits on size of expression trees
> --------------------------------------
>
> Key: IMPALA-4551
> URL: https://issues.apache.org/jira/browse/IMPALA-4551
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.8.0
> Reporter: Tim Armstrong
> Assignee: Joe McDonnell
> Priority: Major
> Fix For: Impala 3.3.0
>
> Attachments: huge_case.patch
>
>
> Very large expression trees can cause havoc in various Impala components. I
> have been experimenting with the attached test that generates large case
> statements of varying depths and widths, and have been able to hit limits in
> the frontend (Java OOM) and caused various runaway memory usage problems in
> the backend (thrift structures, LLVM IR, codegen, etc).
> We should set some kind of limit here, either on the number of nodes in the
> expression trees, or on the size of the query text, and then make sure that
> we can execute queries of the maximum size end-to-end.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
