[
https://issues.apache.org/jira/browse/IMPALA-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892293#comment-16892293
]
Joe McDonnell commented on IMPALA-4551:
---------------------------------------
Taking a look at this, so I'll assign it to myself.
Applying a maximum total number of expressions is relatively simple. When I
apply a limit of 100,000 expressions, the test case fails quickly. After I
reduce the fanout on the provided test case so that it is less than 100,000, it
finishes in about 10-15 minutes. It spends most of its time in codegen.
An alternative is to limit the actual query text size along with the text size
of the views referenced. Either way, the limit will be configurable and we'll
need to think about an appropriate default.
> Set limits on size of expression trees
> --------------------------------------
>
> Key: IMPALA-4551
> URL: https://issues.apache.org/jira/browse/IMPALA-4551
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.8.0
> Reporter: Tim Armstrong
> Priority: Major
> Attachments: huge_case.patch
>
>
> Very large expression trees can cause havoc in various Impala components. I
> have been experimenting with the attached test that generates large case
> statements of varying depths and widths, and have been able to hit limits in
> the frontend (Java OOM) and caused various runaway memory usage problems in
> the backend (thrift structures, LLVM IR, codegen, etc).
> We should set some kind of limit here, either on the number of nodes in the
> expression trees, or on the size of the query text, and then make sure that
> we can execute queries of the maximum size end-to-end.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
