[
https://issues.apache.org/jira/browse/IMPALA-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111510#comment-17111510
]
Tim Armstrong commented on IMPALA-2651:
---------------------------------------
This has improved a lot over time, I think to a large degree because of reduced
inlining, and maybe the LLVM upgrades help. The overhead can still be high and
it is not cancellable.
> codegen overhead can be extremely large
> ---------------------------------------
>
> Key: IMPALA-2651
> URL: https://issues.apache.org/jira/browse/IMPALA-2651
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.2, Impala 2.3.0
> Reporter: Silvius Rus
> Priority: Minor
> Labels: codegen, performance
>
> We received reports of excessive codegen compilation/optimization times for
> very large expressions generated by visualization tools.
> We should:
> # Expose codegen optimization levels as query options. Currently there is
> only an all or nothing codegen query option. It's likely that overly complex
> expressions such as hundreds of cascading conditions take very long and
> benefit very little from an O2 optimization level, but they could still run
> significantly faster even at O0 or O1 versus interpreted.
> # Consider dropping to O1 (or turn off riskier passes individually)
> automatically for very large expressions.
> # Consider parameterizing the compilation duration time limits and set a
> reasonable default, say 10 seconds. Either disable codegen or reduce it to,
> say, O0 if compilation takes longer than the preset limit.
> *Workaround*
> In some cases disabling codegen can help.
> {code}
> SET disable_codegen=true;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
