[
https://issues.apache.org/jira/browse/IMPALA-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-2651.
-----------------------------------
Resolution: Fixed
This is no longer actionable on its own. Several linked tickets address
specific ideas, and other efforts - async codegen, codegen caching, other
improvements to codegen time - are also done or planned.
> codegen overhead can be high
> ----------------------------
>
> Key: IMPALA-2651
> URL: https://issues.apache.org/jira/browse/IMPALA-2651
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.2, Impala 2.3.0
> Reporter: Silvius Rus
> Assignee: Michael Smith
> Priority: Minor
> Labels: codegen, performance
>
> We received reports of excessive codegen compilation/optimization times for
> very large expressions generated by visualization tools.
> We should:
> # Expose codegen optimization levels as query options. Currently there is
> only an all or nothing codegen query option. It's likely that overly complex
> expressions such as hundreds of cascading conditions take very long and
> benefit very little from an O2 optimization level, but they could still run
> significantly faster even at O0 or O1 versus interpreted.
> # Consider dropping to O1 (or turn off riskier passes individually)
> automatically for very large expressions.
> # Consider parameterizing the compilation duration time limits and set a
> reasonable default, say 10 seconds. Either disable codegen or reduce it to,
> say, O0 if compilation takes longer than the preset limit.
> *Workaround*
> In some cases disabling codegen can help.
> {code}
> SET disable_codegen=true;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
