[
https://issues.apache.org/jira/browse/IMPALA-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16764557#comment-16764557
]
Tim Armstrong commented on IMPALA-6590:
---------------------------------------
I guess to avoid the quadratic behaviour we'd need to replace the current
ExprSubstitutionMap with a hash map or similar. I vaguely remember that that
was avoided due to some concerns about being able to implement hashCode()
correctly for expressions. That sounds more like a challenge than a genuine
concern to me.
I'd imagine a lot of the overhead with the constant folding is all the setup in
Java_org_apache_impala_service_FeSupport_NativeEvalExprsWithoutRow() to
evaluate each batch of expressions. Probably gets expensive to do if there's
one expression per batch. Which suggests an alternative solution (which I'm not
advocating for) would be to batch up the calls to the backend.
> Disable expr rewrites and codegen for VALUES() statements
> ---------------------------------------------------------
>
> Key: IMPALA-6590
> URL: https://issues.apache.org/jira/browse/IMPALA-6590
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
> Reporter: Alexander Behm
> Priority: Major
> Labels: perf, planner, ramp-up, regression
>
> The analysis of statements with big VALUES clauses like INSERT INTO <tbl>
> VALUES is slow due to expression rewrites like constant folding. The
> performance of such statements has regressed since the introduction of expr
> rewrites and constant folding in IMPALA-1788.
> We should skip expr rewrites for VALUES altogether since it mostly provides
> no benefit but can have a large overhead due to evaluation of expressions in
> the backend (constant folding). These expressions are ultimately evaluated
> and materialized in the backend anyway, so there's no point in folding them
> during analysis.
> Similarly, there is no point in doing codegen for these exprs in the backend
> union node.
> *Workaround*
> {code}
> SET ENABLE_EXPR_REWRITES=FALSE;
> SET DISABLE_CODEGEN=TRUE;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
