[
https://issues.apache.org/jira/browse/CALCITE-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17871750#comment-17871750
]
Krisztian Kasa commented on CALCITE-6513:
-----------------------------------------
Fixed the javadoc comment of the test:
[https://github.com/apache/calcite/pull/3900/commits/a556178641e714385e1ae95e96b01958cd6a217f]
Please review.
> FilterProjectTransposeRule may cause OOM when Project expressions are complex
> -----------------------------------------------------------------------------
>
> Key: CALCITE-6513
> URL: https://issues.apache.org/jira/browse/CALCITE-6513
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.38.0
>
>
> CALCITE-3774 addresses preventing merging projects when the resulting
> expressions in the merged project are too complex and lead to slow
> compilation or out of memory.
> However, when there is a {{Filter}} on top of the {{Projects}} with a
> predicate referencing the complex expressions {{FilterProjectTransposeRule}}
> tries to push down the {{Filter}} below the bottom {{Project}} merging the
> expressions and causing OOM.
> The issue was initially reproduced using Hive with the Hive version of
> {{FilterProjectTransposeRule}}. See: HIVE-28264
> Calcite is also affected:
> [https://github.com/kasakrisz/calcite/commit/b35a02f368624a9c4768f348cd072a95ed6de3e1]
> Let's see the following query
> {code}
> SELECT x1 from
> (SELECT 'L1' || x0 || x0 || x0 || x0 as x1 from
> (SELECT 'L0' || ENAME || ENAME || ENAME || ENAME as x0 from emp) t1)
> t2
> WHERE x1 = 'Something'
> {code}
> Let's set the bloat property of RelBuilder.Config to 3.
> The initial plan of the query is:
> {code}
> LogicalProject(X1=[$0])
> LogicalFilter(condition=[=($0, 'Something')])
> LogicalProject(X1=[||(||(||(||('L1', $0), $0), $0), $0)])
> LogicalProject(X0=[||(||(||(||('L0', $1), $1), $1), $1)])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The expressions in the {{Project}} operators are mergeable, but the resulting
> expression's complexity exceeds the limit of 3 in our example.
> However, while applying {{FilterProjectTransposeRule}} the expressions in the
> {{Project}} operators are merged because the expression in the upper
> {{Project}} references the expression in the lower {{Project}} and the
> predicate in the {{Filter}} operator also references it. The limit is not
> applied this case, so we end up with a plan
> {code}
> LogicalProject(X1=[$0])
> LogicalProject(X1=[||(||(||(||('L1', $0), $0), $0), $0)])
> LogicalProject(X0=[||(||(||(||('L0', $1), $1), $1), $1)])
> LogicalFilter(condition=[=(||(||(||(||('L1', ||(||(||(||('L0', $1),
> $1), $1), $1)), ||(||(||(||('L0', $1), $1), $1), $1)), ||(||(||(||('L0', $1),
> $1), $1), $1)), ||(||(||(||('L0', $1), $1), $1), $1)), 'Something')])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
