[
https://issues.apache.org/jira/browse/IGNITE-24688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Timonin updated IGNITE-24688:
------------------------------------
Description:
In the plan below Filter rule is under IgniteTableScan. This leads to memory
overhead. Let's fix it by adding new filter push down rule - Calcite's
FILTER_CORRELATE rule
{code:java}
IgniteColocatedSortAggregate(group=[{0}], ORDER_COUNT=[COUNT()], collation=[[0
ASC-nulls-first]]):
IgniteProject(O_ORDERPRIORITY=[$7])
IgniteFilter(condition=[AND(>=($6, 1998-01-01), <($6, +(1998-01-01,
3:INTERVAL MONTH)))])
IgniteCorrelatedNestedLoopJoin(condition=[true], joinType=[inner],
variablesSet=[[$cor0]], variablesSet=[[0]], correlationVariables=[[$cor0]])
IgniteExchange(distribution=[single])
IgniteSort(sort0=[$7], dir0=[ASC-nulls-first])
IgniteTableScan(table=[[PUBLIC, ORDERS]])
IgniteColocatedHashAggregate(group=[{0}])
IgniteProject(i=[true])
IgniteHashIndexSpool(readType=[LAZY], writeType=[EAGER],
searchRow=[[$cor0.O_ORDERKEY, null, null]], condition=[AND(=($0,
$cor0.O_ORDERKEY), <($1, $2))], allowNulls=[false])
IgniteExchange(distribution=[single])
IgniteTableScan(table=[[PUBLIC, LINEITEM]],
requiredColumns=[{2, 13, 14}])
{code}
was:
In the plan below Filter rule is under IgniteTableScan. This leads to memory
overhead. Let's fix it by adding new filter push down rule - Calcite's
FILTER_CORRELATE rule
{code:java}
IgniteColocatedSortAggregate(group=[{0}], ORDER_COUNT=[COUNT()], collation=[[0
ASC-nulls-first]]): rowcount = 187500.0, cumulative cost = IgniteCost
[rowCount=4.500912075E13, cpu=3.60073862443905E13, memory=1.08021942000009E14,
io=0.0, network=1.08021936E14], id = 2193
IgniteProject(O_ORDERPRIORITY=[$7]): rowcount = 375000.0, cumulative cost =
IgniteCost [rowCount=4.5009120375E13, cpu=3.60073858693905E13,
memory=1.08021942E14, io=0.0, network=1.08021936E14], id = 2192
IgniteFilter(condition=[AND(>=($6, 1998-01-01), <($6, +(1998-01-01,
3:INTERVAL MONTH)))]): rowcount = 375000.0, cumulative cost = IgniteCost
[rowCount=4.500912E13, cpu=3.60073854943905E13, memory=1.08021942E14, io=0.0,
network=1.08021936E14], id = 2191
IgniteCorrelatedNestedLoopJoin(condition=[true], joinType=[inner],
variablesSet=[[$cor0]], variablesSet=[[0]], correlationVariables=[[$cor0]]):
rowcount = 1500000.0, cumulative cost = IgniteCost [rowCount=4.50091185E13,
cpu=3.60073794943905E13, memory=1.08021942E14, io=0.0, network=1.08021936E14],
id = 2190
IgniteExchange(distribution=[single]): rowcount = 1500000.0, cumulative
cost = IgniteCost [rowCount=4500000.0, cpu=6.849439049732597E7, memory=6.6E7,
io=0.0, network=6.6E7], id = 2185
IgniteSort(sort0=[$7], dir0=[ASC-nulls-first]): rowcount = 1500000.0,
cumulative cost = IgniteCost [rowCount=3000000.0, cpu=6.699439049732597E7,
memory=6.6E7, io=0.0, network=0.0], id = 2184
IgniteTableScan(table=[[PUBLIC, ORDERS]]): rowcount = 1500000.0,
cumulative cost = IgniteCost [rowCount=1500000.0, cpu=1500000.0, memory=0.0,
io=0.0, network=0.0], id = 172
IgniteColocatedHashAggregate(group=[{0}]): rowcount = 1.0, cumulative
cost = IgniteCost [rowCount=3.0006075E7, cpu=2.400487E7, memory=7.2014584E7,
io=0.0, network=7.201458E7], id = 2189
IgniteProject(i=[true]): rowcount = 6001215.0, cumulative cost =
IgniteCost [rowCount=2.400486E7, cpu=1.8003655E7, memory=7.201458E7, io=0.0,
network=7.201458E7], id = 2188
IgniteHashIndexSpool(readType=[LAZY], writeType=[EAGER],
searchRow=[[$cor0.O_ORDERKEY, null, null]], condition=[AND(=($0,
$cor0.O_ORDERKEY), <($1, $2))], allowNulls=[false]): rowcount = 6001215.0,
cumulative cost = IgniteCost [rowCount=1.8003645E7, cpu=1.200244E7,
memory=7.201458E7, io=0.0, network=7.201458E7], id = 2187
IgniteExchange(distribution=[single]): rowcount = 6001215.0,
cumulative cost = IgniteCost [rowCount=1.200243E7, cpu=1.200243E7, memory=0.0,
io=0.0, network=7.201458E7], id = 2186
IgniteTableScan(table=[[PUBLIC, LINEITEM]],
requiredColumns=[{2, 13, 14}]): rowcount = 6001215.0, cumulative cost =
IgniteCost [rowCount=6001215.0, cpu=6001215.0, memory=0.0, io=0.0,
network=0.0], id = 224
{code}
> Add FILTER_CORRELATE rule to HEP push down list
> -----------------------------------------------
>
> Key: IGNITE-24688
> URL: https://issues.apache.org/jira/browse/IGNITE-24688
> Project: Ignite
> Issue Type: Improvement
> Reporter: Maksim Timonin
> Assignee: Maksim Timonin
> Priority: Major
> Labels: ise
> Fix For: 2.18
>
>
>
> In the plan below Filter rule is under IgniteTableScan. This leads to memory
> overhead. Let's fix it by adding new filter push down rule - Calcite's
> FILTER_CORRELATE rule
>
> {code:java}
> IgniteColocatedSortAggregate(group=[{0}], ORDER_COUNT=[COUNT()],
> collation=[[0 ASC-nulls-first]]):
> IgniteProject(O_ORDERPRIORITY=[$7])
> IgniteFilter(condition=[AND(>=($6, 1998-01-01), <($6, +(1998-01-01,
> 3:INTERVAL MONTH)))])
> IgniteCorrelatedNestedLoopJoin(condition=[true], joinType=[inner],
> variablesSet=[[$cor0]], variablesSet=[[0]], correlationVariables=[[$cor0]])
> IgniteExchange(distribution=[single])
> IgniteSort(sort0=[$7], dir0=[ASC-nulls-first])
> IgniteTableScan(table=[[PUBLIC, ORDERS]])
> IgniteColocatedHashAggregate(group=[{0}])
> IgniteProject(i=[true])
> IgniteHashIndexSpool(readType=[LAZY], writeType=[EAGER],
> searchRow=[[$cor0.O_ORDERKEY, null, null]], condition=[AND(=($0,
> $cor0.O_ORDERKEY), <($1, $2))], allowNulls=[false])
> IgniteExchange(distribution=[single])
> IgniteTableScan(table=[[PUBLIC, LINEITEM]],
> requiredColumns=[{2, 13, 14}])
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
