[jira] [Commented] (CALCITE-2204) Volcano Planner may not choose the cheapest cost plan
[ https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818505#comment-16818505 ] LeoWangLZ commented on CALCITE-2204: [~danny0405] The bug is some different with CALCITE-2018 , it may be loop so it skip to calculate the cost and lost the best plan while propagate to calculate cost > Volcano Planner may not choose the cheapest cost plan > -- > > Key: CALCITE-2204 > URL: https://issues.apache.org/jira/browse/CALCITE-2204 > Project: Calcite > Issue Type: Bug >Reporter: LeoWangLZ >Priority: Major > Labels: pull-request-available > > Volcano Planner will propagate the cost improvement to parents that one of > the inputs has the best plan. But it not propagate to all parents firstly, it > propagate one parent and go to the parents of parent. In the way, if one > parent may propagate to other parent on the same level, and the cost maybe > less than others, but when propagate the parent again, it will not propagate > because the cost is already calculated, and it's can't be less then the cost > of the self. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2204) Volcano Planner may not choose the cheapest cost plan
[ https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818504#comment-16818504 ] LeoWangLZ commented on CALCITE-2204: [~Chunwei Lei] ok > Volcano Planner may not choose the cheapest cost plan > -- > > Key: CALCITE-2204 > URL: https://issues.apache.org/jira/browse/CALCITE-2204 > Project: Calcite > Issue Type: Bug >Reporter: LeoWangLZ >Priority: Major > Labels: pull-request-available > > Volcano Planner will propagate the cost improvement to parents that one of > the inputs has the best plan. But it not propagate to all parents firstly, it > propagate one parent and go to the parents of parent. In the way, if one > parent may propagate to other parent on the same level, and the cost maybe > less than others, but when propagate the parent again, it will not propagate > because the cost is already calculated, and it's can't be less then the cost > of the self. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2204) Volcano Planner may not choose the cheapest cost plan
[ https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LeoWangLZ updated CALCITE-2204: --- Description: Volcano Planner will propagate the cost improvement to parents that one of the inputs has the best plan. But it not propagate to all parents firstly, it propagate one parent and go to the parents of parent. In the way, if one parent may propagate to other parent on the same level, and the cost maybe less than others, but when propagate the parent again, it will not propagate because the cost is already calculated, and it's can't be less then the cost of the self. (was: Volcano Planner will propagate the cost improvement to parents that one of the inputs has the best plan. But it not propagate to all parents firstly, it propagate one parent and go to the parent s of parent. In the way, if one parent may propagate to other parent on the same level, and the cost maybe less than others, but when propagate the parent again, it will not propagate because the cost is already calculated, and it's can't be less then the cost of the self.) > Volcano Planner may not choose the cheapest cost plan > -- > > Key: CALCITE-2204 > URL: https://issues.apache.org/jira/browse/CALCITE-2204 > Project: Calcite > Issue Type: Bug >Reporter: LeoWangLZ >Assignee: Julian Hyde >Priority: Major > > Volcano Planner will propagate the cost improvement to parents that one of > the inputs has the best plan. But it not propagate to all parents firstly, it > propagate one parent and go to the parents of parent. In the way, if one > parent may propagate to other parent on the same level, and the cost maybe > less than others, but when propagate the parent again, it will not propagate > because the cost is already calculated, and it's can't be less then the cost > of the self. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2204) Volcano Planner may not choose the cheapest cost plan
[ https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LeoWangLZ updated CALCITE-2204: --- Summary: Volcano Planner may not choose the cheapest cost plan (was: Volcano Planner may not choose the cheapest cost plan wrongly) > Volcano Planner may not choose the cheapest cost plan > -- > > Key: CALCITE-2204 > URL: https://issues.apache.org/jira/browse/CALCITE-2204 > Project: Calcite > Issue Type: Bug >Reporter: LeoWangLZ >Assignee: Julian Hyde >Priority: Major > > Volcano Planner will propagate the cost improvement to parents that one of > the inputs has the best plan. But it not propagate to all parents firstly, it > propagate one parent and go to the parent s of parent. In the way, if one > parent may propagate to other parent on the same level, and the cost maybe > less than others, but when propagate the parent again, it will not propagate > because the cost is already calculated, and it's can't be less then the cost > of the self. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2204) Volcano Planner may not choose the cheapest cost plan wrongly
[ https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LeoWangLZ updated CALCITE-2204: --- Summary: Volcano Planner may not choose the cheapest cost plan wrongly (was: Volcano Planner may not choose the cheapest cost of plan wrongly) > Volcano Planner may not choose the cheapest cost plan wrongly > - > > Key: CALCITE-2204 > URL: https://issues.apache.org/jira/browse/CALCITE-2204 > Project: Calcite > Issue Type: Bug >Reporter: LeoWangLZ >Assignee: Julian Hyde >Priority: Major > > Volcano Planner will propagate the cost improvement to parents that one of > the inputs has the best plan. But it not propagate to all parents firstly, it > propagate one parent and go to the parent s of parent. In the way, if one > parent may propagate to other parent on the same level, and the cost maybe > less than others, but when propagate the parent again, it will not propagate > because the cost is already calculated, and it's can't be less then the cost > of the self. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2204) Volcano Planner may not choose the cheapest cost of plan wrongly
[
https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385600#comment-16385600
]
LeoWangLZ edited comment on CALCITE-2204 at 3/5/18 5:35 AM:
[~julianhyde] In function propagateCostImprovements0, This subset is already in
the chain being propagated to. This means that the graph is cyclic, and
therefore the cost of this relational expression - not this subset - must be
infinite. it may miss the best plan. I think we should change the way of
propagation. now the algorithm like pre-order, it may use Breadth-first Search.
like
{code:java}
parent21,parent22
parent11, parent12
input{code}
now the order of propagation is input->parent11->parent21->parent22->parent12.
use BFS, it maybe input->parent11->parent12->parent21->parent22.
Suppose the one parent of parent21 is parent12, then it exists a cycle.
and the first propagation of parent12 happen in
input->parent11->parent21->parent12, so it can get the cost and best relNode,
and go to parent22, it's RelSubset maybe exists in activeSet, so it will stop.
and when the next propagation of parent12, the cost have be calculated, and so
it would not be less then itself, and it will not propagate to parents.
then the best of parent12 can't be chosen.
was (Author: perid007):
[~julianhyde] In function propagateCostImprovements0, This subset is already in
the chain being propagated to. This means that the graph is cyclic, and
therefore the cost of this relational expression - not this subset - must be
infinite. it may miss the best plan. I think we should change the way of
propagation. now the algorithm like pre-order, it may use Breadth-first Search.
like
{code:java}
parent21,parent22
parent11, parent12
input{code}
now the order of propagation is input->parent11->parent22->parent12.
use BFS, it maybe input->parent11->parent12->parent21->parent22,
and so, it only concern the relNode in RelSubset instead of RelSubset. it will
no have a cycle in RelSubset, the cost will be propagated normally
> Volcano Planner may not choose the cheapest cost of plan wrongly
>
>
> Key: CALCITE-2204
> URL: https://issues.apache.org/jira/browse/CALCITE-2204
> Project: Calcite
> Issue Type: Bug
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>Priority: Major
>
> Volcano Planner will propagate the cost improvement to parents that one of
> the inputs has the best plan. But it not propagate to all parents firstly, it
> propagate one parent and go to the parent s of parent. In the way, if one
> parent may propagate to other parent on the same level, and the cost maybe
> less than others, but when propagate the parent again, it will not propagate
> because the cost is already calculated, and it's can't be less then the cost
> of the self.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2204) Volcano Planner may not choose the cheapest cost of plan wrongly
[
https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385600#comment-16385600
]
LeoWangLZ edited comment on CALCITE-2204 at 3/5/18 5:49 AM:
[~julianhyde] In function propagateCostImprovements0, This subset is already in
the chain being propagated to. This means that the graph is cyclic, and
therefore the cost of this relational expression - not this subset - must be
infinite. it may miss the best plan. I think we should change the way of
propagation. now the algorithm like pre-order, it may use Breadth-first Search.
like
{code:java}
parent21,parent22
parent11, parent12
input{code}
now the order of propagation is input->parent11->parent21->parent22->parent12.
use BFS, it maybe input->parent11->parent12->parent21->parent22.
Suppose the one parent of parent21 is parent12, then it exists a cycle.
and the first propagation of parent12 happen in
input->parent11->parent21->parent12, so it can get the cost and best relNode,
and go to parent22, it's RelSubset maybe exists in activeSet, so it will stop.
and when the next propagation of parent12, the cost have be calculated, and so
it would not be less then itself, and it will not propagate to parents.
then the best of parent12 can't be chosen.
I propose my solution. Please review it.
https://github.com/apache/calcite/pull/643
and it's hard for adding test for it. so I don't add test in this patch.
and I will try to write one. Thanks
was (Author: perid007):
[~julianhyde] In function propagateCostImprovements0, This subset is already in
the chain being propagated to. This means that the graph is cyclic, and
therefore the cost of this relational expression - not this subset - must be
infinite. it may miss the best plan. I think we should change the way of
propagation. now the algorithm like pre-order, it may use Breadth-first Search.
like
{code:java}
parent21,parent22
parent11, parent12
input{code}
now the order of propagation is input->parent11->parent21->parent22->parent12.
use BFS, it maybe input->parent11->parent12->parent21->parent22.
Suppose the one parent of parent21 is parent12, then it exists a cycle.
and the first propagation of parent12 happen in
input->parent11->parent21->parent12, so it can get the cost and best relNode,
and go to parent22, it's RelSubset maybe exists in activeSet, so it will stop.
and when the next propagation of parent12, the cost have be calculated, and so
it would not be less then itself, and it will not propagate to parents.
then the best of parent12 can't be chosen.
> Volcano Planner may not choose the cheapest cost of plan wrongly
>
>
> Key: CALCITE-2204
> URL: https://issues.apache.org/jira/browse/CALCITE-2204
> Project: Calcite
> Issue Type: Bug
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>Priority: Major
>
> Volcano Planner will propagate the cost improvement to parents that one of
> the inputs has the best plan. But it not propagate to all parents firstly, it
> propagate one parent and go to the parent s of parent. In the way, if one
> parent may propagate to other parent on the same level, and the cost maybe
> less than others, but when propagate the parent again, it will not propagate
> because the cost is already calculated, and it's can't be less then the cost
> of the self.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2204) Volcano Planner may not choose the cheapest cost of plan wrongly
[
https://issues.apache.org/jira/browse/CALCITE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385600#comment-16385600
]
LeoWangLZ commented on CALCITE-2204:
[~julianhyde] In function propagateCostImprovements0, This subset is already in
the chain being propagated to. This means that the graph is cyclic, and
therefore the cost of this relational expression - not this subset - must be
infinite. it may miss the best plan. I think we should change the way of
propagation. now the algorithm like pre-order, it may use Breadth-first Search.
like
{code:java}
parent21,parent22
parent11, parent12
input{code}
now the order of propagation is input->parent11->parent22->parent12.
use BFS, it maybe input->parent11->parent12->parent21->parent22,
and so, it only concern the relNode in RelSubset instead of RelSubset. it will
no have a cycle in RelSubset, the cost will be propagated normally
> Volcano Planner may not choose the cheapest cost of plan wrongly
>
>
> Key: CALCITE-2204
> URL: https://issues.apache.org/jira/browse/CALCITE-2204
> Project: Calcite
> Issue Type: Bug
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>Priority: Major
>
> Volcano Planner will propagate the cost improvement to parents that one of
> the inputs has the best plan. But it not propagate to all parents firstly, it
> propagate one parent and go to the parent s of parent. In the way, if one
> parent may propagate to other parent on the same level, and the cost maybe
> less than others, but when propagate the parent again, it will not propagate
> because the cost is already calculated, and it's can't be less then the cost
> of the self.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2116) The digests are not same for the common sub-expressions in HepPlanner
[
https://issues.apache.org/jira/browse/CALCITE-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385589#comment-16385589
]
LeoWangLZ commented on CALCITE-2116:
[~julianhyde] ok, it like Spool that it has one producer and multiple
consumers. Thanks
> The digests are not same for the common sub-expressions in HepPlanner
> -
>
> Key: CALCITE-2116
> URL: https://issues.apache.org/jira/browse/CALCITE-2116
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.16.0
>
>
> The digests are not same for the same sub-query.
> like query
> {code:java}
> select * from (select * from dept union all select * from dept)a;
> {code}
> When run the query, The union relNode has two inputs, and it's same expectly
> and it's the same object instead of the print of plan, but Now they are not
> same.
> And for Test testReplaceCommonSubexpression, the rule is only firing once,
> but now it's two.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-2204) Volcano Planner may not choose the cheapest cost of plan wrongly
LeoWangLZ created CALCITE-2204: -- Summary: Volcano Planner may not choose the cheapest cost of plan wrongly Key: CALCITE-2204 URL: https://issues.apache.org/jira/browse/CALCITE-2204 Project: Calcite Issue Type: Bug Reporter: LeoWangLZ Assignee: Julian Hyde Volcano Planner will propagate the cost improvement to parents that one of the inputs has the best plan. But it not propagate to all parents firstly, it propagate one parent and go to the parent s of parent. In the way, if one parent may propagate to other parent on the same level, and the cost maybe less than others, but when propagate the parent again, it will not propagate because the cost is already calculated, and it's can't be less then the cost of the self. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2116) The digests are not same for the common sub-expressions in HepPlanner
[
https://issues.apache.org/jira/browse/CALCITE-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323510#comment-16323510
]
LeoWangLZ edited comment on CALCITE-2116 at 1/12/18 4:05 AM:
-
[~julianhyde] Thanks
I have another concern about "If a node has only one consumer it would continue
to use ListSink."
If we can reuse every HepRelVertex, then one node maybe the input of more than
one parents, it's not related with the number of consumer, it depends on the
number of parents, if one parent consume the one, then others can't consume it
because the input's empty.
I try to write one test that one node have more than one parents.
Suppose If it's no problem, that's great.
was (Author: perid007):
[~julianhyde] I have another concern about "If a node has only one consumer it
would continue to use ListSink."
If we can reuse every HepRelVertex, then one node maybe the input of more than
one parents, it's not related with the number of consumer, it depends on the
number of parents, if one parent consume the one, then other can't consume it
because the input's empty.
I try to write one test that one node have more than one parents.
Suppose If it's no problem, that's great.
> The digests are not same for the common sub-expressions in HepPlanner
> -
>
> Key: CALCITE-2116
> URL: https://issues.apache.org/jira/browse/CALCITE-2116
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> The digests are not same for the same sub-query.
> like query
> {code:java}
> select * from (select * from dept union all select * from dept)a;
> {code}
> When run the query, The union relNode has two inputs, and it's same expectly
> and it's the same object instead of the print of plan, but Now they are not
> same.
> And for Test testReplaceCommonSubexpression, the rule is only firing once,
> but now it's two.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2116) The digests are not same for the common sub-expressions in HepPlanner
[
https://issues.apache.org/jira/browse/CALCITE-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16323510#comment-16323510
]
LeoWangLZ commented on CALCITE-2116:
[~julianhyde] I have another concern about "If a node has only one consumer it
would continue to use ListSink."
If we can reuse every HepRelVertex, then one node maybe the input of more than
one parents, it's not related with the number of consumer, it depends on the
number of parents, if one parent consume the one, then other can't consume it
because the input's empty.
I try to write one test that one node have more than one parents.
Suppose If it's no problem, that's great.
> The digests are not same for the common sub-expressions in HepPlanner
> -
>
> Key: CALCITE-2116
> URL: https://issues.apache.org/jira/browse/CALCITE-2116
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> The digests are not same for the same sub-query.
> like query
> {code:java}
> select * from (select * from dept union all select * from dept)a;
> {code}
> When run the query, The union relNode has two inputs, and it's same expectly
> and it's the same object instead of the print of plan, but Now they are not
> same.
> And for Test testReplaceCommonSubexpression, the rule is only firing once,
> but now it's two.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2116) The digests are not same for the common sub-expressions in HepPlanner
[
https://issues.apache.org/jira/browse/CALCITE-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312397#comment-16312397
]
LeoWangLZ commented on CALCITE-2116:
[~julianhyde] https://github.com/apache/calcite/pull/597
For the same subquery, we think they have same relNode, but when adding relNode
to graph, the digest are not the same,
like testReplaceCommonSubexpression, for subquery "select * from dept", it's
expected that the trees of relNode are same, but not.
The reason is that when finding the digest of one relNode by mapDigestToVertex,
the digest is from constructor instead of recompute, so the digest can't
contain the input of digest and it's not match for the same subquery.
and if the problem is fixed, the union has same inputs for the same subquery.
and while executing the result, the source of union are the same, and get one
row and remove it for ListSource, but it need be got twice, then the another
source is removed at the same time because of the inputs are the same, so the
code of ListSource would be modified that the row can't be removed
> The digests are not same for the common sub-expressions in HepPlanner
> -
>
> Key: CALCITE-2116
> URL: https://issues.apache.org/jira/browse/CALCITE-2116
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> The digests are not same for the same sub-query.
> like query
> {code:java}
> select * from (select * from dept union all select * from dept)a;
> {code}
> When run the query, The union relNode has two inputs, and it's same expectly
> and it's the same object instead of the print of plan, but Now they are not
> same.
> And for Test testReplaceCommonSubexpression, the rule is only firing once,
> but now it's two.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-2111) Make HepPlanner more efficient by applying rules depth-first
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312316#comment-16312316
]
LeoWangLZ edited comment on CALCITE-2111 at 1/5/18 1:21 AM:
[~julianhyde] ok. Thanks
was (Author: perid007):
[~julianhyde] I got it, Thanks
> Make HepPlanner more efficient by applying rules depth-first
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2111) Make HepPlanner more efficient by applying rules depth-first
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312316#comment-16312316
]
LeoWangLZ commented on CALCITE-2111:
[~julianhyde] I got, Thanks
> Make HepPlanner more efficient by applying rules depth-first
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-2111) Make HepPlanner more efficient by applying rules depth-first
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16312316#comment-16312316
]
LeoWangLZ edited comment on CALCITE-2111 at 1/5/18 1:21 AM:
[~julianhyde] I got it, Thanks
was (Author: perid007):
[~julianhyde] I got, Thanks
> Make HepPlanner more efficient by applying rules depth-first
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
> Fix For: 1.16.0
>
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307160#comment-16307160
]
LeoWangLZ edited comment on CALCITE-2111 at 12/31/17 10:02 AM:
---
[~julianhyde] I have fixed the bug, and Tests are passed.Please review again
was (Author: perid007):
[~julianhyde] I fix the bug, Please review again, and Tests are passed.
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307160#comment-16307160
]
LeoWangLZ commented on CALCITE-2111:
[~julianhyde] I fix the bug, Please review again, and Tests are passed.
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-2116) The digests are not same for the common sub-expressions in HepPlanner
LeoWangLZ created CALCITE-2116:
--
Summary: The digests are not same for the common sub-expressions
in HepPlanner
Key: CALCITE-2116
URL: https://issues.apache.org/jira/browse/CALCITE-2116
Project: Calcite
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: LeoWangLZ
Assignee: Julian Hyde
The digests are not same for the same sub-query.
like query
{code:java}
select * from (select * from dept union all select * from dept)a;
{code}
When run the query, The union relNode has two inputs, and it's same expectly
and it's the same object instead of the print of plan, but Now they are not
same.
And for Test testReplaceCommonSubexpression, the rule is only firing once, but
now it's two.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307015#comment-16307015
]
LeoWangLZ edited comment on CALCITE-2111 at 12/31/17 12:26 AM:
---
[~julianhyde] Thanks
It's a bug that It don't update the digest of vertex rightly when updateVertex.
when a transformation happen, if the current relNode is still the origin
relNode, then only update new vertex instead of generating new current relNode,
and the new digest of new vertex is the same as the origin, but
mapDigestToVertex direct the old vertex, and when collectGC, remove the origin
vertex, and the new vertex is not in mapDigestToVertex, so when run
belongsToDag, it not exists in graph.
But why is it ok before? Because It don't affect the rule apply although vertex
is not in mapDigestToVertex. and The graph iterator is regenerate, and it don't
using mapDigestToVertex when applying rule.
was (Author: perid007):
[~julianhyde] Thanks
It don't update the digest of vertex rightly when updateVertex. when a
transformation happen, if the current relNode is still the origin relNode, then
only update new vertex instead of generating new current relNode, and the new
digest of new vertex is the same as the origin, but mapDigestToVertex direct
the old vertex, and when collectGC, remove the origin vertex, and the new
vertex is not in mapDigestToVertex, so when run belongsToDag, it not exists in
graph.
But why is it ok before? Because It don't affect the rule apply although vertex
is not in mapDigestToVertex. and The graph iterator is regenerate, and it don't
using mapDigestToVertex when applying rule.
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307015#comment-16307015
]
LeoWangLZ commented on CALCITE-2111:
[~julianhyde] Thanks
It don't update the digest of vertex rightly when updateVertex. when a
transformation happen, if the current relNode is still the origin relNode, then
only update new vertex instead of generating new current relNode, and the new
digest of new vertex is the same as the origin, but mapDigestToVertex direct
the old vertex, and when collectGC, remove the origin vertex, and the new
vertex is not in mapDigestToVertex, so when run belongsToDag, it not exists in
graph.
But why is it ok before? Because It don't affect the rule apply although vertex
is not in mapDigestToVertex. and The graph iterator is regenerate, and it don't
using mapDigestToVertex when applying rule.
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304242#comment-16304242
]
LeoWangLZ edited comment on CALCITE-2111 at 12/27/17 5:42 AM:
--
[~julianhyde] I have fixed the tests broke. Because I have no permission to
push code to your branch, and I push again on my branch.
I make Depth First Order to restart from root like ARBITRARY, or it lost the
chance of next rule apply. and so it increases the count of rule apply for the
testRuleApply test.
Please review again https://github.com/apache/calcite/pull/592 . The second
commit:
https://github.com/apache/calcite/pull/592/commits/8d58c1918144758b3ad9e1fc4608cad8505f4083
.Thanks
was (Author: perid007):
[~julianhyde] I have fixed the tests broke. Because I have no permission to
push code to your branch, and I push again on my branch.
I make Depth First Order to restart from root like ARBITRARY, or it lost the
chance of next rule apply. and so it increase the count of rule apply for the
testRuleApply test.
Please review again https://github.com/apache/calcite/pull/592 . The second
commit:
https://github.com/apache/calcite/pull/592/commits/8d58c1918144758b3ad9e1fc4608cad8505f4083
.Thanks
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304242#comment-16304242
]
LeoWangLZ commented on CALCITE-2111:
[~julianhyde] I have fixed the tests broke. Because I have no permission to
push code to your branch, and I push again on my branch.
I make Depth First Order to restart from root like ARBITRARY, or it lost the
chance of next rule apply. and so it increase the count of rule apply for the
testRuleApply test.
Please review again https://github.com/apache/calcite/pull/592 . The second
commit:
https://github.com/apache/calcite/pull/592/commits/8d58c1918144758b3ad9e1fc4608cad8505f4083
.Thanks
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304158#comment-16304158
]
LeoWangLZ commented on CALCITE-2111:
[~julianhyde] ok, I will see your dev branch to focus several tests broke
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16303432#comment-16303432
]
LeoWangLZ edited comment on CALCITE-2111 at 12/26/17 5:51 AM:
--
Thanks [~julianhyde]
The advantages are decreasing the rule apply that has no transformation, and
Contrast to other mode, it is the most efficient. For my test, rule applied
decrease some orders of magnitude. The query contains more and more union all,
it can reduce more rule apply. so for complex query it has more efficient.
And it has no other disadvantages compared with other modes.
When I fix the bug, I think whether we need to modify the implementation of
ARBITRARY Order that is most efficient compared to other two, or add new mode,
but I choose the latter.
was (Author: perid007):
Thanks [~julianhyde]
The advantages are decreasing the rule apply that has no transformation, and
Contrast to other mode, the count of apply rule is the most efficient. For my
test, rule applied decrease some orders of magnitude. The query contains more
and more union all, it can reduce more rule apply. so for complex query it has
more efficient.
And it has no other disadvantages compared with other modes.
When I fix the bug, I think whether we need to modify the implementation of
ARBITRARY Order that is most efficient compared to other two, or add new mode,
but I choose the latter.
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LeoWangLZ updated CALCITE-2111:
---
Affects Version/s: 1.15.0
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16303432#comment-16303432
]
LeoWangLZ commented on CALCITE-2111:
Thanks [~julianhyde]
The advantages are decreasing the rule apply that has no transformation, and
Contrast to other mode, the count of apply rule is the most efficient. For my
test, rule applied decrease some orders of magnitude. The query contains more
and more union all, it can reduce more rule apply. so for complex query it has
more efficient.
And it has no other disadvantages compared with other modes.
When I fix the bug, I think whether we need to modify the implementation of
ARBITRARY Order that is most efficient compared to other two, or add new mode,
but I choose the latter.
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-2111) HepPlanner apply more and more rules
[
https://issues.apache.org/jira/browse/CALCITE-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16303313#comment-16303313
]
LeoWangLZ commented on CALCITE-2111:
[~julianhyde] https://github.com/apache/calcite/pull/592
It apply more rules for HepPlanner, after transformation in one rule apply, and
it apply rule from the current new vertex in ARBITRARY Order, then it restart
from the root. so the previous relNodes apply again and again when generating
new vertex in the first time.
we apply rule for all relNodes in depth first order, and then all rules for all
input's relNodes are applied, it would restart from the root instead of one
transformation in the first time.
> HepPlanner apply more and more rules
>
>
> Key: CALCITE-2111
> URL: https://issues.apache.org/jira/browse/CALCITE-2111
> Project: Calcite
> Issue Type: Bug
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
>
> HepPlanner#applyRules will fully restart after transformation.
> if the rule for the inner relNode happens transformation, then the previous
> all relNode will be apply for all rules again. and the accumulated rule apply
> will be exploded as rule apply.
> like query that it contains hundreds of union all
> {code:java}
> select * from (
> select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50011895 union all
> select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013023 union all
> select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013032 union all
> select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013024 union all
> select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50004204 union all
> select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50013043 union all
> select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 290903 union all
> select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
> require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 50008261 union all
> select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
> require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
> require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
> and SAL = 124478013 union all
> ...
> )a;
> {code}
> For rules ReduceExpressionsRule.FILTER_INSTANCE and
> ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (CALCITE-2111) HepPlanner apply more and more rules
LeoWangLZ created CALCITE-2111:
--
Summary: HepPlanner apply more and more rules
Key: CALCITE-2111
URL: https://issues.apache.org/jira/browse/CALCITE-2111
Project: Calcite
Issue Type: Bug
Reporter: LeoWangLZ
Assignee: Julian Hyde
HepPlanner#applyRules will fully restart after transformation.
if the rule for the inner relNode happens transformation, then the previous all
relNode will be apply for all rules again. and the accumulated rule apply will
be exploded as rule apply.
like query that it contains hundreds of union all
{code:java}
select * from (
select ENAME, 50011895 as cat_id, '1' as cat_name, 1 as
require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 50011895 union all
select ENAME, 50013023 as cat_id, '2' as cat_name, 0 as
require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 50013023 union all
select ENAME, 50013032 as cat_id, '3' as cat_name, 0 as
require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 50013032 union all
select ENAME, 50013024 as cat_id, '4' as cat_name, 0 as
require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 50013024 union all
select ENAME, 50004204 as cat_id, '5' as cat_name, 0 as
require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 50004204 union all
select ENAME, 50013043 as cat_id, '6' as cat_name, 0 as
require_free_postage, 0 as require_15return, 0 as require_48hour,0 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 50013043 union all
select ENAME, 290903 as cat_id, '7' as cat_name, 1 as
require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 290903 union all
select ENAME, 50008261 as cat_id, '8' as cat_name, 1 as
require_free_postage, 0 as require_15return, 0 as require_48hour,1 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 50008261 union all
select ENAME, 124478013 as cat_id, '9' as cat_name, 0 as
require_free_postage, 0 as require_15return, 1 as require_48hour,0 as
require_insurance from emp where EMPNO = 20171216 and MGR = 0 and ENAME = 'Y'
and SAL = 124478013 union all
...
)a;
{code}
For rules ReduceExpressionsRule.FILTER_INSTANCE and
ReduceExpressionsRule.PROJECT_INSTANCE, it apply 1W+ rules.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1990) Make RelDistribution extend RelMultipleTrait
[
https://issues.apache.org/jira/browse/CALCITE-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288617#comment-16288617
]
LeoWangLZ commented on CALCITE-1990:
Thanks [~julianhyde], Thank for your test cases and guava's Orderings
> Make RelDistribution extend RelMultipleTrait
>
>
> Key: CALCITE-1990
> URL: https://issues.apache.org/jira/browse/CALCITE-1990
> Project: Calcite
> Issue Type: Bug
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
> Fix For: 1.15.0
>
> Original Estimate: 0.2h
> Remaining Estimate: 0.2h
>
> In Distributed System, RelDistribution is used for Exchange or SortExchange.
> for some operators it may deliver RelDistribution Trait, but some operator
> like SortedMergeJoin may deliver multiple traits.
> eg:
> {code:java}
> Query:
> select * from T1 join T2 on T1.c1=T2.d1;
> Suppose Plan:
> SortedMergeJoin
> Exchange(c1)
> T1(c1)
> Exchange(d1)
> T2(d1)
> {code}
> than SortedMergeJoin can deliver RelDistribution(hash\[c1\]) or
> RelDistribution(hash\[d1\]).
> we can consider RelDistribution extend RelMultipleTrait like RelCollation.
> EnumerableMergeJoin is the case for RelCollation, and RelDistribution is also
> fit for SortedMergeJoin in Distributed system
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (CALCITE-1960) RelMdPredicates.getPredicates is slow if there are many equivalent columns
[
https://issues.apache.org/jira/browse/CALCITE-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176394#comment-16176394
]
LeoWangLZ commented on CALCITE-1960:
Thanks
> RelMdPredicates.getPredicates is slow if there are many equivalent columns
> --
>
> Key: CALCITE-1960
> URL: https://issues.apache.org/jira/browse/CALCITE-1960
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: LeoWangLZ
>Assignee: Julian Hyde
> Fix For: 1.15.0
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When inferring pulled up predicates, multiple mappings are generated to
> make sure equivalent expressions can be substitute. E.g., for an expression
> 'a + b + c' and the following equivalences:
> {code:sql}
> a : {a, b}
> b : {a, b}
> c : {c, e}
> {code}
> should generate:
> {code:sql}
> a + a + c
> a + a + e
> a + b + c
> a + b + e
> b + a + c
> b + a + e
> b + b + c
> b + b + e
> {code}
> The mapping generation is a typical permutation generation process. However,
> the current code is not handling the permutation well, thus causing
> duplicated
> mappings.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
