Re: Infinite loop with JoinPushTransitivePredicatesRule

[Vitalii Diravka]

The cases is similar to mine.
I have described it in CALCITE-2205 ticket.

Thank you.

Kind regards
Vitalii

On Wed, Mar 7, 2018 at 9:29 AM, Julian Hyde  wrote:

> Do you mean, for example, given
>
>  Filter(x > 5, Scan(Foo))
>
> does it make sense to add the same filter, as follows:
>
>  Filter(x > 5, Filter(x > 5, Scan(Foo)))
>
>
> The second Filter has no effect, because all of its incoming rows already
> obey the predicate “x > 5”. So clearly it doesn’t make sense to add it.
>
> In https://issues.apache.org/jira/browse/CALCITE-1995 <
> https://issues.apache.org/jira/browse/CALCITE-1995> we added that
> capability to the RelBuilder.filter method. I’m not sure why it isn’t
> working in your case. Maybe you’re not using RelBuilder.filter, or maybe
> you didn’t provide an executor, or maybe RelOptPredicateList does not have
> the correct value.
>
> Julian
>
>
> > On Mar 6, 2018, at 12:25 PM, Vitalii Diravka 
> wrote:
> >
> > Hi all.
> >
> > I found two issues which are related to this: CALCITE-2200 (is resolved
> > already) and CALCITE-2205.
> >
> > I have general question: does it makes sense to create LogicalFilter with
> > particular condition for some HepRelVertex with currentRel,
> > which is actually the same desired LogicalFilter?
> >
> > Kind regards
> > Vitalii
> >
> > On Fri, Mar 2, 2018 at 7:42 PM, Chunhui Shi  wrote:
> >
> >> Could you file correspondent JIRAs to both Calcite and Drill with
> detailed
> >> repro steps? This sounds a bug to me. I think this is an issue. We are
> >> seeing some stuck planning when run on latest Calcite with Drill too -
> not
> >> sure if this is relevant though. We could exchange details directly.
> >>
> >> 
> >> From: Vitalii Diravka 
> >> Sent: Thursday, March 1, 2018 4:10:31 PM
> >> To: dev@calcite.apache.org
> >> Subject: Infinite loop with JoinPushTransitivePredicatesRule
> >>
> >> Hi all!
> >>
> >> I got the infinite loop while using FilterIntoJoinRule +
> >> JoinPushTransitivePredicatesRule in HEP planner for some correlated
> >> queries, for instance:
> >> *select d.deptno from sales.emp d where d.deptno IN (select e.deptno
> from
> >> sales.emp e where e.deptno = d.deptno or e.deptno = 4)*
> >>
> >> I have a reproduce in Calcite. We always get HepRelVertex for one side
> of
> >> LogicalJoin relNode or LogicalFilter, therefore the rule is fired every
> >> time and infinitely in result.
> >>
> >> I've noticed that the rule works fine after JoinToCorrelateRule. But I
> >> think this is not a decision for using JoinPushTransitivePredicatesRu
> le,
> >> even from the performance perspective.
> >>
> >> How I can resolve it or maybe I missed something else?
> >>
> >> Kind regards
> >> Vitalii
> >>
>
>

Re: Infinite loop with JoinPushTransitivePredicatesRule

[Julian Hyde]

Do you mean, for example, given

 Filter(x > 5, Scan(Foo))

does it make sense to add the same filter, as follows:

 Filter(x > 5, Filter(x > 5, Scan(Foo)))


The second Filter has no effect, because all of its incoming rows already obey 
the predicate “x > 5”. So clearly it doesn’t make sense to add it.

In https://issues.apache.org/jira/browse/CALCITE-1995 
 we added that capability 
to the RelBuilder.filter method. I’m not sure why it isn’t working in your 
case. Maybe you’re not using RelBuilder.filter, or maybe you didn’t provide an 
executor, or maybe RelOptPredicateList does not have the correct value.

Julian


> On Mar 6, 2018, at 12:25 PM, Vitalii Diravka  
> wrote:
> 
> Hi all.
> 
> I found two issues which are related to this: CALCITE-2200 (is resolved
> already) and CALCITE-2205.
> 
> I have general question: does it makes sense to create LogicalFilter with
> particular condition for some HepRelVertex with currentRel,
> which is actually the same desired LogicalFilter?
> 
> Kind regards
> Vitalii
> 
> On Fri, Mar 2, 2018 at 7:42 PM, Chunhui Shi  wrote:
> 
>> Could you file correspondent JIRAs to both Calcite and Drill with detailed
>> repro steps? This sounds a bug to me. I think this is an issue. We are
>> seeing some stuck planning when run on latest Calcite with Drill too - not
>> sure if this is relevant though. We could exchange details directly.
>> 
>> 
>> From: Vitalii Diravka 
>> Sent: Thursday, March 1, 2018 4:10:31 PM
>> To: dev@calcite.apache.org
>> Subject: Infinite loop with JoinPushTransitivePredicatesRule
>> 
>> Hi all!
>> 
>> I got the infinite loop while using FilterIntoJoinRule +
>> JoinPushTransitivePredicatesRule in HEP planner for some correlated
>> queries, for instance:
>> *select d.deptno from sales.emp d where d.deptno IN (select e.deptno from
>> sales.emp e where e.deptno = d.deptno or e.deptno = 4)*
>> 
>> I have a reproduce in Calcite. We always get HepRelVertex for one side of
>> LogicalJoin relNode or LogicalFilter, therefore the rule is fired every
>> time and infinitely in result.
>> 
>> I've noticed that the rule works fine after JoinToCorrelateRule. But I
>> think this is not a decision for using JoinPushTransitivePredicatesRule,
>> even from the performance perspective.
>> 
>> How I can resolve it or maybe I missed something else?
>> 
>> Kind regards
>> Vitalii
>> 

Re: Infinite loop with JoinPushTransitivePredicatesRule

[Vitalii Diravka]

Hi all.

I found two issues which are related to this: CALCITE-2200 (is resolved
already) and CALCITE-2205.

I have general question: does it makes sense to create LogicalFilter with
particular condition for some HepRelVertex with currentRel,
which is actually the same desired LogicalFilter?

Kind regards
Vitalii

On Fri, Mar 2, 2018 at 7:42 PM, Chunhui Shi  wrote:

> Could you file correspondent JIRAs to both Calcite and Drill with detailed
> repro steps? This sounds a bug to me. I think this is an issue. We are
> seeing some stuck planning when run on latest Calcite with Drill too - not
> sure if this is relevant though. We could exchange details directly.
>
> 
> From: Vitalii Diravka 
> Sent: Thursday, March 1, 2018 4:10:31 PM
> To: dev@calcite.apache.org
> Subject: Infinite loop with JoinPushTransitivePredicatesRule
>
> Hi all!
>
> I got the infinite loop while using FilterIntoJoinRule +
> JoinPushTransitivePredicatesRule in HEP planner for some correlated
> queries, for instance:
> *select d.deptno from sales.emp d where d.deptno IN (select e.deptno from
> sales.emp e where e.deptno = d.deptno or e.deptno = 4)*
>
> I have a reproduce in Calcite. We always get HepRelVertex for one side of
> LogicalJoin relNode or LogicalFilter, therefore the rule is fired every
> time and infinitely in result.
>
> I've noticed that the rule works fine after JoinToCorrelateRule. But I
> think this is not a decision for using JoinPushTransitivePredicatesRule,
> even from the performance perspective.
>
> How I can resolve it or maybe I missed something else?
>
> Kind regards
> Vitalii
>