[jira] [Commented] (CALCITE-3982) FilterMergeRule can lead to AssertionError

2020-05-09 Thread Julian Hyde (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103609#comment-17103609
 ] 

Julian Hyde commented on CALCITE-3982:
--

That sounds right. There are a few places where we use RexProgram as an 
intermediate utility and now that RelBuilder exists there are probably better 
(and more efficient) ways.

> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This could potentially happen since Filter creation has a check on whether 
> the expression is flat 
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
>  and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT 
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
>   at org.apache.calcite.rel.core.Filter.(Filter.java:74)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
>   at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
>   at 
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3982) FilterMergeRule can lead to AssertionError

2020-05-09 Thread Jesus Camacho Rodriguez (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103589#comment-17103589
 ] 

Jesus Camacho Rodriguez commented on CALCITE-3982:
--

It seems this issue appears in Hive because we override some of the methods in 
the builder to avoid calling expression simplification when we create a filter.

I have not been able to reproduce the issue in Calcite. However, it seems to me 
that using the RexProgram in FilterMergeRule is unnecessary in any case 
([~zabetak] pointed this out in the Hive PR)? I have created a patch that 
removes the usage of the program and relies solely on the builder, and I could 
not find any test regression: https://github.com/apache/calcite/pull/1968 .

> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This could potentially happen since Filter creation has a check on whether 
> the expression is flat 
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
>  and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT 
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
>   at org.apache.calcite.rel.core.Filter.(Filter.java:74)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
>   at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
>   at 
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3982) FilterMergeRule can lead to AssertionError

2020-05-08 Thread Julian Hyde (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103012#comment-17103012
 ] 

Julian Hyde commented on CALCITE-3982:
--

Does that mean that some other component is not flattening? Let's fix that 
component.

> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> This could potentially happen since Filter creation has a check on whether 
> the expression is flat 
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
>  and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT 
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
>   at org.apache.calcite.rel.core.Filter.(Filter.java:74)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
>   at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
>   at 
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3982) FilterMergeRule can lead to AssertionError

2020-05-08 Thread Jesus Camacho Rodriguez (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102973#comment-17102973
 ] 

Jesus Camacho Rodriguez commented on CALCITE-3982:
--

FilterMergeRule [relies on the 
builder|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/FilterMergeRule.java#L82],
 however it seems it only flattens the top expression. I think one possible fix 
is to create a variant of ExpansionShuttle that flattens the expression as it 
is being expanded.

> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> This could potentially happen since Filter creation has a check on whether 
> the expression is flat 
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
>  and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT 
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
>   at org.apache.calcite.rel.core.Filter.(Filter.java:74)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
>   at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
>   at 
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3982) FilterMergeRule can lead to AssertionError

2020-05-08 Thread Julian Hyde (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102971#comment-17102971
 ] 

Julian Hyde commented on CALCITE-3982:
--

Consider using RelBuilder. It flattens.

> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> This could potentially happen since Filter creation has a check on whether 
> the expression is flat 
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
>  and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT 
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
>   at org.apache.calcite.rel.core.Filter.(Filter.java:74)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
>   at 
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
>   at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
>   at 
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)