[jira] [Closed] (CALCITE-4551) Reusing Immutable metadata cache keys
[ https://issues.apache.org/jira/browse/CALCITE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez closed CALCITE-4551. Fix Version/s: 1.29.0 Resolution: Fixed > Reusing Immutable metadata cache keys > - > > Key: CALCITE-4551 > URL: https://issues.apache.org/jira/browse/CALCITE-4551 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Fix For: 1.29.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Create cache keys for metadata calls generates a fair bit of object turn in > trivial cases, more expensive than the actual metadata call. Many metadata > calls could reuse their cache keys since the are functionally identical. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-4551) Reusing Immutable metadata cache keys
[ https://issues.apache.org/jira/browse/CALCITE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated CALCITE-4551: - Summary: Reusing Immutable metadata cache keys (was: Fly Weight for MD Cache Keys) > Reusing Immutable metadata cache keys > - > > Key: CALCITE-4551 > URL: https://issues.apache.org/jira/browse/CALCITE-4551 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Create cache keys for metadata calls generates a fair bit of object turn in > trivial cases, more expensive than the actual metadata call. Many metadata > calls could reuse their cache keys since the are functionally identical. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4551) Fly Weight for MD Cache Keys
[ https://issues.apache.org/jira/browse/CALCITE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435652#comment-17435652 ] Jesus Camacho Rodriguez commented on CALCITE-4551: -- Fixed in [2c17f7a|https://github.com/apache/calcite/commit/2c17f7afec58b79f5e715cae5b43f9ca8da39cf4]. > Fly Weight for MD Cache Keys > > > Key: CALCITE-4551 > URL: https://issues.apache.org/jira/browse/CALCITE-4551 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Create cache keys for metadata calls generates a fair bit of object turn in > trivial cases, more expensive than the actual metadata call. Many metadata > calls could reuse their cache keys since the are functionally identical. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-4551) Fly Weight for MD Cache Keys
[ https://issues.apache.org/jira/browse/CALCITE-4551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned CALCITE-4551: Assignee: James Starr > Fly Weight for MD Cache Keys > > > Key: CALCITE-4551 > URL: https://issues.apache.org/jira/browse/CALCITE-4551 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Create cache keys for metadata calls generates a fair bit of object turn in > trivial cases, more expensive than the actual metadata call. Many metadata > calls could reuse their cache keys since the are functionally identical. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-4550) Simplify JaninoRelMetadataProvider API for binding methods
[ https://issues.apache.org/jira/browse/CALCITE-4550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-4550. -- Fix Version/s: 1.28.0 Resolution: Fixed Fixed in [77ff9f6|https://github.com/apache/calcite/commit/77ff9f6145272a899f694c6a2d8ab8ff5422d0c6]. Thanks [~jamesstarr]! > Simplify JaninoRelMetadataProvider API for binding methods > -- > > Key: CALCITE-4550 > URL: https://issues.apache.org/jira/browse/CALCITE-4550 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > JaninoRelMetadataProvider uses the a complicated convention for binding > handler instances to a the dynamically created handler. It requires the > method to stated once in Metadata, then again MetadataHandler with slightly > different signature and finally again a reference to the Metadata method must > be included MetadataDef. This is needlessly rather complicated and verbose > for something that is trying to reduce boiler plate. > Having JaninoRelMetadataProvider get the declared methods directly from the > handler interface would decouple JaninoRelMetadataProvider from method > definitions in Metadata and MetadataDef. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4834) JaninoRelMetadataProvider uses hardcoded class name
[
https://issues.apache.org/jira/browse/CALCITE-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425051#comment-17425051
]
Jesus Camacho Rodriguez commented on CALCITE-4834:
--
Thanks [~rubenql], good catch!
> JaninoRelMetadataProvider uses hardcoded class name
> ---
>
> Key: CALCITE-4834
> URL: https://issues.apache.org/jira/browse/CALCITE-4834
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Ruben Q L
>Assignee: Ruben Q L
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.28.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This seems caused by CALCITE-4546
> ([fbbdf46|https://github.com/apache/calcite/commit/fbbdf465df46b4e6f9863d7d1dfdcb19a43f2032])
> {{JaninoRelMetadataProvider}} lines 135 & 141 is using (for the generated
> code) the hardcoded name of the class {{MetadataDef}}:
> {code}
> buff.append(" private final org.apache.calcite.rel.metadata.MetadataDef
> def;\n");
> for (Map.Entry, String> handlerAndName :
> handlerToName.entrySet()) {
> ...
> .append(" org.apache.calcite.rel.metadata.MetadataDef def");
> {code}
> This can lead to issues (e.g. if a downstream project shades Calcite
> library). The safer way to do this is using {{Class#getName}}, as it is
> already done in the rest of the code, e.g. in
> {{JaninoRelMetadataProvider:158}}:
> {code}
> .append(MetadataDef.class.getName())
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4546) Change metadata dispatch to avoid registration of all RelNode subtypes
[
https://issues.apache.org/jira/browse/CALCITE-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-4546:
-
Fix Version/s: 1.28.0
> Change metadata dispatch to avoid registration of all RelNode subtypes
> --
>
> Key: CALCITE-4546
> URL: https://issues.apache.org/jira/browse/CALCITE-4546
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: James Starr
>Assignee: James Starr
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.28.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the
> last 3 results as reported by intellij. I then repeated this 3 times.
> After bench marking Janino, my prototype and using legacy java reflections, I
> found:
> | |Janino||*Prototype*||*java reflection*||
> |Average|326.444|315.556|1525.89|
> |Standard Deviation|27.12983188|14.75729575|75.8476177|
> The prototype was a static code with out caching or cycle detection. I
> latter added cycle detection and caching, but the results were with in one
> standard deviation. So I didn't not follow up further.
> I was doing the dispatch with instance of instead of scanning an array of
> known classes.
> {code:java}
> if(node instanceof ...){
> return handler.call((...) node);
> } else if(node instanceof ...) {
> return handler.call((...) node);
> }
> {code}
> If janino compiler dispatch was changed to use instanceof, it would remove
> the requirement know all relnode subtypes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4546) Change metadata dispatch to avoid registration of all RelNode subtypes
[
https://issues.apache.org/jira/browse/CALCITE-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-4546.
--
Resolution: Fixed
Fixed in
[fbbdf46|https://github.com/apache/calcite/commit/fbbdf465df46b4e6f9863d7d1dfdcb19a43f2032].
Thanks [~jamesstarr]!
> Change metadata dispatch to avoid registration of all RelNode subtypes
> --
>
> Key: CALCITE-4546
> URL: https://issues.apache.org/jira/browse/CALCITE-4546
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: James Starr
>Assignee: James Starr
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the
> last 3 results as reported by intellij. I then repeated this 3 times.
> After bench marking Janino, my prototype and using legacy java reflections, I
> found:
> | |Janino||*Prototype*||*java reflection*||
> |Average|326.444|315.556|1525.89|
> |Standard Deviation|27.12983188|14.75729575|75.8476177|
> The prototype was a static code with out caching or cycle detection. I
> latter added cycle detection and caching, but the results were with in one
> standard deviation. So I didn't not follow up further.
> I was doing the dispatch with instance of instead of scanning an array of
> known classes.
> {code:java}
> if(node instanceof ...){
> return handler.call((...) node);
> } else if(node instanceof ...) {
> return handler.call((...) node);
> }
> {code}
> If janino compiler dispatch was changed to use instanceof, it would remove
> the requirement know all relnode subtypes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4819) SemiJoin operator is not skipped in materialized view-based rewriting algorithm
[ https://issues.apache.org/jira/browse/CALCITE-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-4819. -- Fix Version/s: 1.28.0 Resolution: Fixed Fixed in [4704745|https://github.com/apache/calcite/commit/4704745153f8f9c3bdcfdd25201b7192e4982b81]. > SemiJoin operator is not skipped in materialized view-based rewriting > algorithm > --- > > Key: CALCITE-4819 > URL: https://issues.apache.org/jira/browse/CALCITE-4819 > Project: Calcite > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 10m > Remaining Estimate: 0h > > This had been solved in CALCITE-2277 but apparently it was broken in > CALCITE-2969 (MaterializedViewRelOptRulesTest.testJoinMaterialization11 > result was modified assuming the change was correct). The rewriting algorithm > does not support semijoin so this can lead to incorrect results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-4546) Change metadata dispatch to avoid registration of all RelNode subtypes
[
https://issues.apache.org/jira/browse/CALCITE-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned CALCITE-4546:
Assignee: James Starr
> Change metadata dispatch to avoid registration of all RelNode subtypes
> --
>
> Key: CALCITE-4546
> URL: https://issues.apache.org/jira/browse/CALCITE-4546
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: James Starr
>Assignee: James Starr
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the
> last 3 results as reported by intellij. I then repeated this 3 times.
> After bench marking Janino, my prototype and using legacy java reflections, I
> found:
> | |Janino||*Prototype*||*java reflection*||
> |Average|326.444|315.556|1525.89|
> |Standard Deviation|27.12983188|14.75729575|75.8476177|
> The prototype was a static code with out caching or cycle detection. I
> latter added cycle detection and caching, but the results were with in one
> standard deviation. So I didn't not follow up further.
> I was doing the dispatch with instance of instead of scanning an array of
> known classes.
> {code:java}
> if(node instanceof ...){
> return handler.call((...) node);
> } else if(node instanceof ...) {
> return handler.call((...) node);
> }
> {code}
> If janino compiler dispatch was changed to use instanceof, it would remove
> the requirement know all relnode subtypes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4546) Change metadata dispatch to avoid registration of all RelNode subtypes
[
https://issues.apache.org/jira/browse/CALCITE-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-4546:
-
Summary: Change metadata dispatch to avoid registration of all RelNode
subtypes (was: Change metadata dispatch to avoid all RelNode subtypes
registration)
> Change metadata dispatch to avoid registration of all RelNode subtypes
> --
>
> Key: CALCITE-4546
> URL: https://issues.apache.org/jira/browse/CALCITE-4546
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: James Starr
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the
> last 3 results as reported by intellij. I then repeated this 3 times.
> After bench marking Janino, my prototype and using legacy java reflections, I
> found:
> | |Janino||*Prototype*||*java reflection*||
> |Average|326.444|315.556|1525.89|
> |Standard Deviation|27.12983188|14.75729575|75.8476177|
> The prototype was a static code with out caching or cycle detection. I
> latter added cycle detection and caching, but the results were with in one
> standard deviation. So I didn't not follow up further.
> I was doing the dispatch with instance of instead of scanning an array of
> known classes.
> {code:java}
> if(node instanceof ...){
> return handler.call((...) node);
> } else if(node instanceof ...) {
> return handler.call((...) node);
> }
> {code}
> If janino compiler dispatch was changed to use instanceof, it would remove
> the requirement know all relnode subtypes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4546) Change metadata dispatch to avoid all RelNode subtypes registration
[
https://issues.apache.org/jira/browse/CALCITE-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-4546:
-
Summary: Change metadata dispatch to avoid all RelNode subtypes
registration (was: No Longer require RelNode subtype registration for Metadata)
> Change metadata dispatch to avoid all RelNode subtypes registration
> ---
>
> Key: CALCITE-4546
> URL: https://issues.apache.org/jira/browse/CALCITE-4546
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: James Starr
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the
> last 3 results as reported by intellij. I then repeated this 3 times.
> After bench marking Janino, my prototype and using legacy java reflections, I
> found:
> | |Janino||*Prototype*||*java reflection*||
> |Average|326.444|315.556|1525.89|
> |Standard Deviation|27.12983188|14.75729575|75.8476177|
> The prototype was a static code with out caching or cycle detection. I
> latter added cycle detection and caching, but the results were with in one
> standard deviation. So I didn't not follow up further.
> I was doing the dispatch with instance of instead of scanning an array of
> known classes.
> {code:java}
> if(node instanceof ...){
> return handler.call((...) node);
> } else if(node instanceof ...) {
> return handler.call((...) node);
> }
> {code}
> If janino compiler dispatch was changed to use instanceof, it would remove
> the requirement know all relnode subtypes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4546) No Longer require RelNode subtype registration for Metadata
[
https://issues.apache.org/jira/browse/CALCITE-4546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-4546:
-
Description:
Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the last
3 results as reported by intellij. I then repeated this 3 times.
After bench marking Janino, my prototype and using legacy java reflections, I
found:
| |Janino||*Prototype*||*java reflection*||
|Average|326.444|315.556|1525.89|
|Standard Deviation|27.12983188|14.75729575|75.8476177|
The prototype was a static code with out caching or cycle detection. I latter
added cycle detection and caching, but the results were with in one standard
deviation. So I didn't not follow up further.
I was doing the dispatch with instance of instead of scanning an array of known
classes.
{code:java}
if(node instanceof ...){
return handler.call((...) node);
} else if(node instanceof ...) {
return handler.call((...) node);
}
{code}
If janino compiler dispatch was changed to use instanceof, it would remove the
requirement know all relnode subtypes.
was:
Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the last
3 results as reported by intellij. I then repeated this 3 times.
After bench marking Janio, my prototype and using legacy java reflections, I
found:
| |Janio||*Prototype*||*java reflection*||
|Average|326.444|315.556|1525.89|
|Standard Deviation|27.12983188|14.75729575|75.8476177|
The prototype was a static code with out caching or cycle detection. I latter
added cycle detection and caching, but the results were with in one standard
deviation. So I didn't not follow up further.
I was doing the dispatch with instance of instead of scanning an array of known
classes.
{code:java}
if(node instanceof ...){
return handler.call((...) node);
} else if(node instanceof ...) {
return handler.call((...) node);
}
{code}
If janino compiler dispatch was changed to use instanceof, it would remove the
requirement know all relnode subtypes.
> No Longer require RelNode subtype registration for Metadata
> ---
>
> Key: CALCITE-4546
> URL: https://issues.apache.org/jira/browse/CALCITE-4546
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: James Starr
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Using JdbcTest.testJoinFiveWay, I ran the query six times then I took the
> last 3 results as reported by intellij. I then repeated this 3 times.
> After bench marking Janino, my prototype and using legacy java reflections, I
> found:
> | |Janino||*Prototype*||*java reflection*||
> |Average|326.444|315.556|1525.89|
> |Standard Deviation|27.12983188|14.75729575|75.8476177|
> The prototype was a static code with out caching or cycle detection. I
> latter added cycle detection and caching, but the results were with in one
> standard deviation. So I didn't not follow up further.
> I was doing the dispatch with instance of instead of scanning an array of
> known classes.
> {code:java}
> if(node instanceof ...){
> return handler.call((...) node);
> } else if(node instanceof ...) {
> return handler.call((...) node);
> }
> {code}
> If janino compiler dispatch was changed to use instanceof, it would remove
> the requirement know all relnode subtypes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-4819) SemiJoin operator is not skipped in materialized view-based rewriting algorithm
[ https://issues.apache.org/jira/browse/CALCITE-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated CALCITE-4819: - Description: This had been solved in CALCITE-2277 but apparently it was broken in CALCITE-2969 (MaterializedViewRelOptRulesTest.testJoinMaterialization11 result was modified assuming the change was correct). The rewriting algorithm does not support semijoin so this can lead to incorrect results. was: This had been solved in CALCITE-2277 but apparently it was broken in CALCITE-2696 (MaterializedViewRelOptRulesTest.testJoinMaterialization11 result was modified assuming the change was correct). The rewriting algorithm does not support semijoin so this can lead to incorrect results. > SemiJoin operator is not skipped in materialized view-based rewriting > algorithm > --- > > Key: CALCITE-4819 > URL: https://issues.apache.org/jira/browse/CALCITE-4819 > Project: Calcite > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > This had been solved in CALCITE-2277 but apparently it was broken in > CALCITE-2969 (MaterializedViewRelOptRulesTest.testJoinMaterialization11 > result was modified assuming the change was correct). The rewriting algorithm > does not support semijoin so this can lead to incorrect results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-4819) SemiJoin operator is not skipped in materialized view-based rewriting algorithm
Jesus Camacho Rodriguez created CALCITE-4819: Summary: SemiJoin operator is not skipped in materialized view-based rewriting algorithm Key: CALCITE-4819 URL: https://issues.apache.org/jira/browse/CALCITE-4819 Project: Calcite Issue Type: Bug Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez This had been solved in CALCITE-2277 but apparently it was broken in CALCITE-2696 (MaterializedViewRelOptRulesTest.testJoinMaterialization11 result was modified assuming the change was correct). The rewriting algorithm does not support semijoin so this can lead to incorrect results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3409) Add a method in RelOptMaterializations to allow registering UnifyRule
[
https://issues.apache.org/jira/browse/CALCITE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3409.
--
Fix Version/s: 1.28.0
Resolution: Fixed
Fixed in
[56d5dda|https://github.com/apache/calcite/commit/56d5ddab9e54e15473df3b3dc41205103be11e6e].
Thanks [~jinxing6...@126.com]!
> Add a method in RelOptMaterializations to allow registering UnifyRule
> -
>
> Key: CALCITE-3409
> URL: https://issues.apache.org/jira/browse/CALCITE-3409
> Project: Calcite
> Issue Type: Improvement
>Reporter: Jin Xing
>Assignee: Jin Xing
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.28.0
>
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> In current code of MaterializedViewSubstitutionVisitor, all matching rules
> are internal defined. The existing rules support the most popular scenarios.
> But my customers sometimes ask for the ability to self define some matching
> rules, thus to support some special scenarios.
> I take below example as an illustration:
> {code:java}
> Query:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd hh") >= "20190909 00"
> and from_unixtime(_EVENT_TIME_, "mmdd hh") <= "20190909 23" ;
> Materialized View:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd") = "20190909";{code}
> It's hard to enumerate the matching pattern for different functions in
> internal matching rules. We can expose a method to register new UnifyRules
> and allow user to extend the ability of MV matching
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3409) Add a method in RelOptMaterializations to allow registering UnifyRule
[
https://issues.apache.org/jira/browse/CALCITE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3409:
-
Summary: Add a method in RelOptMaterializations to allow registering
UnifyRule (was: Add an interface in RelOptMaterializations to allow
registering UnifyRule)
> Add a method in RelOptMaterializations to allow registering UnifyRule
> -
>
> Key: CALCITE-3409
> URL: https://issues.apache.org/jira/browse/CALCITE-3409
> Project: Calcite
> Issue Type: Improvement
>Reporter: Jin Xing
>Assignee: Jin Xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h
> Remaining Estimate: 0h
>
> In current code of MaterializedViewSubstitutionVisitor, all matching rules
> are internal defined. The existing rules support the most popular scenarios.
> But my customers sometimes ask for the ability to self define some matching
> rules, thus to support some special scenarios.
> I take below example as an illustration:
> {code:java}
> Query:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd hh") >= "20190909 00"
> and from_unixtime(_EVENT_TIME_, "mmdd hh") <= "20190909 23" ;
> Materialized View:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd") = "20190909";{code}
> It's hard to enumerate the matching pattern for different functions in
> internal matching rules. We can expose a method to register new UnifyRules
> and allow user to extend the ability of MV matching
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3409) Add an interface in RelOptMaterializations to allow registering UnifyRule
[
https://issues.apache.org/jira/browse/CALCITE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3409:
-
Summary: Add an interface in RelOptMaterializations to allow registering
UnifyRule (was: Add an interface in MaterializedViewSubstitutionVisitor to
allow registering UnifyRule)
> Add an interface in RelOptMaterializations to allow registering UnifyRule
> -
>
> Key: CALCITE-3409
> URL: https://issues.apache.org/jira/browse/CALCITE-3409
> Project: Calcite
> Issue Type: Improvement
>Reporter: Jin Xing
>Assignee: Jin Xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h
> Remaining Estimate: 0h
>
> In current code of MaterializedViewSubstitutionVisitor, all matching rules
> are internal defined. The existing rules support the most popular scenarios.
> But my customers sometimes ask for the ability to self define some matching
> rules, thus to support some special scenarios.
> I take below example as an illustration:
> {code:java}
> Query:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd hh") >= "20190909 00"
> and from_unixtime(_EVENT_TIME_, "mmdd hh") <= "20190909 23" ;
> Materialized View:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd") = "20190909";{code}
> It's hard to enumerate the matching pattern for different functions in
> internal matching rules. We can expose a method to register new UnifyRules
> and allow user to extend the ability of MV matching
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4544) Deprecate Metadata API backed by Java Reflection
[ https://issues.apache.org/jira/browse/CALCITE-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-4544. -- Fix Version/s: 1.28.0 Resolution: Fixed Fixed in [b51a2e1|https://github.com/apache/calcite/commit/b51a2e14343df089ee0b6c3e8a20c00c51186421]. Thanks [~jamesstarr]! > Deprecate Metadata API backed by Java Reflection > > > Key: CALCITE-4544 > URL: https://issues.apache.org/jira/browse/CALCITE-4544 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Fix For: 1.28.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > A large portion of the metadata api is functionally deprecated, we should > make it explicit. > Apache Calcite currently has 2 ways deriving metadata about rel node, one > through runtime generated handler classes that are compiled using Janino, and > the other using java reflection. The former will be referred to as Janino > and the latter as java reflection for brevity. Java reflection is not used > due to performance reason in calcite main, nor is it tested, and using it can > lead to degraded performance due to interactions with caching of the janino > handlers. Both share a common way of registering classes with implementation > by using ReflectiveRelMetadataProvider and ChainedRelMetadataProvider. There > are also 2 different subsystems for supporting metadata caching, one where a > reference to a cache is stored in the metadata query and the other where > caching is a wrapper pattern around the providers. There are also 2 separate > ways for invalidating the caching, one with a counter stored in the planner > used by java reflection api, and another where volcano planner tracks the > parents of all rel nodes that it mutates and then explicitly invalidates > those parents used by the janino api. > So the following could be removed without the lose of any functionality: > * CachingRelMetadataProvider - used for java reflection base metadata caching > * RelOptPlanner.getRelMetadataTimestamp - used by java reflection to > invalidate the metadata cache > * RelOptPlanner.registerMetadataProviders(List list) - > used by java reflection API to support of Hep and Volcano nodes. > * MetadataFactory - used to caching an internal artifact of the > implementation of java reflection API > * HepRelMetadataProvider - used by java reflection to support Hep node > delegation to a child node. > * VolcanoRelMetadataProvider - used by java reflection to support RelSubset > node delegation to a child node. > * AbstractRelNode.metadata - i think was an even older api that now bridges > to the java reflection api > * RelNode.metadata - see above > * MetadataDef.metadtaClass - used by java reflection and for a unique > descriptive name of the janino generated handler. Janino dependency on it is > easily broken. > * ReflectiveRelMetadataProvider. map and metadataClass0 - used by java > reflection implementation > * RelMetadataProvider.apply() - most of the actual implementation java > reflection API > Metadata and MetadataDef could also be removed, however removing it would > require reworking portions of janino code do so. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4544) Deprecate Metadata API backed by Java Reflection
[ https://issues.apache.org/jira/browse/CALCITE-4544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17417827#comment-17417827 ] Jesus Camacho Rodriguez commented on CALCITE-4544: -- [~jamesstarr], I'll help reviewing this patch and others related to CALCITE-4539. I took a look at your PR and left a few minor comments. > Deprecate Metadata API backed by Java Reflection > > > Key: CALCITE-4544 > URL: https://issues.apache.org/jira/browse/CALCITE-4544 > Project: Calcite > Issue Type: Improvement >Reporter: James Starr >Assignee: James Starr >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > A large portion of the metadata api is functionally deprecated, we should > make it explicit. > Apache Calcite currently has 2 ways deriving metadata about rel node, one > through runtime generated handler classes that are compiled using Janino, and > the other using java reflection. The former will be referred to as Janino > and the latter as java reflection for brevity. Java reflection is not used > due to performance reason in calcite main, nor is it tested, and using it can > lead to degraded performance due to interactions with caching of the janino > handlers. Both share a common way of registering classes with implementation > by using ReflectiveRelMetadataProvider and ChainedRelMetadataProvider. There > are also 2 different subsystems for supporting metadata caching, one where a > reference to a cache is stored in the metadata query and the other where > caching is a wrapper pattern around the providers. There are also 2 separate > ways for invalidating the caching, one with a counter stored in the planner > used by java reflection api, and another where volcano planner tracks the > parents of all rel nodes that it mutates and then explicitly invalidates > those parents used by the janino api. > So the following could be removed without the lose of any functionality: > * CachingRelMetadataProvider - used for java reflection base metadata caching > * RelOptPlanner.getRelMetadataTimestamp - used by java reflection to > invalidate the metadata cache > * RelOptPlanner.registerMetadataProviders(List list) - > used by java reflection API to support of Hep and Volcano nodes. > * MetadataFactory - used to caching an internal artifact of the > implementation of java reflection API > * HepRelMetadataProvider - used by java reflection to support Hep node > delegation to a child node. > * VolcanoRelMetadataProvider - used by java reflection to support RelSubset > node delegation to a child node. > * AbstractRelNode.metadata - i think was an even older api that now bridges > to the java reflection api > * RelNode.metadata - see above > * MetadataDef.metadtaClass - used by java reflection and for a unique > descriptive name of the janino generated handler. Janino dependency on it is > easily broken. > * ReflectiveRelMetadataProvider. map and metadataClass0 - used by java > reflection implementation > * RelMetadataProvider.apply() - most of the actual implementation java > reflection API > Metadata and MetadataDef could also be removed, however removing it would > require reworking portions of janino code do so. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3409) Add an interface in MaterializedViewSubstitutionVisitor to allow registering UnifyRule
[
https://issues.apache.org/jira/browse/CALCITE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17417703#comment-17417703
]
Jesus Camacho Rodriguez commented on CALCITE-3409:
--
[~xzh_dz], following up on [~zabetak]'s feedback above. Until now, we had not
exposed the {{UnifyRules}} in the planner and they were rather restricted to
the {{SubstitutionVisitor}}. It seems you do not need them to be in the planner
either, instead it would be sufficient for you to have a
{{SubstitutionVisitor}} that is extensible. Maybe we could limit this patch to
the changes in the visitor, and avoid changing the {{RelOptPlanner}} APIs for
the time being. If it turns out that extensibility from the planner is needed,
we can revisit the idea later on. Thoughts?
> Add an interface in MaterializedViewSubstitutionVisitor to allow registering
> UnifyRule
> --
>
> Key: CALCITE-3409
> URL: https://issues.apache.org/jira/browse/CALCITE-3409
> Project: Calcite
> Issue Type: Improvement
>Reporter: Jin Xing
>Assignee: Jin Xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> In current code of MaterializedViewSubstitutionVisitor, all matching rules
> are internal defined. The existing rules support the most popular scenarios.
> But my customers sometimes ask for the ability to self define some matching
> rules, thus to support some special scenarios.
> I take below example as an illustration:
> {code:java}
> Query:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd hh") >= "20190909 00"
> and from_unixtime(_EVENT_TIME_, "mmdd hh") <= "20190909 23" ;
> Materialized View:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd") = "20190909";{code}
> It's hard to enumerate the matching pattern for different functions in
> internal matching rules. We can expose a method to register new UnifyRules
> and allow user to extend the ability of MV matching
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4716) ClassCastException converting SARG in RelNode to SQL
[
https://issues.apache.org/jira/browse/CALCITE-4716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-4716.
--
Fix Version/s: 1.28.0
Resolution: Fixed
Thanks for the feedback [~julianhyde].
Fixed in
[53ee1baa7|https://github.com/apache/calcite/commit/53ee1baa7b2415d38d1f2566730ccf03b8b17b3a].
> ClassCastException converting SARG in RelNode to SQL
>
>
> Key: CALCITE-4716
> URL: https://issues.apache.org/jira/browse/CALCITE-4716
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.28.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The stacktrace is the following:
> {noformat}
> class org.apache.calcite.rex.RexLocalRef cannot be cast to class
> org.apache.calcite.rex.RexLiteral (org.apache.calcite.rex.RexLocalRef and
> org.apache.calcite.rex.RexLiteral are in unnamed module of loader 'app')
> java.lang.ClassCastException: class org.apache.calcite.rex.RexLocalRef cannot
> be cast to class org.apache.calcite.rex.RexLiteral
> (org.apache.calcite.rex.RexLocalRef and org.apache.calcite.rex.RexLiteral are
> in unnamed module of loader 'app')
> at
> org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:695)
> at
> org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:597)
> ...
> {noformat}
> The relevant expressions in the Calc operator are the following:
> {code}
> ...expr#5=[Sarg[(10..11]]], expr#6=[SEARCH($t0, $t5)]...
> {code}
> The current code in {{SqlImplementor}} considers the second argument to
> SEARCH is always a RexLiteral:
> {code}
> ...
> case SEARCH:
> final RexCall search = (RexCall) rex;
> literal = (RexLiteral) search.operands.get(1);
> final Sarg sarg = castNonNull(literal.getValueAs(Sarg.class));
> //noinspection unchecked
> return toSql(program, search.operands.get(0), literal.getType(),
> sarg);
> ...
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4716) ClassCastException converting SARG in RelNode to SQL
Jesus Camacho Rodriguez created CALCITE-4716:
Summary: ClassCastException converting SARG in RelNode to SQL
Key: CALCITE-4716
URL: https://issues.apache.org/jira/browse/CALCITE-4716
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
The stacktrace is the following:
{noformat}
class org.apache.calcite.rex.RexLocalRef cannot be cast to class
org.apache.calcite.rex.RexLiteral (org.apache.calcite.rex.RexLocalRef and
org.apache.calcite.rex.RexLiteral are in unnamed module of loader 'app')
java.lang.ClassCastException: class org.apache.calcite.rex.RexLocalRef cannot
be cast to class org.apache.calcite.rex.RexLiteral
(org.apache.calcite.rex.RexLocalRef and org.apache.calcite.rex.RexLiteral are
in unnamed module of loader 'app')
at
org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:695)
at
org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:597)
...
{noformat}
The relevant expressions in the Calc operator are the following:
{code}
...expr#5=[Sarg[(10..11]]], expr#6=[SEARCH($t0, $t5)]...
{code}
The current code in {{SqlImplementor}} considers the second argument to SEARCH
is always a RexLiteral:
{code}
...
case SEARCH:
final RexCall search = (RexCall) rex;
literal = (RexLiteral) search.operands.get(1);
final Sarg sarg = castNonNull(literal.getValueAs(Sarg.class));
//noinspection unchecked
return toSql(program, search.operands.get(0), literal.getType(), sarg);
...
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4707) Optimize incremental maintenance of materialized views
[
https://issues.apache.org/jira/browse/CALCITE-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17392688#comment-17392688
]
Jesus Camacho Rodriguez commented on CALCITE-4707:
--
Thanks [~julianhyde]. Fwiw a rather similar approach to what is described above
is implemented in Hive using Calcite: It relies on the snapshot information of
the tables referenced by the materialized view to generate the diffs. It
proceeds in two steps: 1) Creating the union rewriting referenced above by
relying on Calcite's MV rewriting algorithm, and 2) Trigger a Calcite rule to
transform the union rewriting into an insert/merge statement. Slides 19-24
[here|https://www.slideshare.net/Hadoop_Summit/accelerating-query-processing-with-materialized-views-in-apache-hive]
describe it with an example. Unfortunately, the rules for step 2 are quite
specific to Hive (see .\*incremental.\* rules
[here|https://github.com/apache/hive/tree/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views])
since they generate the internal execution plan for a MERGE statement, i.e.,
what MERGE translates into in Hive (outer join + multi-insert statement) rather
than a MERGE operator. Maybe some of the code in Hive could be reused here
rather than reimplemented (it would definitely be interesting if Hive could
adopt the Calcite implementation). Cc [~kkasa], since you have been working in
this space in Hive.
> Optimize incremental maintenance of materialized views
> --
>
> Key: CALCITE-4707
> URL: https://issues.apache.org/jira/browse/CALCITE-4707
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
>
> Optimize incremental maintenance of materialized views, when we know what DML
> has occurred since the last build.
> The goal here is to develop an algebraic approach (i.e. new relational
> operators and transformation rules) that will generate an optimal query for
> maintaining the view.
> Consider a materialized view
> {code}
> CREATE TABLE EmpsByDeptno AS
> SELECT deptno, COUNT(*) AS c, SUM(sal) AS ss
> FROM Emps
> GROUP BY deptno;
> {code}
> We built it at time T1, when the value of {{Emps}} was {{Emps1}}, and it had
> the value {{EmpsByDeptno1}}.
> It is now T2, and since then, new employees have been created in the
> {{NewEmps}} table. Thus {{Emps2 = Emps1 UNION ALL NewEmps}}. We could build a
> new MV,
> {code}
> CREATE TABLE EmpsByDeptno2 AS
> SELECT deptno, COUNT(*) AS c, SUM(sal) AS ss
> FROM (SELECT * FROM Emps1
> UNION ALL
> SELECT * FROM NewEmps)
> GROUP BY deptno;
> {code}but we would prefer to generate a DML command to modify
> {{EmpsByDeptno}} in place:{code}
> MERGE INTO EmpsByDeptno AS e
> USING (SELECT deptno, COUNT(*) AS c, SUM(sal) AS ss
> FROM NewEmps) AS de
> ON de.deptno = e.deptno
> WHEN MATCHED THEN
> UPDATE SET c = c + de.c, ss = ss + de.ss
> WHEN NOT MATCHED THEN
> INSERT (deptno, c, ss)
> {code}
> We propose two new relational operators:
> * {{Diff(S, R)}} computes the difference between two relations. It has a
> same schema as R and S but with an additional column {{delta}}. Rows in S but
> not in R are called insertions, and have delta=1. Rows that are in R but not
> in S are called deletions, and have delta=-1.
> * {{Patch(R, P)}} applies a difference {{P}} to a relation. {{P}} has the
> same schema as {{R}} but with an additional delta column. The output is the
> rows of R, minus the rows in {{P}} that have delta=-1, union the rows in
> {{P}} that have delta=1.
> The relational operators are named by analogy with the UNIX utilities
> {{diff}} and {{patch}}. (But {{Diff}}'s arguments are reversed compared to
> {{diff}}.) Consider:
> {code}
> grep r /usr/share/dict/words > r.txt
> grep s /usr/share/dict/words > s.txt
> diff r.txt s.txt > p.patch
> patch -p1 r.txt < p.patch
> cmp r.txt s.txt. # r and s are now identical
> {code}
> The relation {code}R = Patch(S, Diff(R, S)){code} always holds. {{Diff}}
> computes the difference between R and S, and when {{Patch}} applies that
> difference to {{S}} you get {{R}}.
> {{Patch}} and {{Diff}} are intended by be generalizations of set operators.
> If there are only additions, i.e. {{R}} is a superset of {{S}}, then you can
> substitute {{Minus}} for {{Diff}}, and {{Union}} for {{Patch}}:
> {code}
> R = Union(S, Minus(R, S))
> {code}
> So, how do these relational operators solve our original problem?
> An {{INSERT}} statement is assignment to a relation of itself union some new
> rows. ({{INSERT INTO R SELECT * FROM P}} is {{R R UNION P}}).
> A {{MERGE}} statement is similar to {{INSERT}} but allows updates.
> The {{MERGE}} query above is represented as {code}EmpsByDeptno
> Patch(EmpsByDeptno, Diff(NewEmpsByDeptno, EmpsByDeptno)){code}
>
[jira] [Resolved] (CALCITE-4499) FilterJoinRule misses opportunity to push filter to semijoin input
[
https://issues.apache.org/jira/browse/CALCITE-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-4499.
--
Fix Version/s: 1.28.0
Resolution: Fixed
Fixed in
[de48e55|https://github.com/apache/calcite/commit/de48e55783a140e6927f88a445d9cbdf2e7623b5].
> FilterJoinRule misses opportunity to push filter to semijoin input
> --
>
> Key: CALCITE-4499
> URL: https://issues.apache.org/jira/browse/CALCITE-4499
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.28.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Consider the following plan:
> {code}
> LogicalProject(DNAME=[$1])
> LogicalJoin(condition=[AND(=($0, $10), =($8, 100))], joinType=[semi])
> LogicalTableScan(table=[[scott, DEPT]])
> LogicalTableScan(table=[[scott, EMP]])
> {code}
> The second conjunct only refers to columns from the right input, thus it
> could be pushed to the right input. However, {{FilterJoinRule}} misses this
> opportunity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4499) FilterJoinRule misses opportunity to push filter to semijoin input
Jesus Camacho Rodriguez created CALCITE-4499:
Summary: FilterJoinRule misses opportunity to push filter to
semijoin input
Key: CALCITE-4499
URL: https://issues.apache.org/jira/browse/CALCITE-4499
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Consider the following plan:
{code}
LogicalProject(DNAME=[$1])
LogicalJoin(condition=[AND(=($0, $10), =($8, 100))], joinType=[semi])
LogicalTableScan(table=[[scott, DEPT]])
LogicalTableScan(table=[[scott, EMP]])
{code}
The second conjunct only refers to columns from the right input, thus it could
be pushed to the right input. However, {{FilterJoinRule}} misses this
opportunity.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-4467) Incorrect simplification for 'NaN' value
[
https://issues.apache.org/jira/browse/CALCITE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-4467.
--
Resolution: Won't Fix
> Incorrect simplification for 'NaN' value
>
>
> Key: CALCITE-4467
> URL: https://issues.apache.org/jira/browse/CALCITE-4467
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly
> <= and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
> This may not be applicable in some cases. For instance, if the type of x is
> floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
> (e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
> standard. For the latest, the rewriting above will result in incorrect
> results.
> I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4467) Incorrect simplification for 'NaN' value
[
https://issues.apache.org/jira/browse/CALCITE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265541#comment-17265541
]
Jesus Camacho Rodriguez commented on CALCITE-4467:
--
{quote}
whether NaN values should be grouped in GROUP BY
{quote}
Yes, that's a very good point. Impala faced that issue and they decided to
stick to {{NaN != NaN}} but group NaN values together:
https://issues.apache.org/jira/browse/IMPALA-6661
> Incorrect simplification for 'NaN' value
>
>
> Key: CALCITE-4467
> URL: https://issues.apache.org/jira/browse/CALCITE-4467
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly
> <= and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
> This may not be applicable in some cases. For instance, if the type of x is
> floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
> (e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
> standard. For the latest, the rewriting above will result in incorrect
> results.
> I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-4467) Incorrect simplification for 'NaN' value
[
https://issues.apache.org/jira/browse/CALCITE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265428#comment-17265428
]
Jesus Camacho Rodriguez edited comment on CALCITE-4467 at 1/14/21, 11:07 PM:
-
I don't advocate for changing Calcite's default semantics. However, different
engines seem to implement this differently. Thus, the main idea behind this
JIRA was to explore whether it would make sense to skip the simplification in
Calcite and leave the evaluation of such expressions to the specific engine
implementation. I also understand that even if we decided to do that, the
current PR should be extended to cover all cases in which current
simplifications would not be correct anymore in presence of 'NaN', which is not
trivial.
If we assume that 'NaN' is not a possible value in Calcite, then we may close
this JIRA as not an issue.
was (Author: jcamachorodriguez):
I don't advocate for changing Calcite's default semantics. However, different
engines seem to be implement this differently. Thus, the main idea behind this
JIRA was to explore whether it would make sense to skip the simplification in
Calcite and leave the evaluation of such expressions to the specific engine
implementation. I also understand that even if we decided to do that, the
current PR should be extended to cover all cases in which current
simplifications would not be correct anymore in presence of 'NaN', which is not
trivial.
If we assume that 'NaN' is not a possible value in Calcite, then we may close
this JIRA as not an issue.
> Incorrect simplification for 'NaN' value
>
>
> Key: CALCITE-4467
> URL: https://issues.apache.org/jira/browse/CALCITE-4467
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly
> <= and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
> This may not be applicable in some cases. For instance, if the type of x is
> floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
> (e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
> standard. For the latest, the rewriting above will result in incorrect
> results.
> I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4467) Incorrect simplification for 'NaN' value
[
https://issues.apache.org/jira/browse/CALCITE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17265428#comment-17265428
]
Jesus Camacho Rodriguez commented on CALCITE-4467:
--
I don't advocate for changing Calcite's default semantics. However, different
engines seem to be implement this differently. Thus, the main idea behind this
JIRA was to explore whether it would make sense to skip the simplification in
Calcite and leave the evaluation of such expressions to the specific engine
implementation. I also understand that even if we decided to do that, the
current PR should be extended to cover all cases in which current
simplifications would not be correct anymore in presence of 'NaN', which is not
trivial.
If we assume that 'NaN' is not a possible value in Calcite, then we may close
this JIRA as not an issue.
> Incorrect simplification for 'NaN' value
>
>
> Key: CALCITE-4467
> URL: https://issues.apache.org/jira/browse/CALCITE-4467
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly
> <= and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
> This may not be applicable in some cases. For instance, if the type of x is
> floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
> (e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
> standard. For the latest, the rewriting above will result in incorrect
> results.
> I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4467) Incorrect simplification for 'NaN' value
[
https://issues.apache.org/jira/browse/CALCITE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264950#comment-17264950
]
Jesus Camacho Rodriguez commented on CALCITE-4467:
--
{quote}
It will not be easy to remove all simplifications concerning equality.
{quote}
[~julianhyde], are there other simplifications that come to your mind? Tbh I
did not take an exhaustive look but I assumed Calcite would not do folding and
instead relies on specific executor implementation (which would then enforce
the correct implementation-specific semantics), and that probably limits the
number of cases on which we will hit this issue.
{quote}
We could introduce a configuration property; people who strictly follow the
standard can benefit from the simplifications and those who don't can still run
the simplifier.
{quote}
In this specific case, I hit the issue because the RelBuilder calls
{{simplify}} when a Project is created. That's already configurable but I did
not want to disable all simplifications when a Project is created.
I could introduce some configuration variable to control the flow but I am not
sure if it is worth it if this would be the only simplification that would be
disabled? I don't have a strong opinion, I'd like to hear your ideas.
> Incorrect simplification for 'NaN' value
>
>
> Key: CALCITE-4467
> URL: https://issues.apache.org/jira/browse/CALCITE-4467
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly
> <= and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
> This may not be applicable in some cases. For instance, if the type of x is
> floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
> (e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
> standard. For the latest, the rewriting above will result in incorrect
> results.
> I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4467) Incorrect simplification for 'NaN' value
[
https://issues.apache.org/jira/browse/CALCITE-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264582#comment-17264582
]
Jesus Camacho Rodriguez commented on CALCITE-4467:
--
Fwiw, from PG documentation:
{quote}
In order to allow numeric values to be sorted and used in tree-based indexes,
PostgreSQL treats NaN values as equal, and greater than all non-NaN values.
{quote}
> Incorrect simplification for 'NaN' value
>
>
> Key: CALCITE-4467
> URL: https://issues.apache.org/jira/browse/CALCITE-4467
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> {{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly
> <= and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
> This may not be applicable in some cases. For instance, if the type of x is
> floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
> (e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
> standard. For the latest, the rewriting above will result in incorrect
> results.
> I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4467) Incorrect simplification for 'NaN' value
Jesus Camacho Rodriguez created CALCITE-4467:
Summary: Incorrect simplification for 'NaN' value
Key: CALCITE-4467
URL: https://issues.apache.org/jira/browse/CALCITE-4467
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
{{RexSimplify}} simplifies {{x = x}} to {{null or x is not null}} (similarly <=
and >=), and {{x != x}} to {{null and x is null}} (similarly < and >).
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexSimplify.java#L363
This may not be applicable in some cases. For instance, if the type of x is
floating-point, x could be 'NaN'. While some RDBMS consider 'NaN' = 'NaN'
(e.g., Postgres), some others consider 'NaN' != 'NaN' following the IEEE 754
standard. For the latest, the rewriting above will result in incorrect results.
I think we should simply ignore this simplification for floating-point type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3409) Add an interface in MaterializedViewSubstitutionVisitor to allow registering UnifyRule
[
https://issues.apache.org/jira/browse/CALCITE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224883#comment-17224883
]
Jesus Camacho Rodriguez commented on CALCITE-3409:
--
[~julianhyde], thanks for pinging me. I have not used the unification rules
that heavily, and the change does not seem to affect other materialization
rules. I have left a couple of comments in the PR though.
> Add an interface in MaterializedViewSubstitutionVisitor to allow registering
> UnifyRule
> --
>
> Key: CALCITE-3409
> URL: https://issues.apache.org/jira/browse/CALCITE-3409
> Project: Calcite
> Issue Type: Improvement
>Reporter: Jin Xing
>Assignee: Jin Xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> In current code of MaterializedViewSubstitutionVisitor, all matching rules
> are internal defined. The existing rules support the most popular scenarios.
> But my customers sometimes ask for the ability to self define some matching
> rules, thus to support some special scenarios.
> I take below example as an illustration:
> {code:java}
> Query:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd hh") >= "20190909 00"
> and from_unixtime(_EVENT_TIME_, "mmdd hh") <= "20190909 23" ;
> Materialized View:
> select * from table
> where from_unixtime(_EVENT_TIME_, "mmdd") = "20190909";{code}
> It's hard to enumerate the matching pattern for different functions in
> internal matching rules. We can expose a method to register new UnifyRules
> and allow user to extend the ability of MV matching
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4253) RelOptUtil#findAllTables should probably use RelMetadataQuery#getTableReferences
[
https://issues.apache.org/jira/browse/CALCITE-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195654#comment-17195654
]
Jesus Camacho Rodriguez commented on CALCITE-4253:
--
Thanks for pinging me [~julianhyde]. It's been some time but I believe
{{RelOptUtil.findAllTables}} relying on {{RelMetadataQuery}} was simply to have
a single code path, which may not have been the right choice based on your
comment above. As for not using {{getTableReferences}}, maybe it was introduced
after CALCITE-1682? Automatic caching (since the rule may make the same call
repeatedly) and extensibility were the main factors to rely on the metadata
providers in the MV rewriting rules indeed.
> RelOptUtil#findAllTables should probably use
> RelMetadataQuery#getTableReferences
>
>
> Key: CALCITE-4253
> URL: https://issues.apache.org/jira/browse/CALCITE-4253
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Affects Versions: 1.25.0
>Reporter: Vladimir Sitnikov
>Priority: Minor
>
> It looks like both methods do exactly the same thing, so it would probably
> make sense to divert {{findAllTables}} to
> {{RelMetadataQuery#getTableReferences}}
> If methods are different, it would make sense to add the relevant
> documentation and cross-references.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3999) Simplify DialectPool implementation using Guava cache
[
https://issues.apache.org/jira/browse/CALCITE-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108395#comment-17108395
]
Jesus Camacho Rodriguez edited comment on CALCITE-3999 at 5/15/20, 3:40 PM:
Thanks for looking in detail into this [~zabetak].
{quote}
Moreover, most of the methods in DatabaseMetaData just return some hardcoded
values so there are no real queries to the database.
{quote}
I checked a few drivers, and while this is true for most of the calls that we
make to create the dialect, I realized that {{getDatabaseProductVersion}} may
still emit a query to the RDBMS in multiple implementations.
>From the top of my head, a few scenarios where I guess the cache may be useful:
- Multitenant deployments with many connections to same RDBMS.
- Reduce latency under network congestion.
- Reduce compilation latency variability (memory vs network access).
What do you think about these? There is probably a trade-off here, but I think
it is not straightforward to simulate all the possible different scenarios.
was (Author: jcamachorodriguez):
Thanks for looking in detail into this [~zabetak].
>From the top of my head, a few scenarios where I guess the cache may be useful:
- Multitenant deployments with many connections to same RDBMS.
- Reduce latency under network congestion.
- Reduce compilation latency variability (memory vs network access).
What do you think about these? There is probably a trade-off here, but I think
it is not straightforward to simulate all the possible different scenarios.
> Simplify DialectPool implementation using Guava cache
> -
>
> Key: CALCITE-3999
> URL: https://issues.apache.org/jira/browse/CALCITE-3999
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> JdbcUtils contains a pool to cache SqlDialect objects. Currently, it relies
> on multiple maps and a synchronized {{get}} method. Although I am not very
> familiar with that code, it seems the implementation could be made simpler
> and more efficient by using a Guava cache. In addition, since we would not
> have a single synchronized get method, multiple threads could concurrently
> create dialects for distinct data sources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3999) Simplify DialectPool implementation using Guava cache
[
https://issues.apache.org/jira/browse/CALCITE-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17108395#comment-17108395
]
Jesus Camacho Rodriguez commented on CALCITE-3999:
--
Thanks for looking in detail into this [~zabetak].
>From the top of my head, a few scenarios where I guess the cache may be useful:
- Multitenant deployments with many connections to same RDBMS.
- Reduce latency under network congestion.
- Reduce compilation latency variability (memory vs network access).
What do you think about these? There is probably a trade-off here, but I think
it is not straightforward to simulate all the possible different scenarios.
> Simplify DialectPool implementation using Guava cache
> -
>
> Key: CALCITE-3999
> URL: https://issues.apache.org/jira/browse/CALCITE-3999
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> JdbcUtils contains a pool to cache SqlDialect objects. Currently, it relies
> on multiple maps and a synchronized {{get}} method. Although I am not very
> familiar with that code, it seems the implementation could be made simpler
> and more efficient by using a Guava cache. In addition, since we would not
> have a single synchronized get method, multiple threads could concurrently
> create dialects for distinct data sources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3999) Simplify DialectPool implementation using Guava cache
[
https://issues.apache.org/jira/browse/CALCITE-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106808#comment-17106808
]
Jesus Camacho Rodriguez commented on CALCITE-3999:
--
I have created a PR in https://github.com/apache/calcite/pull/1977/.
> Simplify DialectPool implementation using Guava cache
> -
>
> Key: CALCITE-3999
> URL: https://issues.apache.org/jira/browse/CALCITE-3999
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> JdbcUtils contains a pool to cache SqlDialect objects. Currently, it relies
> on multiple maps and a synchronized {{get}} method. Although I am not very
> familiar with that code, it seems the implementation could be made simpler
> and more efficient by using a Guava cache. In addition, since we would not
> have a single synchronized get method, multiple threads could concurrently
> create dialects for distinct data sources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3999) Simplify DialectPool implementation using Guava cache
[
https://issues.apache.org/jira/browse/CALCITE-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3999:
-
Summary: Simplify DialectPool implementation using Guava cache (was:
Simplify DialectPool implementation)
> Simplify DialectPool implementation using Guava cache
> -
>
> Key: CALCITE-3999
> URL: https://issues.apache.org/jira/browse/CALCITE-3999
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> JdbcUtils contains a pool to cache SqlDialect objects. Currently, it relies
> on multiple maps and a synchronized {{get}} method. Although I am not very
> familiar with that code, it seems the implementation could be made simpler
> and more efficient by using a Guava cache. In addition, since we would not
> have a single synchronized get method, multiple threads could concurrently
> create dialects for distinct data sources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3999) Simplify DialectPool implementation
Jesus Camacho Rodriguez created CALCITE-3999:
Summary: Simplify DialectPool implementation
Key: CALCITE-3999
URL: https://issues.apache.org/jira/browse/CALCITE-3999
Project: Calcite
Issue Type: Improvement
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
JdbcUtils contains a pool to cache SqlDialect objects. Currently, it relies on
multiple maps and a synchronized {{get}} method. Although I am not very
familiar with that code, it seems the implementation could be made simpler and
more efficient by using a Guava cache. In addition, since we would not have a
single synchronized get method, multiple threads could concurrently create
dialects for distinct data sources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3982) Simplify FilterMergeRule to rely on RelBuilder instead of RexProgram
[
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3982.
--
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
[de3bef9b06502886f87b60ab36d8e54ebc55f05c|https://github.com/apache/calcite/commit/de3bef9b06502886f87b60ab36d8e54ebc55f05c].
Thanks [~julianhyde] and [~zabetak] for the feedback.
> Simplify FilterMergeRule to rely on RelBuilder instead of RexProgram
>
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 1.23.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This could potentially happen since Filter creation has a check on whether
> the expression is flat
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
> and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
> at org.apache.calcite.rel.core.Filter.(Filter.java:74)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
> at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
> at
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3982) Simplify FilterMergeRule to rely on RelBuilder instead of RexProgram
[
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3982:
-
Summary: Simplify FilterMergeRule to rely on RelBuilder instead of
RexProgram (was: FilterMergeRule can lead to AssertionError)
> Simplify FilterMergeRule to rely on RelBuilder instead of RexProgram
>
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This could potentially happen since Filter creation has a check on whether
> the expression is flat
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
> and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
> at org.apache.calcite.rel.core.Filter.(Filter.java:74)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
> at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
> at
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3982) FilterMergeRule can lead to AssertionError
[
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103589#comment-17103589
]
Jesus Camacho Rodriguez commented on CALCITE-3982:
--
It seems this issue appears in Hive because we override some of the methods in
the builder to avoid calling expression simplification when we create a filter.
I have not been able to reproduce the issue in Calcite. However, it seems to me
that using the RexProgram in FilterMergeRule is unnecessary in any case
([~zabetak] pointed this out in the Hive PR)? I have created a patch that
removes the usage of the program and relies solely on the builder, and I could
not find any test regression: https://github.com/apache/calcite/pull/1968 .
> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This could potentially happen since Filter creation has a check on whether
> the expression is flat
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
> and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
> at org.apache.calcite.rel.core.Filter.(Filter.java:74)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
> at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
> at
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3982) FilterMergeRule can lead to AssertionError
[
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned CALCITE-3982:
Assignee: Jesus Camacho Rodriguez
> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This could potentially happen since Filter creation has a check on whether
> the expression is flat
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
> and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
> at org.apache.calcite.rel.core.Filter.(Filter.java:74)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
> at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
> at
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3982) FilterMergeRule can lead to AssertionError
[
https://issues.apache.org/jira/browse/CALCITE-3982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102973#comment-17102973
]
Jesus Camacho Rodriguez commented on CALCITE-3982:
--
FilterMergeRule [relies on the
builder|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/rules/FilterMergeRule.java#L82],
however it seems it only flattens the top expression. I think one possible fix
is to create a variant of ExpansionShuttle that flattens the expression as it
is being expanded.
> FilterMergeRule can lead to AssertionError
> --
>
> Key: CALCITE-3982
> URL: https://issues.apache.org/jira/browse/CALCITE-3982
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> This could potentially happen since Filter creation has a check on whether
> the expression is flat
> ([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
> and Filter merge does not flatten an expression when it is created.
> {noformat}
> java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT
> NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
> at org.apache.calcite.rel.core.Filter.(Filter.java:74)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
> at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
> at
> org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3982) FilterMergeRule can lead to AssertionError
Jesus Camacho Rodriguez created CALCITE-3982:
Summary: FilterMergeRule can lead to AssertionError
Key: CALCITE-3982
URL: https://issues.apache.org/jira/browse/CALCITE-3982
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
This could potentially happen since Filter creation has a check on whether the
expression is flat
([here|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/Filter.java#L74])
and Filter merge does not flatten an expression when it is created.
{noformat}
java.lang.AssertionError: AND(=($3, 100), OR(OR(null, IS NOT
NULL(CAST(100):INTEGER)), =(CAST(100):INTEGER, CAST(200):INTEGER)))
at org.apache.calcite.rel.core.Filter.(Filter.java:74)
at
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter.(HiveFilter.java:39)
at
org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories$HiveFilterFactoryImpl.createFilter(HiveRelFactories.java:126)
at
org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelBuilder.filter(HiveRelBuilder.java:99)
at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1055)
at
org.apache.calcite.rel.rules.FilterMergeRule.onMatch(FilterMergeRule.java:81)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3149) CACHE in RelDataTypeFactoryImpl can't be garbage collected
[
https://issues.apache.org/jira/browse/CALCITE-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085294#comment-17085294
]
Jesus Camacho Rodriguez commented on CALCITE-3149:
--
{quote}
We did encounter this problem in production for our test sets.
{quote}
Could the problem be that {{keyToType}} is not calling canonize?
> CACHE in RelDataTypeFactoryImpl can't be garbage collected
> ---
>
> Key: CALCITE-3149
> URL: https://issues.apache.org/jira/browse/CALCITE-3149
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Assignee: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Key and Value are pointing to the same object, even with soft value
> references, the objects in CACHE will not be garbage collected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3914) Improve SubstitutionVisitor to consider RexCall of type PLUS and TIMES for canonicalization
[ https://issues.apache.org/jira/browse/CALCITE-3914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-3914. -- Fix Version/s: 1.23.0 Resolution: Fixed Fixed in [ee1a9d2cacb67da4c5d7e8f0441c94a40dc69d66|https://github.com/apache/calcite/commit/ee1a9d2cacb67da4c5d7e8f0441c94a40dc69d66]. Thanks [~vgarg]! > Improve SubstitutionVisitor to consider RexCall of type PLUS and TIMES for > canonicalization > > > Key: CALCITE-3914 > URL: https://issues.apache.org/jira/browse/CALCITE-3914 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3914) Improve SubstitutionVisitor to consider RexCall of type PLUS and TIMES for canonicalization
[ https://issues.apache.org/jira/browse/CALCITE-3914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated CALCITE-3914: - Summary: Improve SubstitutionVisitor to consider RexCall of type PLUS and TIMES for canonicalization (was: Improve SubsitutionVisitor to consider RexCall of type PLUS and TIMES for canonicalization ) > Improve SubstitutionVisitor to consider RexCall of type PLUS and TIMES for > canonicalization > > > Key: CALCITE-3914 > URL: https://issues.apache.org/jira/browse/CALCITE-3914 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3908) JoinCommuteRule should update all input references in join condition
[
https://issues.apache.org/jira/browse/CALCITE-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3908.
--
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
[16c9c36f319b06ebcf1338f0e40e9de2ee912adf|https://github.com/apache/calcite/commit/16c9c36f319b06ebcf1338f0e40e9de2ee912adf].
> JoinCommuteRule should update all input references in join condition
>
>
> Key: CALCITE-3908
> URL: https://issues.apache.org/jira/browse/CALCITE-3908
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{JoinCommuteRule}} swaps the inputs of a join. It relies on an internal
> class {{VariableReplacer}} to update the references in the join condition.
> However, this class does not extend {{RexShuttle}} and ends up ignoring some
> of the references, e.g., those in {{RexFieldAccess}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3149) CACHE in RelDataTypeFactoryImpl can't be garbage collected
[ https://issues.apache.org/jira/browse/CALCITE-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080152#comment-17080152 ] Jesus Camacho Rodriguez commented on CALCITE-3149: -- [~hyuan], I have a quick question. There seem to be some places where we rely on reference equality to compare types: https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rex/RexUtil.java#L2610 Is that contract still valid after this patch went in? > CACHE in RelDataTypeFactoryImpl can't be garbage collected > --- > > Key: CALCITE-3149 > URL: https://issues.apache.org/jira/browse/CALCITE-3149 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Haisheng Yuan >Assignee: Haisheng Yuan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Key and Value are pointing to the same object, even with soft value > references, the objects in CACHE will not be garbage collected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3908) JoinCommuteRule should update all input references in join condition
[
https://issues.apache.org/jira/browse/CALCITE-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3908:
-
Description: {{JoinCommuteRule}} swaps the inputs of a join. It relies on
an internal class {{VariableReplacer}} to update the references in the join
condition. However, this class does not extend {{RexShuttle}} and ends up
ignoring some of the references, e.g., those in {{RexFieldAccess}}. (was:
{{JoinCommuteRule}} swaps the inputs of a join. It relies on an internal class
{{VariableReplacer}} to update the references in the join condition. However,
this class does not implement {{RexShuttle}} and ends up ignoring some of the
references, e.g., those in {{RexFieldAccess}}.)
> JoinCommuteRule should update all input references in join condition
>
>
> Key: CALCITE-3908
> URL: https://issues.apache.org/jira/browse/CALCITE-3908
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{JoinCommuteRule}} swaps the inputs of a join. It relies on an internal
> class {{VariableReplacer}} to update the references in the join condition.
> However, this class does not extend {{RexShuttle}} and ends up ignoring some
> of the references, e.g., those in {{RexFieldAccess}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3908) JoinCommuteRule should update all input references in join condition
[
https://issues.apache.org/jira/browse/CALCITE-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3908:
-
Priority: Critical (was: Major)
> JoinCommuteRule should update all input references in join condition
>
>
> Key: CALCITE-3908
> URL: https://issues.apache.org/jira/browse/CALCITE-3908
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{JoinCommuteRule}} swaps the inputs of a join. It relies on an internal
> class {{VariableReplacer}} to update the references in the join condition.
> However, this class does not implement {{RexShuttle}} and ends up ignoring
> some of the references, e.g., those in {{RexFieldAccess}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3908) JoinCommuteRule should update all input references in join condition
[
https://issues.apache.org/jira/browse/CALCITE-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3908:
-
Summary: JoinCommuteRule should update all input references in join
condition (was: JoinCommuteRule does not update all input references in
condition)
> JoinCommuteRule should update all input references in join condition
>
>
> Key: CALCITE-3908
> URL: https://issues.apache.org/jira/browse/CALCITE-3908
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> {{JoinCommuteRule}} swaps the inputs of a join. It relies on an internal
> class {{VariableReplacer}} to update the references in the join condition.
> However, this class does not implement {{RexShuttle}} and ends up ignoring
> some of the references, e.g., those in {{RexFieldAccess}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3908) JoinCommuteRule does not update all input references in condition
Jesus Camacho Rodriguez created CALCITE-3908:
Summary: JoinCommuteRule does not update all input references in
condition
Key: CALCITE-3908
URL: https://issues.apache.org/jira/browse/CALCITE-3908
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
{{JoinCommuteRule}} swaps the inputs of a join. It relies on an internal class
{{VariableReplacer}} to update the references in the join condition. However,
this class does not implement {{RexShuttle}} and ends up ignoring some of the
references, e.g., those in {{RexFieldAccess}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3898) RelOptPredicateList may generate incorrect map of constant values
[
https://issues.apache.org/jira/browse/CALCITE-3898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3898.
--
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
[c3147f477f843d283c43d1e858534d58a7174544|https://github.com/apache/calcite/commit/c3147f477f843d283c43d1e858534d58a7174544].
> RelOptPredicateList may generate incorrect map of constant values
> -
>
> Key: CALCITE-3898
> URL: https://issues.apache.org/jira/browse/CALCITE-3898
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The method relies on {{RexUtil.predicateConstants}} which in turn calls
> {{RexUtil.canAssignFrom}}. {{RexUtil.canAssignFrom}} is skipping any check on
> precision and scale. I observed the error in Hive when two VARCHAR types with
> different precision were given to the method, which was resulting on
> considering the result of the narrowing cast as the value of the reference.
> This lead to incorrect results.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3898) RelOptPredicateList may generate incorrect map of constant values
Jesus Camacho Rodriguez created CALCITE-3898:
Summary: RelOptPredicateList may generate incorrect map of
constant values
Key: CALCITE-3898
URL: https://issues.apache.org/jira/browse/CALCITE-3898
Project: Calcite
Issue Type: Bug
Components: core
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
The method relies on {{RexUtil.predicateConstants}} which in turn calls
{{RexUtil.canAssignFrom}}. {{RexUtil.canAssignFrom}} is skipping any check on
precision and scale. I observed the error in Hive when two VARCHAR types with
different precision were given to the method, which was resulting on
considering the result of the narrowing cast as the value of the reference.
This lead to incorrect results.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3862) Materialized view rewriting algorithm throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CALCITE-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3862.
--
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/0e345fd8d9eaa4223ada8ecc2987a7c82cb0d2b8.
Thanks [~vgarg]!
> Materialized view rewriting algorithm throws IndexOutOfBoundsException
> --
>
> Key: CALCITE-3862
> URL: https://issues.apache.org/jira/browse/CALCITE-3862
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> *Repro*
> {code:sql}
> +sql("select \"deptno\", \"empid\", \"salary\", sum(1) "
> ++ "from \"emps\"\n"
> ++ "group by \"deptno\", \"empid\", \"salary\"",
> +"select sum(1) "
> ++ "from \"emps\"\n"
> ++ "join \"depts\" on \"depts\".\"deptno\" = \"empid\" group by
> \"empid\", \"depts\".\"deptno\"")
> +.withResultContains(
> +"EnumerableCalc(expr#0..1=[{inputs}], EXPR$0=[$t1])\n"
> ++ " EnumerableAggregate(group=[{1}], EXPR$0=[$SUM0($3)])\n"
> ++ "EnumerableHashJoin(condition=[=($1, $4)],
> joinType=[inner])\n"
> ++ " EnumerableTableScan(table=[[hr, m0]])")
> +.ok();
> {code}
> *Error*
> {code}
> Next exception 1: [CIRCULAR REFERENCE SQLException]
> Next exception 2: [CIRCULAR REFERENCE RuntimeException]
> Next exception 3: java.lang.IndexOutOfBoundsException: index (2)
> must be less than size (2)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rex.RexBuilder.makeInputRef(RexBuilder.java:853)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule$3.visitInputRef(MaterializedViewRule.java:1217)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule$3.visitInputRef(MaterializedViewRule.java:1181)
> at
> org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
> at
> org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:277)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule.shuttleReferences(MaterializedViewRule.java:1242)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewAggregateRule.rewriteView(MaterializedViewAggregateRule.java:728)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule.perform(MaterializedViewRule.java:485)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewOnlyAggregateRule.onMatch(MaterializedViewOnlyAggregateRule.java:63)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:238)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:540)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:286)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346)
> at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:165)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:290)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:207)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:634)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:498)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:468)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3862) Materialized view rewriting algorithm throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CALCITE-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3862:
-
Summary: Materialized view rewriting algorithm throws
IndexOutOfBoundsException (was: Rewriting for materialized view consisting of
group by on join keys with aggregate fails)
> Materialized view rewriting algorithm throws IndexOutOfBoundsException
> --
>
> Key: CALCITE-3862
> URL: https://issues.apache.org/jira/browse/CALCITE-3862
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *Repro*
> {code:sql}
> +sql("select \"deptno\", \"empid\", \"salary\", sum(1) "
> ++ "from \"emps\"\n"
> ++ "group by \"deptno\", \"empid\", \"salary\"",
> +"select sum(1) "
> ++ "from \"emps\"\n"
> ++ "join \"depts\" on \"depts\".\"deptno\" = \"empid\" group by
> \"empid\", \"depts\".\"deptno\"")
> +.withResultContains(
> +"EnumerableCalc(expr#0..1=[{inputs}], EXPR$0=[$t1])\n"
> ++ " EnumerableAggregate(group=[{1}], EXPR$0=[$SUM0($3)])\n"
> ++ "EnumerableHashJoin(condition=[=($1, $4)],
> joinType=[inner])\n"
> ++ " EnumerableTableScan(table=[[hr, m0]])")
> +.ok();
> {code}
> *Error*
> {code}
> Next exception 1: [CIRCULAR REFERENCE SQLException]
> Next exception 2: [CIRCULAR REFERENCE RuntimeException]
> Next exception 3: java.lang.IndexOutOfBoundsException: index (2)
> must be less than size (2)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rex.RexBuilder.makeInputRef(RexBuilder.java:853)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule$3.visitInputRef(MaterializedViewRule.java:1217)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule$3.visitInputRef(MaterializedViewRule.java:1181)
> at
> org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
> at
> org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:277)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule.shuttleReferences(MaterializedViewRule.java:1242)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewAggregateRule.rewriteView(MaterializedViewAggregateRule.java:728)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule.perform(MaterializedViewRule.java:485)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewOnlyAggregateRule.onMatch(MaterializedViewOnlyAggregateRule.java:63)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:238)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:540)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:286)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346)
> at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:165)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:290)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:207)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:634)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:498)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:468)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3848) Rewriting for materialized view consisting of group by on join keys fails with Mappings$NoElementException
[
https://issues.apache.org/jira/browse/CALCITE-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3848.
--
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/18b9bc36e8be5b9a038548d9513466a756eeba61.
Thanks [~vgarg]!
> Rewriting for materialized view consisting of group by on join keys fails
> with Mappings$NoElementException
> --
>
> Key: CALCITE-3848
> URL: https://issues.apache.org/jira/browse/CALCITE-3848
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Test case
> {code:java}
> + @Test public void testAggregateOnJoinKeys() {
> +checkMaterialize(
> +"select \"deptno\", \"empid\", \"salary\"\n"
> ++ "from \"emps\"\n"
> ++ "group by \"deptno\", \"empid\", \"salary\"",
> + "select \"empid\", \"depts\".\"deptno\" \n"
> ++ "from \"emps\"\n"
> ++ "join \"depts\" on \"depts\".\"deptno\" = \"empid\" group by
> \"empid\", \"depts\".\"deptno\"",
> +HR_FKUK_MODEL,
> +CalciteAssert.checkResultContains(
> +"EnumerableCalc(expr#0=[{inputs}], empid=[$t0], empid0=[$t0])\n"
> + + " EnumerableAggregate(group=[{1}])\n"
> ++ "EnumerableHashJoin(condition=[=($1, $3)],
> joinType=[inner])\n"
> ++ " EnumerableTableScan(table=[[hr, m0]])"));
> + }
> +
> {code}
> Error:
> {code}
> Caused by: java.lang.RuntimeException: Error while applying rule
> MaterializedViewAggregateRule(Aggregate), args
> [rel#64476:EnumerableAggregate.ENUMERABLE.[](input=RelSubset#64475,group={0,
> 1})]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:260)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:634)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:286)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346)
> at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:165)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:290)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:207)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:634)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:498)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:468)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 16 more
> Next exception 1: [CIRCULAR REFERENCE SQLException]
> Next exception 2: [CIRCULAR REFERENCE RuntimeException]
> Next exception 3:
> org.apache.calcite.util.mapping.Mappings$NoElementException: source #0 has no
> target in mapping [size=1, sourceCount=2, targetCount=7, elements=[1:1]]
> at
> org.apache.calcite.util.mapping.Mappings$AbstractMapping.getTarget(Mappings.java:881)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewAggregateRule.rewriteView(MaterializedViewAggregateRule.java:677)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule.perform(MaterializedViewRule.java:485)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewOnlyAggregateRule.onMatch(MaterializedViewOnlyAggregateRule.java:63)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:233)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:634)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:286)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346)
> at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:165)
> at
>
[jira] [Updated] (CALCITE-3848) Rewriting for materialized view consisting of group by on join keys fails with Mappings$NoElementException
[
https://issues.apache.org/jira/browse/CALCITE-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3848:
-
Summary: Rewriting for materialized view consisting of group by on join
keys fails with Mappings$NoElementException (was: Materialized view rewriting
fails for mv consisting of group by on join keys)
> Rewriting for materialized view consisting of group by on join keys fails
> with Mappings$NoElementException
> --
>
> Key: CALCITE-3848
> URL: https://issues.apache.org/jira/browse/CALCITE-3848
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Test case
> {code:java}
> + @Test public void testAggregateOnJoinKeys() {
> +checkMaterialize(
> +"select \"deptno\", \"empid\", \"salary\"\n"
> ++ "from \"emps\"\n"
> ++ "group by \"deptno\", \"empid\", \"salary\"",
> + "select \"empid\", \"depts\".\"deptno\" \n"
> ++ "from \"emps\"\n"
> ++ "join \"depts\" on \"depts\".\"deptno\" = \"empid\" group by
> \"empid\", \"depts\".\"deptno\"",
> +HR_FKUK_MODEL,
> +CalciteAssert.checkResultContains(
> +"EnumerableCalc(expr#0=[{inputs}], empid=[$t0], empid0=[$t0])\n"
> + + " EnumerableAggregate(group=[{1}])\n"
> ++ "EnumerableHashJoin(condition=[=($1, $3)],
> joinType=[inner])\n"
> ++ " EnumerableTableScan(table=[[hr, m0]])"));
> + }
> +
> {code}
> Error:
> {code}
> Caused by: java.lang.RuntimeException: Error while applying rule
> MaterializedViewAggregateRule(Aggregate), args
> [rel#64476:EnumerableAggregate.ENUMERABLE.[](input=RelSubset#64475,group={0,
> 1})]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:260)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:634)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:286)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346)
> at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:165)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:290)
> at
> org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:207)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:634)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:498)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:468)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 16 more
> Next exception 1: [CIRCULAR REFERENCE SQLException]
> Next exception 2: [CIRCULAR REFERENCE RuntimeException]
> Next exception 3:
> org.apache.calcite.util.mapping.Mappings$NoElementException: source #0 has no
> target in mapping [size=1, sourceCount=2, targetCount=7, elements=[1:1]]
> at
> org.apache.calcite.util.mapping.Mappings$AbstractMapping.getTarget(Mappings.java:881)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewAggregateRule.rewriteView(MaterializedViewAggregateRule.java:677)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewRule.perform(MaterializedViewRule.java:485)
> at
> org.apache.calcite.rel.rules.materialize.MaterializedViewOnlyAggregateRule.onMatch(MaterializedViewOnlyAggregateRule.java:63)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:233)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:634)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:286)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:346)
> at
> org.apache.calcite.prepare.Prepare.optimize(Prepare.java:165)
> at
>
[jira] [Comment Edited] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046157#comment-17046157 ] Jesus Camacho Rodriguez edited comment on CALCITE-3825 at 3/7/20, 12:04 AM: Fixed in https://github.com/apache/calcite/commit/5dea67890a1916771e9a335e45969f2a4d4f7d3c . was (Author: jcamachorodriguez): Fixed in https://github.com/apache/calcite/commit/efa5cb7ba6a754ef98281f4f142c16d895a49df4 . > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046205#comment-17046205 ] Jesus Camacho Rodriguez edited comment on CALCITE-3825 at 3/7/20, 12:04 AM: Pushed addendum in https://github.com/apache/calcite/commit/9b7b6316eaceec12762b209acec915262bda37eb. The plan is correct though the change is unexpected; some weird interaction was observed previously (CALCITE-2953), I am wondering whether root cause is the same. was (Author: jcamachorodriguez): Pushed addendum in https://github.com/apache/calcite/commit/153923bb92110c0b97ae46f45917ce3b94347291. The plan is correct though the change is unexpected; some weird interaction was observed previously (CALCITE-2953), I am wondering whether root cause is the same. > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 40m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-3825. -- Fix Version/s: 1.23.0 Resolution: Fixed > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 40m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3824.
--
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/50cd53845e13f870f245bf9b77bb996b6b8d825c.
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3734) MySQL JDBC rewrite is producing queries with CHAR with range beyond 255
[ https://issues.apache.org/jira/browse/CALCITE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-3734. -- Fix Version/s: 1.23.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/aeaaca6783f9b4461b6b1fedf19de9a67ba0a96f. > MySQL JDBC rewrite is producing queries with CHAR with range beyond 255 > --- > > Key: CALCITE-3734 > URL: https://issues.apache.org/jira/browse/CALCITE-3734 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Queries containing cast to varchar/string is rewritten into cast to CHAR with > range beyond 255 causing query failure. This range/precision should be > limited to 255. > I will provide a test case later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047861#comment-17047861
]
Jesus Camacho Rodriguez commented on CALCITE-3824:
--
I was planning to cherry-pick the same commit, I think it is fine unless you
want to have it open in case there is any feedback.
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3824:
-
Comment: was deleted
(was: Fixed in
https://github.com/apache/calcite/commit/8f39d6d136c427afeed108c5b499331c1c84bbdf.
Thanks for your contribution [~vgarg]!)
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (CALCITE-3734) MySQL JDBC rewrite is producing queries with CHAR with range beyond 255
[ https://issues.apache.org/jira/browse/CALCITE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated CALCITE-3734: - Comment: was deleted (was: Fixed in https://github.com/apache/calcite/commit/e427180b6a55445fe246b00c60259a95f96bdbf2. Thanks for your contribution [~vgarg]!) > MySQL JDBC rewrite is producing queries with CHAR with range beyond 255 > --- > > Key: CALCITE-3734 > URL: https://issues.apache.org/jira/browse/CALCITE-3734 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > Queries containing cast to varchar/string is rewritten into cast to CHAR with > range beyond 255 causing query failure. This range/precision should be > limited to 255. > I will provide a test case later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3824:
-
Fix Version/s: (was: 1.23.0)
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated CALCITE-3825: - Fix Version/s: (was: 1.23.0) > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reopened CALCITE-3825: -- > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 40m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3734) MySQL JDBC rewrite is producing queries with CHAR with range beyond 255
[ https://issues.apache.org/jira/browse/CALCITE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated CALCITE-3734: - Fix Version/s: (was: 1.23.0) > MySQL JDBC rewrite is producing queries with CHAR with range beyond 255 > --- > > Key: CALCITE-3734 > URL: https://issues.apache.org/jira/browse/CALCITE-3734 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > Queries containing cast to varchar/string is rewritten into cast to CHAR with > range beyond 255 causing query failure. This range/precision should be > limited to 255. > I will provide a test case later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reopened CALCITE-3824:
--
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (CALCITE-3734) MySQL JDBC rewrite is producing queries with CHAR with range beyond 255
[ https://issues.apache.org/jira/browse/CALCITE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reopened CALCITE-3734: -- > MySQL JDBC rewrite is producing queries with CHAR with range beyond 255 > --- > > Key: CALCITE-3734 > URL: https://issues.apache.org/jira/browse/CALCITE-3734 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > Queries containing cast to varchar/string is rewritten into cast to CHAR with > range beyond 255 causing query failure. This range/precision should be > limited to 255. > I will provide a test case later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[
https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047085#comment-17047085
]
Jesus Camacho Rodriguez commented on CALCITE-3825:
--
Thanks for the feedback [~julianhyde]. Public classes was for the purpose of
making them extensible by other engines if necessary
({{AbstractMaterializedViewRule}} was public too). Their constructors are
public for non-abstract classes and protected for the abstract ones. I guess
they could all be made protected as they were in the past, although on the
other hand, I am not sure we need to force an engine relying in Calcite to
extend a certain rule if we can just instantiate it?
I realize there may be some concern because we are making all those methods a
public API in Calcite now; however, the rules have been stable for some time,
so I am hoping it is OK. Please, let me know what you think.
> Split AbstractMaterializedViewRule into multiple classes
>
>
> Key: CALCITE-3825
> URL: https://issues.apache.org/jira/browse/CALCITE-3825
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> AbstractMaterializedViewRule contains a materialized view-based rewriting
> algorithm that has been there for multiple releases and it is used by some
> engines relying in Calcite, e.g., Apache Hive.
> The main reason to have a single file/class for the rule was to make the
> logic self-contained instead of spreading it between multiple files from the
> onset, as it was experimental and we were not sure how far the implementation
> would go. In retrospective, we should have refactored that code sooner rather
> than later, since it makes very difficult to understand and maintain logic
> that is already complicated enough.
> This issue is to split AbstractMaterializedViewRule into multiple
> files/classes (it already contained multiple internal classes).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046205#comment-17046205 ] Jesus Camacho Rodriguez commented on CALCITE-3825: -- Pushed addendum in https://github.com/apache/calcite/commit/153923bb92110c0b97ae46f45917ce3b94347291. The plan is correct though the change is unexpected; some weird interaction was observed previously (CALCITE-2953), I am wondering whether root cause is the same. > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 40m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046196#comment-17046196 ] Jesus Camacho Rodriguez commented on CALCITE-3825: -- This change causes a diff in LatticeTest#testDistinctCount2. Created a second PR for the addendum: https://github.com/apache/calcite/pull/1836 > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3824.
--
Fix Version/s: 1.23.0
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/8f39d6d136c427afeed108c5b499331c1c84bbdf.
Thanks for your contribution [~vgarg]!
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.23.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-3825. -- Fix Version/s: 1.23.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/efa5cb7ba6a754ef98281f4f142c16d895a49df4 . > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 20m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
[ https://issues.apache.org/jira/browse/CALCITE-3825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated CALCITE-3825: - Description: AbstractMaterializedViewRule contains a materialized view-based rewriting algorithm that has been there for multiple releases and it is used by some engines relying in Calcite, e.g., Apache Hive. The main reason to have a single file/class for the rule was to make the logic self-contained instead of spreading it between multiple files from the onset, as it was experimental and we were not sure how far the implementation would go. In retrospective, we should have refactored that code sooner rather than later, since it makes very difficult to understand and maintain logic that is already complicated enough. This issue is to split AbstractMaterializedViewRule into multiple files/classes (it already contained multiple internal classes). was: AbstractMaterializedViewRule contains a materialized view-based rewriting algorithm that has been there for multiple releases and it is used by some engines relying in Calcite, e.g., Apache Hive. The main reason to have a single file/class for the rule was to make the logic self-contained instead of spreading it between multiple files from the onset, as it was experimental and we were not sure how far the implementation would go. In retrospective, we should have refactor that code sooner rather than later, since it makes very difficult to understand and maintain logic that is already complicated enough. This issue is to split AbstractMaterializedViewRule into multiple files/classes (it already contained multiple internal classes). > Split AbstractMaterializedViewRule into multiple classes > > > Key: CALCITE-3825 > URL: https://issues.apache.org/jira/browse/CALCITE-3825 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > AbstractMaterializedViewRule contains a materialized view-based rewriting > algorithm that has been there for multiple releases and it is used by some > engines relying in Calcite, e.g., Apache Hive. > The main reason to have a single file/class for the rule was to make the > logic self-contained instead of spreading it between multiple files from the > onset, as it was experimental and we were not sure how far the implementation > would go. In retrospective, we should have refactored that code sooner rather > than later, since it makes very difficult to understand and maintain logic > that is already complicated enough. > This issue is to split AbstractMaterializedViewRule into multiple > files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3825) Split AbstractMaterializedViewRule into multiple classes
Jesus Camacho Rodriguez created CALCITE-3825: Summary: Split AbstractMaterializedViewRule into multiple classes Key: CALCITE-3825 URL: https://issues.apache.org/jira/browse/CALCITE-3825 Project: Calcite Issue Type: Improvement Components: core Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez AbstractMaterializedViewRule contains a materialized view-based rewriting algorithm that has been there for multiple releases and it is used by some engines relying in Calcite, e.g., Apache Hive. The main reason to have a single file/class for the rule was to make the logic self-contained instead of spreading it between multiple files from the onset, as it was experimental and we were not sure how far the implementation would go. In retrospective, we should have refactor that code sooner rather than later, since it makes very difficult to understand and maintain logic that is already complicated enough. This issue is to split AbstractMaterializedViewRule into multiple files/classes (it already contained multiple internal classes). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045079#comment-17045079
]
Jesus Camacho Rodriguez commented on CALCITE-3824:
--
Yes, thanks :)
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3734) MySQL JDBC rewrite is producing queries with CHAR with range beyond 255
[ https://issues.apache.org/jira/browse/CALCITE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved CALCITE-3734. -- Fix Version/s: 1.23.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/e427180b6a55445fe246b00c60259a95f96bdbf2. Thanks for your contribution [~vgarg]! > MySQL JDBC rewrite is producing queries with CHAR with range beyond 255 > --- > > Key: CALCITE-3734 > URL: https://issues.apache.org/jira/browse/CALCITE-3734 > Project: Calcite > Issue Type: Bug > Components: jdbc-adapter >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Fix For: 1.23.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Queries containing cast to varchar/string is rewritten into cast to CHAR with > range beyond 255 causing query failure. This range/precision should be > limited to 255. > I will provide a test case later. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reopened CALCITE-3824:
--
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 1.23.0
>
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3824:
-
Comment: was deleted
(was: Fixed in
https://github.com/apache/calcite/commit/e427180b6a55445fe246b00c60259a95f96bdbf2.
Thanks for your contribution [~vgarg]!)
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 1.23.0
>
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3824:
-
Fix Version/s: (was: 1.23.0)
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3824:
-
Fix Version/s: (was: 1.22.0)
1.23.0
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 1.23.0
>
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3824:
-
Fix Version/s: 1.22.0
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 1.22.0
>
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3824) JoinProjectTransposeRule should skip Projects containing windowing expression
[
https://issues.apache.org/jira/browse/CALCITE-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3824.
--
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/e427180b6a55445fe246b00c60259a95f96bdbf2.
Thanks for your contribution [~vgarg]!
> JoinProjectTransposeRule should skip Projects containing windowing expression
> -
>
> Key: CALCITE-3824
> URL: https://issues.apache.org/jira/browse/CALCITE-3824
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> This rule could push windowing expressions within join condition which
> doesn't make sense.
> For example
> {code:sql}
> select * from dept a
> join (select rank() over (order by name) as r, 1 + 1 from dept) as b
> on a.name = b.r
> {code}
> Above query produces following plan after the rule
> {code}
> LogicalProject(DEPTNO=[$0], NAME=[$1], R=[$3], EXPR$1=[$4])
> LogicalProject(DEPTNO=[$0], NAME=[$1], NAME0=[CAST($1):BIGINT NOT NULL],
> R=[RANK() OVER (ORDER BY $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT
> ROW)], EXPR$1=[+(1, 1)])
> LogicalJoin(condition=[=(CAST($1):BIGINT NOT NULL, RANK() OVER (ORDER BY
> $3 RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW))], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
[
https://issues.apache.org/jira/browse/CALCITE-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3189.
--
Resolution: Fixed
Fix Version/s: 1.21.0
Fixed in
[0732283cab7894ffdce6a22ebe5d31b28d389a4d|https://github.com/apache/calcite/commit/0732283cab7894ffdce6a22ebe5d31b28d389a4d].
> Multiple fixes for Oracle SQL dialect
> -
>
> Key: CALCITE-3189
> URL: https://issues.apache.org/jira/browse/CALCITE-3189
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Among others, it includes i) SQL translation support for specific types (e.g.
> {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type
> (by default 4000), iii) creating datetime literals correctly, and iv) method
> to infer whether a given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
[
https://issues.apache.org/jira/browse/CALCITE-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881626#comment-16881626
]
Jesus Camacho Rodriguez edited comment on CALCITE-3189 at 7/10/19 12:31 AM:
[~julianhyde], not sure if you could take a look since it introduces a new
method {{supportsDataType}} in SQLDialect class? Method can be used by a rule
to determine whether a certain expression may be pushed or not through the JDBC
connection depending on their return type.
was (Author: jcamachorodriguez):
[~julianhyde], not sure if you could take a look since it introduces a new
method {{supportsDataType}} in SQLDialect class? Method can be used by a rule
to determine whether a certain expression may be pushed or not through the JDBC
connection.
> Multiple fixes for Oracle SQL dialect
> -
>
> Key: CALCITE-3189
> URL: https://issues.apache.org/jira/browse/CALCITE-3189
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Among others, it includes i) SQL translation support for specific types (e.g.
> {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type
> (by default 4000), iii) creating datetime literals correctly, and iv) method
> to infer whether a given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
[
https://issues.apache.org/jira/browse/CALCITE-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881626#comment-16881626
]
Jesus Camacho Rodriguez commented on CALCITE-3189:
--
[~julianhyde], not sure if you could take a look since it introduces a new
method {{supportsDataType}} in SQLDialect class? Method can be used by a rule
to determine whether a certain expression may be pushed or not through the JDBC
connection.
> Multiple fixes for Oracle SQL dialect
> -
>
> Key: CALCITE-3189
> URL: https://issues.apache.org/jira/browse/CALCITE-3189
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Among others, it includes i) SQL translation support for specific types (e.g.
> {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type
> (by default 4000), iii) creating datetime literals correctly, and iv) method
> to infer whether a given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
[
https://issues.apache.org/jira/browse/CALCITE-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3189:
-
Description: Among others, it includes i) SQL translation support for
specific types (e.g. {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length
of {{VARCHAR}} type (by default 4000), iii) creating datetime literals
correctly, and iv) method to infer whether a given data type is supported or
not. (was: Among others, it includes i) SQL translation support for specific
types (e.g. {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of
{{VARCHAR}} type, iii) creating datetime literals correctly, and iv) method to
infer whether a given data type is supported or not.)
> Multiple fixes for Oracle SQL dialect
> -
>
> Key: CALCITE-3189
> URL: https://issues.apache.org/jira/browse/CALCITE-3189
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Among others, it includes i) SQL translation support for specific types (e.g.
> {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type
> (by default 4000), iii) creating datetime literals correctly, and iv) method
> to infer whether a given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
[
https://issues.apache.org/jira/browse/CALCITE-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3189:
-
Description: Among others, it includes i) SQL translation support for
specific types (e.g. {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length
of {{VARCHAR}} type, iii) creating datetime literals correctly, and iv) method
to infer whether a given data type is supported or not. (was: Among others, it
includes i) SQL translation support for custom types (e.g. {{SMALLINT}} -->
{{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type, iii) creating
datetime literals correctly, and iv) method to infer whether a given data type
is supported or not.)
> Multiple fixes for Oracle SQL dialect
> -
>
> Key: CALCITE-3189
> URL: https://issues.apache.org/jira/browse/CALCITE-3189
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Among others, it includes i) SQL translation support for specific types (e.g.
> {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type,
> iii) creating datetime literals correctly, and iv) method to infer whether a
> given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
[
https://issues.apache.org/jira/browse/CALCITE-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-3189:
-
Component/s: core
> Multiple fixes for Oracle SQL dialect
> -
>
> Key: CALCITE-3189
> URL: https://issues.apache.org/jira/browse/CALCITE-3189
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Among others, it includes i) SQL translation support for custom types (e.g.
> {{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type,
> iii) creating datetime literals correctly, and iv) method to infer whether a
> given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3189) Multiple fixes for Oracle SQL dialect
Jesus Camacho Rodriguez created CALCITE-3189:
Summary: Multiple fixes for Oracle SQL dialect
Key: CALCITE-3189
URL: https://issues.apache.org/jira/browse/CALCITE-3189
Project: Calcite
Issue Type: Bug
Affects Versions: 1.20.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Among others, it includes i) SQL translation support for custom types (e.g.
{{SMALLINT}} --> {{NUMBER(5)}}), ii) limiting max length of {{VARCHAR}} type,
iii) creating datetime literals correctly, and iv) method to infer whether a
given data type is supported or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-3052) Error while applying rule MaterializedViewAggregateRule(Project-Aggregate): ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CALCITE-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3052.
--
Resolution: Fixed
Fix Version/s: 1.20.0
Fixed in
[046bb81a9a4ac3e5b63c0ca4a0141446df325a44|https://github.com/apache/calcite/commit/046bb81a9a4ac3e5b63c0ca4a0141446df325a44].
> Error while applying rule MaterializedViewAggregateRule(Project-Aggregate):
> ArrayIndexOutOfBoundsException
> --
>
> Key: CALCITE-3052
> URL: https://issues.apache.org/jira/browse/CALCITE-3052
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Anton Haidai
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> *Materialized views enabled:*
> # {{select avg(grade), count\(*), max(grade), sum(grade), min(grade), team
> from students group by team}}
> # {{select avg(grade), count\(*), max(grade), sum(grade), min(grade), team,
> faculty from students group by faculty, team}},
> *Query:*
> # {{select count\(*), team from students group by team}}
> *Error* (stacktrace is obtained using the current *master* branch:
> "247c7d4f76"):
> {noformat}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:60)
> at org.apache.calcite.rex.RexBuilder.makeInputRef(RexBuilder.java:841)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule$MaterializedViewAggregateRule.rewriteView(AbstractMaterializedViewRule.java:1507)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule.perform(AbstractMaterializedViewRule.java:522)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule$MaterializedViewProjectAggregateRule.onMatch(AbstractMaterializedViewRule.java:1776)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 71 common frames omitted
> {noformat}
> Reproducible only if both Materialization views listed are enabled: any
> single one of these two could be successfully used with the query without any
> errors. Looks like is is reproducible when AbstractMaterializedViewRule is
> trying to rewrite one materialized view using the another materialized view.
> Currently, I'm trying to reproduce the issue in "MaterializationTest":
> without a success so far, I'll update the ticket if I'll find a working way
> to reproduce the issue in the test will be discovered.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-3066) RelToSqlConverter may incorrectly throw an AssertionError for some decimal literals
[
https://issues.apache.org/jira/browse/CALCITE-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved CALCITE-3066.
--
Resolution: Fixed
Fix Version/s: 1.20.0
Fixed in
[d6896202c865b38b7821f1e0b644e1de0c95eda2|https://github.com/apache/calcite/commit/d6896202c865b38b7821f1e0b644e1de0c95eda2].
> RelToSqlConverter may incorrectly throw an AssertionError for some decimal
> literals
> ---
>
> Key: CALCITE-3066
> URL: https://issues.apache.org/jira/browse/CALCITE-3066
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Issue can be reproduced adding the following query to
> {{RelToSqlConverterTest}}:
> {code:sql}
> select -0.000123 from "expense_fact";
> {code}
> {code}
> Caused by: java.lang.AssertionError: -1.23E-8
> at
> org.apache.calcite.sql.SqlLiteral.createExactNumeric(SqlLiteral.java:872)
> at
> org.apache.calcite.rel.rel2sql.SqlImplementor$Context.toSql(SqlImplementor.java:502)
> at
> org.apache.calcite.rel.rel2sql.RelToSqlConverter.visit(RelToSqlConverter.java:186)
> ... 34 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3052) Error while applying rule MaterializedViewAggregateRule(Project-Aggregate): ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CALCITE-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16840610#comment-16840610
]
Jesus Camacho Rodriguez commented on CALCITE-3052:
--
Thanks [~anha] for the detailed dumps, otherwise this would have been
impossible! I will merge the changes shortly.
> Error while applying rule MaterializedViewAggregateRule(Project-Aggregate):
> ArrayIndexOutOfBoundsException
> --
>
> Key: CALCITE-3052
> URL: https://issues.apache.org/jira/browse/CALCITE-3052
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Anton Haidai
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> *Materialized views enabled:*
> # {{select avg(grade), count\(*), max(grade), sum(grade), min(grade), team
> from students group by team}}
> # {{select avg(grade), count\(*), max(grade), sum(grade), min(grade), team,
> faculty from students group by faculty, team}},
> *Query:*
> # {{select count\(*), team from students group by team}}
> *Error* (stacktrace is obtained using the current *master* branch:
> "247c7d4f76"):
> {noformat}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:60)
> at org.apache.calcite.rex.RexBuilder.makeInputRef(RexBuilder.java:841)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule$MaterializedViewAggregateRule.rewriteView(AbstractMaterializedViewRule.java:1507)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule.perform(AbstractMaterializedViewRule.java:522)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule$MaterializedViewProjectAggregateRule.onMatch(AbstractMaterializedViewRule.java:1776)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 71 common frames omitted
> {noformat}
> Reproducible only if both Materialization views listed are enabled: any
> single one of these two could be successfully used with the query without any
> errors. Looks like is is reproducible when AbstractMaterializedViewRule is
> trying to rewrite one materialized view using the another materialized view.
> Currently, I'm trying to reproduce the issue in "MaterializationTest":
> without a success so far, I'll update the ticket if I'll find a working way
> to reproduce the issue in the test will be discovered.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
