[jira] [Commented] (CALCITE-3334) Refinement for Substitution-Based MV Matching
[ https://issues.apache.org/jira/browse/CALCITE-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944198#comment-16944198 ] Haisheng Yuan commented on CALCITE-3334: Also please remove all the unused and retired rules/classes, to make it a complete pull request. Maybe in a different commit. > Refinement for Substitution-Based MV Matching > - > > Key: CALCITE-3334 > URL: https://issues.apache.org/jira/browse/CALCITE-3334 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: jin xing >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > The approach of substitution-based MV matching is an effective way for its > simplicity and extensibility. > This JIRA proposes to refine existing implementation by several points: > # Canonicalize before MV matching -- by such canonicalization we can > significantly simplify the algebra tree and lower the difficulty for > materialization matching. > # Separate matching rules into two categories and enumerate common matching > patterns which need to be covered by rules. > Please check the design doc: [Design > Doc|https://docs.google.com/document/d/1JpwGNFE3hw3yXb7W3-95-jXKClZC5UFPKbuhgYDuEu4/edit#] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3334) Refinement for Substitution-Based MV Matching
[ https://issues.apache.org/jira/browse/CALCITE-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944197#comment-16944197 ] Haisheng Yuan commented on CALCITE-3334: [~jinxing6...@126.com] I still don't quite get why you provide an option to disable SubstitutionVisitor. If both code paths can work, why do we need to keep both? > Refinement for Substitution-Based MV Matching > - > > Key: CALCITE-3334 > URL: https://issues.apache.org/jira/browse/CALCITE-3334 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: jin xing >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > The approach of substitution-based MV matching is an effective way for its > simplicity and extensibility. > This JIRA proposes to refine existing implementation by several points: > # Canonicalize before MV matching -- by such canonicalization we can > significantly simplify the algebra tree and lower the difficulty for > materialization matching. > # Separate matching rules into two categories and enumerate common matching > patterns which need to be covered by rules. > Please check the design doc: [Design > Doc|https://docs.google.com/document/d/1JpwGNFE3hw3yXb7W3-95-jXKClZC5UFPKbuhgYDuEu4/edit#] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944182#comment-16944182 ] Haisheng Yuan commented on CALCITE-3349: Still not able to do it. > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3358) Make Function DDLs implement SqlExecutableStatement
[ https://issues.apache.org/jira/browse/CALCITE-3358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944180#comment-16944180 ] Zhenqiu Huang commented on CALCITE-3358: [~danny0405] [~suez1224] Would you please assign this ticket to me? > Make Function DDLs implement SqlExecutableStatement > --- > > Key: CALCITE-3358 > URL: https://issues.apache.org/jira/browse/CALCITE-3358 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944179#comment-16944179 ] Zhenqiu Huang commented on CALCITE-3349: [~hyuan] Would you please assign this ticket to me ? > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3347) IndexOutOfBoundsException in FixNullabilityShuttle when using FilterIntoJoinRule
[ https://issues.apache.org/jira/browse/CALCITE-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-3347. Fix Version/s: 1.22.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/3498026cd9819c52198a89e0d06f564b83ba4a8a, thanks for the PR, [~yanlin-Lynn]! > IndexOutOfBoundsException in FixNullabilityShuttle when using > FilterIntoJoinRule > > > Key: CALCITE-3347 > URL: https://issues.apache.org/jira/browse/CALCITE-3347 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.21.0 >Reporter: Amit Chavan >Priority: Major > Labels: pull-request-available > Fix For: 1.22.0 > > Attachments: TestCalcite.java > > Time Spent: 3h 50m > Remaining Estimate: 0h > > I am reporting a bug that happens in calcite 1.21 release. I have a query as > below > String query = "SELECT * FROM tblspace1.tsql where n1=? and k1 in (SELECT k1 > FROM tblspace1.tsql where n1=?)"; > > I am also attaching the unit test to reproduce this issue. > > The filterJoinRule throws an exception – > java.lang.RuntimeException: Error while applying rule > FilterJoinRule:FilterJoinRule:filter, args > [rel#39:EnumerableFilter.ENUMERABLE.[](input=RelSubset#38,condition==($1, > ?0)), > rel#176:EnumerableHashJoin.ENUMERABLE.[](left=RelSubset#17,right=RelSubset#73,condition==($0, > $3),joinType=semi)] > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631) > at org.apache.calcite.TestCalcite.testQuery(TestCalcite.java:199) > at org.apache.calcite.TestCalcite.problem_with_1_21(TestCalcite.java:256) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) > at > com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) > at > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) > at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) > Caused by: java.lang.IndexOutOfBoundsException: index (3) must be less than > size (3) > at > com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310) > at > com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293) > at > com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67) > at > com.google.common.collect.Lists$TransformingRandomAccessList.get(Lists.java:627) > at > org.apache.calcite.rex.RexUtil$FixNullabilityShuttle.visitInputRef(RexUtil.java:2529) > at > org.apache.calcite.rex.RexUtil$FixNullabilityShuttle.visitInputRef(RexUtil.java:2518) > at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112) > at org.apache.calcite.rex.RexShuttle.visitList(RexShuttle.java:149) > at org.apache.calcite.rex.RexShuttle.visitCall(RexShuttle.java:101) > at org.apache.calcite.rex.RexShuttle.visitCall(RexShuttle.java:34) > at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) > at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:277) > at org.apache.calcite.rex.RexShuttle.mutate(RexShuttle.java:239) > at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:257) > at org.apache.calcite.rex.RexUtil.fixUp(RexUtil.java:1635) > at > org.
[jira] [Commented] (CALCITE-3186) IN expressions in UPDATE statements throws Exceptions
[
https://issues.apache.org/jira/browse/CALCITE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944146#comment-16944146
]
Pressenna commented on CALCITE-3186:
thank you.
> IN expressions in UPDATE statements throws Exceptions
> -
>
> Key: CALCITE-3186
> URL: https://issues.apache.org/jira/browse/CALCITE-3186
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.20.0
>Reporter: Pressenna
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The patch to get correlated sub-queries working in UPDATE statements had this
> side-effect.
>
> {code:java}
> CREATE TABLE t1 (id1 INT, val1 TEXT);
> CREATE TABLE t2 (id2 INT, val2 TEXT);
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (1, 2, 3);
> -- or
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (SELECT id2 FROM t2);{code}
>
> These seem to raise exceptions now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3186) IN expressions in UPDATE statements throws Exceptions
[
https://issues.apache.org/jira/browse/CALCITE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944088#comment-16944088
]
Haisheng Yuan commented on CALCITE-3186:
It has been fixed in 1.21.0, by commit
https://github.com/apache/calcite/commit/b8d24a6f7f2b4c7cb8093eea158f1fa00dbc6ffc.
> IN expressions in UPDATE statements throws Exceptions
> -
>
> Key: CALCITE-3186
> URL: https://issues.apache.org/jira/browse/CALCITE-3186
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.20.0
>Reporter: Pressenna
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The patch to get correlated sub-queries working in UPDATE statements had this
> side-effect.
>
> {code:java}
> CREATE TABLE t1 (id1 INT, val1 TEXT);
> CREATE TABLE t2 (id2 INT, val2 TEXT);
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (1, 2, 3);
> -- or
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (SELECT id2 FROM t2);{code}
>
> These seem to raise exceptions now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3186) IN expressions in UPDATE statements throws Exceptions
[
https://issues.apache.org/jira/browse/CALCITE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haisheng Yuan resolved CALCITE-3186.
Resolution: Invalid
Can't reproduce the bug, I believe it has been fixed, so closing it.
> IN expressions in UPDATE statements throws Exceptions
> -
>
> Key: CALCITE-3186
> URL: https://issues.apache.org/jira/browse/CALCITE-3186
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.20.0
>Reporter: Pressenna
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The patch to get correlated sub-queries working in UPDATE statements had this
> side-effect.
>
> {code:java}
> CREATE TABLE t1 (id1 INT, val1 TEXT);
> CREATE TABLE t2 (id2 INT, val2 TEXT);
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (1, 2, 3);
> -- or
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (SELECT id2 FROM t2);{code}
>
> These seem to raise exceptions now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944077#comment-16944077 ] Zhenqiu Huang commented on CALCITE-3349: [~hyuan] I sent an application email to d...@calcite.apache.org about two hour ago. Would you please try to assign it now. > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943804#comment-16943804 ] Haisheng Yuan commented on CALCITE-3349: [~ZhenqiuHuang] I am not able to assign the ticket to you, you have to apply for Calcite contributor first. But feel free to start your work. > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3349) Add Function DDL into SqlKind DDL enum
[ https://issues.apache.org/jira/browse/CALCITE-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-3349. Fix Version/s: 1.22.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/c09f293b4666b38e41a0724aab73565f8c61dbac. > Add Function DDL into SqlKind DDL enum > -- > > Key: CALCITE-3349 > URL: https://issues.apache.org/jira/browse/CALCITE-3349 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Zhenqiu Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently, Create Function, Drop Function are not added into SqlKind DDL > enum. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3346) Enable some ignored tests in RelOptRuleTests
[ https://issues.apache.org/jira/browse/CALCITE-3346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-3346. Fix Version/s: 1.22.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/87c5bd29364a23ab3c06f289ffce1300e26e8c46. > Enable some ignored tests in RelOptRuleTests > > > Key: CALCITE-3346 > URL: https://issues.apache.org/jira/browse/CALCITE-3346 > Project: Calcite > Issue Type: Test >Reporter: Wang Yanlin >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > When reading the code, i found that some ignored test case in > *RelOptRulesTest* actually can pass the test. Maybe we should try to enable > these tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3381) When generating SQL for BigQuery (via unparsing), convert Calcite SQL types to BigQuery types
[ https://issues.apache.org/jira/browse/CALCITE-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-3381. Fix Version/s: 1.22.0 Resolution: Fixed Fixed in https://github.com/apache/calcite/commit/30a0dd0a2edebdc08292b7d59988c60ccf754bed, thanks for the PR, [~amaliujia]! > When generating SQL for BigQuery (via unparsing), convert Calcite SQL types > to BigQuery types > - > > Key: CALCITE-3381 > URL: https://issues.apache.org/jira/browse/CALCITE-3381 > Project: Calcite > Issue Type: New Feature > Components: core >Affects Versions: 1.21.0 >Reporter: Kirill Kozlov >Priority: Minor > Labels: pull-request-available > Fix For: 1.22.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > When using `SqlImplementor.SimpleContext#toSql` with BigQuerySqlDialect on > Rel nodes containing `CAST` need to convert Calcite SqlTypeName to BigQuery > supported types > ([https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types]). > I believe this can be accomplished by Overriding > BigQuerySqlDialect#getCastSpec method. > Example: VARCHAR should become a STRING, BOOLEAN - BOOL, and so on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943729#comment-16943729
]
Haisheng Yuan commented on CALCITE-2970:
The analysis and solution looks valid at first glance, but it actually remove
the usage of abstract converter by directly adding sort operator for the inputs
of SortMergeJoin, thus it doesn't solve the issue fundamentally. Even if we
agree on the solution, what about the distribution? You have to create an
exchange operator on the inputs of MergeJoin, HashJoin and NestedLoopJoin, as
well as aggregate.
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-2970) Performance issue when enabling abstract converter for EnumerableConvertion
[
https://issues.apache.org/jira/browse/CALCITE-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943616#comment-16943616
]
Stamatis Zampetakis commented on CALCITE-2970:
--
Hey [~jinxing6...@126.com], thanks for working on this. I've seen that you did
some analysis to identify the cause of this issue. For the future, it is better
to include this kind of general information in the Jira rather than in the PR.
I think an important element that is missing from the analysis is the profiling
information. I do understand that the change in {{EnumerableMergeJoinRule}}
greatly improves the performance but how we can be sure that this rule is the
one (or the only one) that needs fixing. I suppose that you could equally
reduce the time by restricting matches in other rules.
> Performance issue when enabling abstract converter for EnumerableConvertion
> ---
>
> Key: CALCITE-2970
> URL: https://issues.apache.org/jira/browse/CALCITE-2970
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Haisheng Yuan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If we enable the use of abstract converter for {{EnumerableConvention}}, by
> making {{useAbstractConvertersForConversion}} return true,
> {{JDBCTest.testJoinManyWay}} will not complete.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
