[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861715#comment-16861715 ] Danny Chan commented on CALCITE-3118: - Okey, finally i believe this is a bug For this case, it happens with VolcanoRuleCall#getOperand0().solveOrder as [1, 0, 2], for a tree with asending match mode like: {code:java} Birel(0) / \ left(1) right(2){code} The VolcanoRuleCall firstly match right(2); then it start to match Birel(0), obviously it is the successor of right(2), but when we do check {code:java} final RelSubset input = (RelSubset) rel.getInput(previousOperand.ordinalInParent); List inputRels = input.set.getRelsFromAllSubsets(); if (!inputRels.contains(previous)) { continue; }{code} It got passed only because right(2) is the child of Birel(0), but as to the sloveOrder, right(2) expects to be the second child of Birel(0), but with this successfull match, it is the first child of Birel(0). > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched RelNode as a child *with the* *expected > child ordinal*. > However, there is a bug in this child ordinal check: > {noformat} > 333if (ascending && operand.childPolicy != > RelOptRuleOperandChildPolicy.UNORDERED) { > 334 // We know that the previous operand was *a* child of its parent, > 335 // but now check that it is the *correct* child. > 336 final RelSubset input = > 337 (RelSubset) rel.getInput(previousOperand.ordinalInParent); > 338 List inputRels = input.set.getRelsFromAllSubsets(); > 339 if (!inputRels.contains(previous)) { > 340continue; > 341 } > 342} > {noformat} > We intend to make sure that "previous" is in Subset "input". However line 338 > looked at RelNodes from the entire RelSet, rather than RelNodes only in > Subset "input". As a result, in some cases, incorrect parent is not skipped > as expected and is matched incorrectly. > The unit test case included is a case that triggers this bug, where a second > child RelNode incorrectly get matched as the first child of the parent > RelNode. > -- > Here's a detailed explanation of the test case that triggers the bug > We constructed a RelNode structure: > {noformat} > PhysBiRel > / \ > Subset1 Subset2 > | | > leftPhyrightPhy > {noformat} > Where PhysBiRel has two inputs: leftPhy and rightPhy, both are logically > equivalent, but with different traits (Convention in this test case), and > thus are in different subsets. > (Two Rels in two subsets in the same RelSet is a necessary condition to > trigger this bug. ) > A rule AssertOperandsDifferentRule is constructed as follows: > {noformat} > operand(PhysBiRel.class, > operand(PhysLeafRel.class, any()), > operand(PhysLeafRel.class, any())) > {noformat} > Obviously the correct match would be: > {noformat} > PhysBiRel > / \ > leftPhyrightPhy > {noformat} > However, with the bug, another match is also returned: > {noformat} > PhysBiRel > / \ > rightPhyrightPhy > {noformat} > *This is wrong because rightPhy is not PhysBiRel's first input at all, and > should not match as parent operand's first child.* > > Here's how the incorrect match occurs. > 1. When rightPhy of class PhysLeafRel is registered, we attempt to match it > to both the left and right child operands of AssertOperandsDifferentRule > above. This is expected. > 2. When matched to the right child operand, it eventually leads to the > correct match result above. > 3. When matched to the left child operand, it should have skipped/failed > matching the parent operand to PhysBiRel because rightPhy is *NOT* > PhysBiRel's first input. But because of the bug, the match succeeded. After > parent is matched, then it attempt to match the right child operand, and > again matched the rightPhy. As a result, both child operand end up matching > the same RelNode rightPhy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2995) Add the DAYNAME,MONTHNAME function
[ https://issues.apache.org/jira/browse/CALCITE-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861660#comment-16861660 ] Forward Xu commented on CALCITE-2995: - hi [~julianhyde] Thank you very much , I will modify him according to your suggestion. > Add the DAYNAME,MONTHNAME function > -- > > Key: CALCITE-2995 > URL: https://issues.apache.org/jira/browse/CALCITE-2995 > Project: Calcite > Issue Type: New Feature >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > MONTHNAME(date) > Returns the full name of the month for date. The language used for the name > is controlled by the system time config. > SQL: > {code:java} > SELECT MONTHNAME('2018-01-01') monthname;{code} > Result: > ||monthname|| > |January| > DAYNAME(date) > Returns the name of the weekday for date. The language used for the name is > controlled by the system time config. > SQL: > {code:java} > SELECT DAYNAME('2018-01-01') dayname;{code} > Result: > ||dayname|| > |Monday| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3124) Infinite rule matching when AggregateRemoveRule is enabled for SUM0
[ https://issues.apache.org/jira/browse/CALCITE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3124: --- Description: Make the following changes (uncomment return clause) to AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {noformat} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {noformat} was: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {noformat} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {noformat} > Infinite rule matching when AggregateRemoveRule is enabled for SUM0 > --- > > Key: CALCITE-3124 > URL: https://issues.apache.org/jira/browse/CALCITE-3124 > Project: Calcite > Issue Type: Bug >Reporter: Haisheng Yuan >Priority: Major > > Make the following changes (uncomment return clause) to AggregateRemoveRule, > the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule > matching. > {noformat} > --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { > if (aggregation.getKind() == SqlKind.SUM0) { > // Bail out for SUM0 to avoid potential infinite rule matching, > // because it may be generated by transforming SUM aggregate > // function to SUM0 and COUNT. > -return; > +//return; >} >final SqlSplittableAggFunction splitter = >Objects.requireNonNull( > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3124) Infinite rule matching when AggregateRemoveRule is enabled for SUM0
[ https://issues.apache.org/jira/browse/CALCITE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3124: --- Description: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {noformat} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {noformat} {code: java} public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} was: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code: java} public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} > Infinite rule matching when AggregateRemoveRule is enabled for SUM0 > --- > > Key: CALCITE-3124 > URL: https://issues.apache.org/jira/browse/CALCITE-3124 > Project: Calcite > Issue Type: Bug >Reporter: Haisheng Yuan >Priority: Major > > Make the following changes (uncomment return clause) to /AggregateRemoveRule, > the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule > matching. > {noformat} > --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { > if (aggregation.getKind() == SqlKind.SUM0) { > // Bail out for SUM0 to avoid potential infinite rule matching, > // because it may be generated by transforming SUM aggregate > // function to SUM0 and COUNT. > -return; > +//return; >} >final SqlSplittableAggFunction splitter = >Objects.requireNonNull( > {noformat} > {code: java} > public void onMatch(RelOptRuleCall call) { > if (aggregation.getKind() == SqlKind.SUM0) { > // Bail out for SUM0 to avoid potential infinite rule matching, > // because it may be generated by transforming SUM aggregate > // function to SUM0 and COUNT. > -return; > +//return; >} >final SqlSplittableAggFunction splitter = >Objects.requireNonNull( > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3124) Infinite rule matching when AggregateRemoveRule is enabled for SUM0
[ https://issues.apache.org/jira/browse/CALCITE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3124: --- Description: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {noformat} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {noformat} was: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {noformat} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {noformat} {code: java} public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} > Infinite rule matching when AggregateRemoveRule is enabled for SUM0 > --- > > Key: CALCITE-3124 > URL: https://issues.apache.org/jira/browse/CALCITE-3124 > Project: Calcite > Issue Type: Bug >Reporter: Haisheng Yuan >Priority: Major > > Make the following changes (uncomment return clause) to /AggregateRemoveRule, > the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule > matching. > {noformat} > --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { > if (aggregation.getKind() == SqlKind.SUM0) { > // Bail out for SUM0 to avoid potential infinite rule matching, > // because it may be generated by transforming SUM aggregate > // function to SUM0 and COUNT. > -return; > +//return; >} >final SqlSplittableAggFunction splitter = >Objects.requireNonNull( > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3124) Infinite rule matching when AggregateRemoveRule is enabled for SUM0
[ https://issues.apache.org/jira/browse/CALCITE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3124: --- Description: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code: java} public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} was: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code: java} @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} > Infinite rule matching when AggregateRemoveRule is enabled for SUM0 > --- > > Key: CALCITE-3124 > URL: https://issues.apache.org/jira/browse/CALCITE-3124 > Project: Calcite > Issue Type: Bug >Reporter: Haisheng Yuan >Priority: Major > > Make the following changes (uncomment return clause) to /AggregateRemoveRule, > the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule > matching. > {code: java} > public void onMatch(RelOptRuleCall call) { > if (aggregation.getKind() == SqlKind.SUM0) { > // Bail out for SUM0 to avoid potential infinite rule matching, > // because it may be generated by transforming SUM aggregate > // function to SUM0 and COUNT. > -return; > +//return; >} >final SqlSplittableAggFunction splitter = >Objects.requireNonNull( > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3124) Infinite rule matching when AggregateRemoveRule is enabled for SUM0
[ https://issues.apache.org/jira/browse/CALCITE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3124: --- Description: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code: java} @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} was: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code: java} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} > Infinite rule matching when AggregateRemoveRule is enabled for SUM0 > --- > > Key: CALCITE-3124 > URL: https://issues.apache.org/jira/browse/CALCITE-3124 > Project: Calcite > Issue Type: Bug >Reporter: Haisheng Yuan >Priority: Major > > Make the following changes (uncomment return clause) to /AggregateRemoveRule, > the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule > matching. > {code: java} > @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { > if (aggregation.getKind() == SqlKind.SUM0) { > // Bail out for SUM0 to avoid potential infinite rule matching, > // because it may be generated by transforming SUM aggregate > // function to SUM0 and COUNT. > -return; > +//return; >} >final SqlSplittableAggFunction splitter = >Objects.requireNonNull( > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3124) Infinite rule matching when AggregateRemoveRule is enabled for SUM0
[ https://issues.apache.org/jira/browse/CALCITE-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3124: --- Description: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code: java} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} was: Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code:diff} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} > Infinite rule matching when AggregateRemoveRule is enabled for SUM0 > --- > > Key: CALCITE-3124 > URL: https://issues.apache.org/jira/browse/CALCITE-3124 > Project: Calcite > Issue Type: Bug >Reporter: Haisheng Yuan >Priority: Major > > Make the following changes (uncomment return clause) to /AggregateRemoveRule, > the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule > matching. > {code: java} > --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java > @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { > if (aggregation.getKind() == SqlKind.SUM0) { > // Bail out for SUM0 to avoid potential infinite rule matching, > // because it may be generated by transforming SUM aggregate > // function to SUM0 and COUNT. > -return; > +//return; >} >final SqlSplittableAggFunction splitter = >Objects.requireNonNull( > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3124) Infinite rule matching when AggregateRemoveRule is enabled for SUM0
Haisheng Yuan created CALCITE-3124: -- Summary: Infinite rule matching when AggregateRemoveRule is enabled for SUM0 Key: CALCITE-3124 URL: https://issues.apache.org/jira/browse/CALCITE-3124 Project: Calcite Issue Type: Bug Reporter: Haisheng Yuan Make the following changes (uncomment return clause) to /AggregateRemoveRule, the test case {{JdbcTest.testHavingNot2}} won't complete due to infinite rule matching. {code:diff} --- a/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java +++ b/core/src/main/java/org/apache/calcite/rel/rules/AggregateRemoveRule.java @@ -102,7 +102,7 @@ public void onMatch(RelOptRuleCall call) { if (aggregation.getKind() == SqlKind.SUM0) { // Bail out for SUM0 to avoid potential infinite rule matching, // because it may be generated by transforming SUM aggregate // function to SUM0 and COUNT. -return; +//return; } final SqlSplittableAggFunction splitter = Objects.requireNonNull( {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[ https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861546#comment-16861546 ] Julian Hyde commented on CALCITE-3121: -- No one has made a case for why the behavior was better (or worse) before CALCITE-2798. Just arguments along the lines of "well, it works for me". > VolcanoPlanner hangs due to removing ORDER BY from sub-query > > > Key: CALCITE-3121 > URL: https://issues.apache.org/jira/browse/CALCITE-3121 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.19.0 >Reporter: Bohdan Kazydub >Assignee: Danny Chan >Priority: Blocker > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After the fix for CALCITE-2798 some queries hang during planning in > VolcanoPlanner (similar issue was reported in CALCITE-2223). > Here is a test case which should be added to the {{RelOptRulesTest}} class: > {code:java} > @Test public void testSubQueryWithOrderByHang() { > String sql = "select n.n_regionkey from ( select * from " > + "( select * from sales.customer) t order by t.n_regionkey) n where > n.n_nationkey >1 "; > VolcanoPlanner planner = new VolcanoPlanner(null, null); > planner.addRelTraitDef(ConventionTraitDef.INSTANCE); > Tester dynamicTester = createDynamicTester().withDecorrelation(true) > .withClusterFactory( > relOptCluster -> RelOptCluster.create(planner, > relOptCluster.getRexBuilder())); > RelRoot root = dynamicTester.convertSqlToRel(sql); > String planBefore = NL + RelOptUtil.toString(root.rel); > getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore); > PushProjector.ExprCondition exprCondition = expr -> { > if (expr instanceof RexCall) { > RexCall call = (RexCall) expr; > return > "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT)); > } > return false; > }; > RuleSet ruleSet = > RuleSets.ofList( > FilterProjectTransposeRule.INSTANCE, > FilterMergeRule.INSTANCE, > ProjectMergeRule.INSTANCE, > new ProjectFilterTransposeRule(Project.class, Filter .class, > RelFactories.LOGICAL_BUILDER, exprCondition), > EnumerableRules.ENUMERABLE_PROJECT_RULE, > EnumerableRules.ENUMERABLE_FILTER_RULE, > EnumerableRules.ENUMERABLE_SORT_RULE, > EnumerableRules.ENUMERABLE_LIMIT_RULE, > EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE); > Program program = Programs.of(ruleSet); > RelTraitSet toTraits = > root.rel.getCluster().traitSet() > .replace(0, EnumerableConvention.INSTANCE); > RelNode relAfter = program.run(planner, root.rel, toTraits, > Collections.emptyList(), Collections.emptyList()); > String planAfter = NL + RelOptUtil.toString(relAfter); > getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter); > } > {code} > Please note that if {{LIMIT }} is added to the sub-query with order by > (so it is not removed due to the fix for CALCITE-2798) the test succeeds. > Though the issue with hanging is more general, I think that if it wouldn't be > fixed, the fix for CALCITE-2798 should be reverted to reduce cases when > planner may hang. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861388#comment-16861388 ] Khai Tran commented on CALCITE-3122: [~julianhyde], Here is the pull request: [https://github.com/apache/calcite/pull/1265] (mvn install passed). In calcire-core, I added some bug fixes, several new simple APIs, and also some HACKs like reset planner for relNode to make it works. Otherwise, implementation is self-contained in piglet module. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3122: Labels: pull-request-available (was: ) > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > Labels: pull-request-available > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3087) AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its Mapping breaks ordering
[ https://issues.apache.org/jira/browse/CALCITE-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861269#comment-16861269 ] Haisheng Yuan commented on CALCITE-3087: [~donnyzone] Can you log the issue you described and provide a test case to reproduce the bug? > AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its > Mapping breaks ordering > --- > > Key: CALCITE-3087 > URL: https://issues.apache.org/jira/browse/CALCITE-3087 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Haisheng Yuan >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Add the following test to MaterializationTest: > {code:java} > @Test public void testAggregateGroupSetsRollUp0() { > checkMaterialize( > "select \"empid\", \"deptno\", count(*) as c, sum(\"salary\") as s > from \"emps\" " > + "group by \"empid\", \"deptno\"", > "select count(*) + 1 as c, \"deptno\" from \"emps\" group by > cube(\"deptno\", \"empid\")", > HR_FKUK_MODEL, > CalciteAssert.checkResultContains( > "EnumerableCalc(expr#0..2=[{inputs}], expr#3=[1], " > + "expr#4=[+($t2, $t3)], C=[$t4], deptno=[$t1])\n" > + " EnumerableAggregate(group=[{0, 1}], groups=[[{0, 1}, > {0}, {1}, {}]], agg#0=[$SUM0($2)])\n" > + "EnumerableTableScan(table=[[hr, m0]])")); > } > {code} > We got error: > {code:java} > Caused by: java.lang.IllegalArgumentException: must be sorted: [{0, 1}, {1}, > {0}, {}] > at > org.apache.calcite.rel.core.Aggregate$Group.induce(Aggregate.java:472) > at > org.apache.calcite.rel.mutable.MutableAggregate.getGroupType(MutableAggregate.java:83) > at > org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1281) > at > org.apache.calcite.plan.SubstitutionVisitor$AggregateOnProjectToAggregateUnifyRule.apply(SubstitutionVisitor.java:1345) > at > org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:531) > at > org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:466) > at > org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56) > at > org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200) > at > org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:348) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:584) > {code} > The reason is that Mappings.apply2 doesn't sort. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3087) AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its Mapping breaks ordering
[ https://issues.apache.org/jira/browse/CALCITE-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-3087. Resolution: Fixed Fix Version/s: 1.20.0 Fixed in https://github.com/apache/calcite/commit/a3c56be7bccc58859524ba39e5b30b7078f97d00, thanks for the PR, [~donnyzone]! > AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its > Mapping breaks ordering > --- > > Key: CALCITE-3087 > URL: https://issues.apache.org/jira/browse/CALCITE-3087 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Haisheng Yuan >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Add the following test to MaterializationTest: > {code:java} > @Test public void testAggregateGroupSetsRollUp0() { > checkMaterialize( > "select \"empid\", \"deptno\", count(*) as c, sum(\"salary\") as s > from \"emps\" " > + "group by \"empid\", \"deptno\"", > "select count(*) + 1 as c, \"deptno\" from \"emps\" group by > cube(\"deptno\", \"empid\")", > HR_FKUK_MODEL, > CalciteAssert.checkResultContains( > "EnumerableCalc(expr#0..2=[{inputs}], expr#3=[1], " > + "expr#4=[+($t2, $t3)], C=[$t4], deptno=[$t1])\n" > + " EnumerableAggregate(group=[{0, 1}], groups=[[{0, 1}, > {0}, {1}, {}]], agg#0=[$SUM0($2)])\n" > + "EnumerableTableScan(table=[[hr, m0]])")); > } > {code} > We got error: > {code:java} > Caused by: java.lang.IllegalArgumentException: must be sorted: [{0, 1}, {1}, > {0}, {}] > at > org.apache.calcite.rel.core.Aggregate$Group.induce(Aggregate.java:472) > at > org.apache.calcite.rel.mutable.MutableAggregate.getGroupType(MutableAggregate.java:83) > at > org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1281) > at > org.apache.calcite.plan.SubstitutionVisitor$AggregateOnProjectToAggregateUnifyRule.apply(SubstitutionVisitor.java:1345) > at > org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:531) > at > org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:466) > at > org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56) > at > org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200) > at > org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:348) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:584) > {code} > The reason is that Mappings.apply2 doesn't sort. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2995) Add the DAYNAME,MONTHNAME function
[ https://issues.apache.org/jira/browse/CALCITE-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861258#comment-16861258 ] Julian Hyde commented on CALCITE-2995: -- I saw some new changes. Looks like an improvement: * Does forbiddenApis give errors when you call {{Locale.getDefault()}}? I thought it should. * I don't think you should call {{Locale.getLanguage()}} to convert it to a string. * In SqlFunctions, why do we need {{monthName(int date)}}? Library code must not use default locale. There should be only one line of code that reads the default locale. Everywhere else must use locale from the current context. > Add the DAYNAME,MONTHNAME function > -- > > Key: CALCITE-2995 > URL: https://issues.apache.org/jira/browse/CALCITE-2995 > Project: Calcite > Issue Type: New Feature >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > MONTHNAME(date) > Returns the full name of the month for date. The language used for the name > is controlled by the system time config. > SQL: > {code:java} > SELECT MONTHNAME('2018-01-01') monthname;{code} > Result: > ||monthname|| > |January| > DAYNAME(date) > Returns the name of the weekday for date. The language used for the name is > controlled by the system time config. > SQL: > {code:java} > SELECT DAYNAME('2018-01-01') dayname;{code} > Result: > ||dayname|| > |Monday| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[ https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861249#comment-16861249 ] Volodymyr Vysotskyi commented on CALCITE-3121: -- If there are no objections, I'll merge the PR soon. > VolcanoPlanner hangs due to removing ORDER BY from sub-query > > > Key: CALCITE-3121 > URL: https://issues.apache.org/jira/browse/CALCITE-3121 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.19.0 >Reporter: Bohdan Kazydub >Assignee: Danny Chan >Priority: Blocker > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > After the fix for CALCITE-2798 some queries hang during planning in > VolcanoPlanner (similar issue was reported in CALCITE-2223). > Here is a test case which should be added to the {{RelOptRulesTest}} class: > {code:java} > @Test public void testSubQueryWithOrderByHang() { > String sql = "select n.n_regionkey from ( select * from " > + "( select * from sales.customer) t order by t.n_regionkey) n where > n.n_nationkey >1 "; > VolcanoPlanner planner = new VolcanoPlanner(null, null); > planner.addRelTraitDef(ConventionTraitDef.INSTANCE); > Tester dynamicTester = createDynamicTester().withDecorrelation(true) > .withClusterFactory( > relOptCluster -> RelOptCluster.create(planner, > relOptCluster.getRexBuilder())); > RelRoot root = dynamicTester.convertSqlToRel(sql); > String planBefore = NL + RelOptUtil.toString(root.rel); > getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore); > PushProjector.ExprCondition exprCondition = expr -> { > if (expr instanceof RexCall) { > RexCall call = (RexCall) expr; > return > "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT)); > } > return false; > }; > RuleSet ruleSet = > RuleSets.ofList( > FilterProjectTransposeRule.INSTANCE, > FilterMergeRule.INSTANCE, > ProjectMergeRule.INSTANCE, > new ProjectFilterTransposeRule(Project.class, Filter .class, > RelFactories.LOGICAL_BUILDER, exprCondition), > EnumerableRules.ENUMERABLE_PROJECT_RULE, > EnumerableRules.ENUMERABLE_FILTER_RULE, > EnumerableRules.ENUMERABLE_SORT_RULE, > EnumerableRules.ENUMERABLE_LIMIT_RULE, > EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE); > Program program = Programs.of(ruleSet); > RelTraitSet toTraits = > root.rel.getCluster().traitSet() > .replace(0, EnumerableConvention.INSTANCE); > RelNode relAfter = program.run(planner, root.rel, toTraits, > Collections.emptyList(), Collections.emptyList()); > String planAfter = NL + RelOptUtil.toString(relAfter); > getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter); > } > {code} > Please note that if {{LIMIT }} is added to the sub-query with order by > (so it is not removed due to the fix for CALCITE-2798) the test succeeds. > Though the issue with hanging is more general, I think that if it wouldn't be > fixed, the fix for CALCITE-2798 should be reverted to reduce cases when > planner may hang. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
[ https://issues.apache.org/jira/browse/CALCITE-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861217#comment-16861217 ] Julian Hyde edited comment on CALCITE-3119 at 6/11/19 5:07 PM: --- * Remove, don't deprecate, unused methods in {{EnumerableDefaults.java}}. That class is not a public API. * The method {{toLinq4jCorrelateJoinType}} was only just created, since 1.19. Remove it. * I considered just removing {{enum CorrelateJoinType}}, but instead I think we should keep it for one minor release; I think we should mark it 'deprecated to be removed before 1.21'. was (Author: julianhyde): Remove, don't deprecate, unused methods in EnumerableDefaults.java. That class is not public. > Deprecate Linq4j CorrelateJoinType (in favor of JoinType) > - > > Key: CALCITE-3119 > URL: https://issues.apache.org/jira/browse/CALCITE-3119 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a > result, we have right now two enums for (almost) the same purpose: > CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, > SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all > types of join, mirroring core's JoinRelType). > The idea is to flag CorrelateJoinType as deprecated, and replace all its > usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
[ https://issues.apache.org/jira/browse/CALCITE-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861217#comment-16861217 ] Julian Hyde commented on CALCITE-3119: -- Remove, don't deprecate, unused methods in EnumerableDefaults.java. That class is not public. > Deprecate Linq4j CorrelateJoinType (in favor of JoinType) > - > > Key: CALCITE-3119 > URL: https://issues.apache.org/jira/browse/CALCITE-3119 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a > result, we have right now two enums for (almost) the same purpose: > CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, > SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all > types of join, mirroring core's JoinRelType). > The idea is to flag CorrelateJoinType as deprecated, and replace all its > usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
[ https://issues.apache.org/jira/browse/CALCITE-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861038#comment-16861038 ] Michael Mior commented on CALCITE-3119: --- [~julianhyde] PR looks good to me but do you mind giving it a quick scan? > Deprecate Linq4j CorrelateJoinType (in favor of JoinType) > - > > Key: CALCITE-3119 > URL: https://issues.apache.org/jira/browse/CALCITE-3119 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a > result, we have right now two enums for (almost) the same purpose: > CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, > SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all > types of join, mirroring core's JoinRelType). > The idea is to flag CorrelateJoinType as deprecated, and replace all its > usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)