[jira] [Commented] (CALCITE-2804) Druid adapter: Cast does not work when casting to timestamp
[
https://issues.apache.org/jira/browse/CALCITE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859112#comment-16859112
]
Hongze Zhang commented on CALCITE-2804:
---
[~jszeluga] - Do you have any comments on the issue?
> Druid adapter: Cast does not work when casting to timestamp
> ---
>
> Key: CALCITE-2804
> URL: https://issues.apache.org/jira/browse/CALCITE-2804
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Justin Szeluga
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When trying to execute a simple statement of
> {code:java}
> select cast("__time" as timestamp) from my_datasource limit 10;{code}
> the query fails with a 500 error
>
> {code:java}
> Caused by: java.io.IOException: Server returned HTTP response code: 500 for
> URL: http://localhost:58082/druid/v2/?pretty
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
> at org.apache.calcite.runtime.HttpUtils.executeMethod(HttpUtils.java:137)
> at org.apache.calcite.runtime.HttpUtils.post(HttpUtils.java:110)
> at
> org.apache.calcite.adapter.druid.DruidConnectionImpl.request(DruidConnectionImpl.java:109)
> ... 29 more
> {code}
> because the JSON generated is incorrect. Here is the generated JSON
>
>
> {code:java}
> {
> "queryType": "scan",
> "dataSource": "my_datasource",
> "intervals": [
> "1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"
> ],
> "virtualColumns": [
> {
> "type": "expression",
> "name": "vc",
> "expression":
> "timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'','UTC')",
> "outputType": "LONG"
> }
> ],
> "columns": [
> "vc"
> ],
> "resultFormat": "compactedList",
> "limit": 10
> }
> {code}
> There are two problems. -First, the "__time" dimension is not formatted
> correctly in timestamp_format. The escaped quotes need to be removed.-
> Also, the call to timestamp_parse is incorrect. If a timezone is going to be
> provided then a date time format needs to be provided as well. Currently, it
> is only empty string
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2804) Druid adapter: Cast does not work when casting to timestamp
[
https://issues.apache.org/jira/browse/CALCITE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859110#comment-16859110
]
Hongze Zhang commented on CALCITE-2804:
---
It is weird. I noticed the quote difference but I see in code we already used
single quote[1]. So I thought it was related to display of the test output.
Regarding "__time". I believe we still need the double quote around column
name. See discussion[2] in PR. So JIRA description about this point is no
longer valid.
I'll try to install the IT VMs (it may take some time), but not sure it can
work as expected, since I remember that I was blocked by GFW when I install
them last time. :(
[1]
[https://github.com/apache/calcite/pull/1014/files#diff-88c776102776d0dfedaf74e4be27854fR590]
[2] [https://github.com/apache/calcite/pull/1014#discussion_r256784061]
> Druid adapter: Cast does not work when casting to timestamp
> ---
>
> Key: CALCITE-2804
> URL: https://issues.apache.org/jira/browse/CALCITE-2804
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Justin Szeluga
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When trying to execute a simple statement of
> {code:java}
> select cast("__time" as timestamp) from my_datasource limit 10;{code}
> the query fails with a 500 error
>
> {code:java}
> Caused by: java.io.IOException: Server returned HTTP response code: 500 for
> URL: http://localhost:58082/druid/v2/?pretty
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
> at org.apache.calcite.runtime.HttpUtils.executeMethod(HttpUtils.java:137)
> at org.apache.calcite.runtime.HttpUtils.post(HttpUtils.java:110)
> at
> org.apache.calcite.adapter.druid.DruidConnectionImpl.request(DruidConnectionImpl.java:109)
> ... 29 more
> {code}
> because the JSON generated is incorrect. Here is the generated JSON
>
>
> {code:java}
> {
> "queryType": "scan",
> "dataSource": "my_datasource",
> "intervals": [
> "1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"
> ],
> "virtualColumns": [
> {
> "type": "expression",
> "name": "vc",
> "expression":
> "timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'','UTC')",
> "outputType": "LONG"
> }
> ],
> "columns": [
> "vc"
> ],
> "resultFormat": "compactedList",
> "limit": 10
> }
> {code}
> There are two problems. -First, the "__time" dimension is not formatted
> correctly in timestamp_format. The escaped quotes need to be removed.-
> Also, the call to timestamp_parse is incorrect. If a timezone is going to be
> provided then a date time format needs to be provided as well. Currently, it
> is only empty string
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859104#comment-16859104
]
Danny Chan commented on CALCITE-3121:
-
It seems that after some planning rules matched, In ProjectMergeRule, the top
project reference a wrong RelSubSet as input, so it just add new project over
and over again, and never ends up.
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858693#comment-16858693
]
Botong Huang edited comment on CALCITE-3118 at 6/8/19 4:13 AM:
---
{quote}It is not wrong provided the children are identical
{quote}
The leftPhy and rightPhy are logically equivalent, but has different traits and
are in different subsets.
I think you missed my point. The structure the _AssertOperandsDifferentRule_ is
looking for is:
A
/ \
B C
where *B is A's first child*. What the match returned was:
Rel1
/ \
Rel2.1 Rel2.2
where Rel2.1 is *NOT* Rel1's first child.
was (Author: botong):
{quote}It is not wrong provided the children are identical
{quote}
I think you missed my point. The structure the _AssertOperandsDifferentRule_ is
looking for is:
A
/ \
B C
where *B is A's first child*. What the match returned was:
Rel1
/ \
Rel2.1 Rel2.2
where Rel2.1 is *NOT* Rel1's first child.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where a second child
> RelNode get matched as first child of the parent RelNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3121:
---
Assignee: Danny Chan
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859078#comment-16859078
]
Danny Chan edited comment on CALCITE-3118 at 6/8/19 2:20 AM:
-
I'm -1 on this fix.
Just like you said, _AssertOperandsDifferentRule's_ match pattern is:
{code:java}
PhysBiRel
/ \
leftPhy rightPhy{code}
Firstly you construct a RelNode tree like what the pattern describes, then you
register that the `leftPhy` and `rightPhy` are equivalent. It's not that
surprising you got a tree like:
{code:java}
PhysBiRel
/ \
rightPhy rightPhy
{code}
This tree also satisfied _AssertOperandsDifferentRule_ patttern.
This is Volcano Planner so the possibility space boost. BTW, the tree you build
manually would still be matched.
We have no contract that the `RelOptRule` must match same convention nodes
which would reduce many promotion possibilities.
was (Author: danny0405):
I'm -1 on this fix.
Just like you said, _AssertOperandsDifferentRule's_ match pattern is:
{code:java}
PhysBiRel
/ \
leftPhy rightPhy{code}
Firstly you construct a RelNode tree like what the pattern describes, then you
register that the `leftPhy` and `rightPhy` are equivalent. It's not that
surprising you got a tree like:
{code:java}
PhysBiRel
/ \
rightPhy rightPhy
{code}
This tree also satisfied _AssertOperandsDifferentRule_ patttern.
This is Volcano Planner so the possibility space boost. BTW, the tree you build
manually would still be matched.
We have contract that the `RelOptRule` must match same convention nodes which
would reduce many promotion possibilities.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where a second child
> RelNode get matched as first child of the parent RelNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859078#comment-16859078
]
Danny Chan commented on CALCITE-3118:
-
I'm -1 on this fix.
Just like you said, _AssertOperandsDifferentRule's_ match pattern is:
{code:java}
PhysBiRel
/ \
leftPhy rightPhy{code}
Firstly you construct a RelNode tree like what the pattern describes, then you
register that the `leftPhy` and `rightPhy` are equivalent. It's not that
surprising you got a tree like:
{code:java}
PhysBiRel
/ \
rightPhy rightPhy
{code}
This tree also satisfied _AssertOperandsDifferentRule_ patttern.
This is Volcano Planner so the possibility space boost. BTW, the tree you build
manually would still be matched.
We have contract that the `RelOptRule` must match same convention nodes which
would reduce many promotion possibilities.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where a second child
> RelNode get matched as first child of the parent RelNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-2822) Allow multi-join rules to be used with any project/filter
[ https://issues.apache.org/jira/browse/CALCITE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis resolved CALCITE-2822. -- Resolution: Fixed Fix Version/s: 1.20.0 Fixed in [0cb51c64715dd45f8bbcd533cc9096b4bb7b70af|https://github.com/apache/calcite/tree/0cb51c64715dd45f8bbcd533cc9096b4bb7b70af]. Thanks for the PR [~siddteotia]! > Allow multi-join rules to be used with any project/filter > - > > Key: CALCITE-2822 > URL: https://issues.apache.org/jira/browse/CALCITE-2822 > Project: Calcite > Issue Type: Task > Components: core >Reporter: Siddharth Teotia >Priority: Minor > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 5h 50m > Remaining Estimate: 0h > > Allow multi join rules to be used with any subclass of Project or Filter > instead of having just LogicalProject and LogicalFilter -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2804) Druid adapter: Cast does not work when casting to timestamp
[
https://issues.apache.org/jira/browse/CALCITE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859027#comment-16859027
]
Stamatis Zampetakis commented on CALCITE-2804:
--
Two differences that I see are:
* a few \ that are missing in the actual plan
* '-MM-dd starts with single quote instead of \"-MM-dd with double
quotes
In the description of this JIRA it says the following:
{quote}First, the "__time" dimension is not formatted correctly in
timestamp_format. The escaped quotes need to be removed.
{quote}
so I don't feel comfortable just changing the plan. Something seems to be off.
> Druid adapter: Cast does not work when casting to timestamp
> ---
>
> Key: CALCITE-2804
> URL: https://issues.apache.org/jira/browse/CALCITE-2804
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Justin Szeluga
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When trying to execute a simple statement of
> {code:java}
> select cast("__time" as timestamp) from my_datasource limit 10;{code}
> the query fails with a 500 error
>
> {code:java}
> Caused by: java.io.IOException: Server returned HTTP response code: 500 for
> URL: http://localhost:58082/druid/v2/?pretty
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
> at org.apache.calcite.runtime.HttpUtils.executeMethod(HttpUtils.java:137)
> at org.apache.calcite.runtime.HttpUtils.post(HttpUtils.java:110)
> at
> org.apache.calcite.adapter.druid.DruidConnectionImpl.request(DruidConnectionImpl.java:109)
> ... 29 more
> {code}
> because the JSON generated is incorrect. Here is the generated JSON
>
>
> {code:java}
> {
> "queryType": "scan",
> "dataSource": "my_datasource",
> "intervals": [
> "1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"
> ],
> "virtualColumns": [
> {
> "type": "expression",
> "name": "vc",
> "expression":
> "timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'','UTC')",
> "outputType": "LONG"
> }
> ],
> "columns": [
> "vc"
> ],
> "resultFormat": "compactedList",
> "limit": 10
> }
> {code}
> There are two problems. -First, the "__time" dimension is not formatted
> correctly in timestamp_format. The escaped quotes need to be removed.-
> Also, the call to timestamp_parse is incorrect. If a timezone is going to be
> provided then a date time format needs to be provided as well. Currently, it
> is only empty string
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3123) In RelBuilder, eliminate duplicate aggregate calls
Julian Hyde created CALCITE-3123:
Summary: In RelBuilder, eliminate duplicate aggregate calls
Key: CALCITE-3123
URL: https://issues.apache.org/jira/browse/CALCITE-3123
Project: Calcite
Issue Type: Bug
Reporter: Julian Hyde
In {{RelBuilder}}, eliminate duplicate aggregate calls. For example, if you
call {{RelBuilder.aggregate}} with {{sum($1)}}, {{sum($2)}}, {{count($1)}},
{{sum($1)}} then it would create an {{Aggregate}} with 3 aggregate calls
(because {{sum($1)}} occurs twice), then add a {{Project [$1, $2, $3, $1]}}.
The {{Project}} is necessary so that you get the number of output fields that
you are expecting.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858983#comment-16858983 ] Julian Hyde commented on CALCITE-3122: -- +1 Let's get a PR first, then we can review and iterate. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858962#comment-16858962 ] Khai Tran commented on CALCITE-3122: Thanks a lot, [~julianhyde]. This is my first time contributing to open source, so really excited. I have tests for all Pig operators and expressions using EMP/DEPT tables, but those tests could not cover expressions on nested data types due to the flatten schemas. So nested data types are tested on actual LinkedIn pig scripts, which cannot be replicated here. Let me create a pull request first and hope to get more feedback from the community. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3114) Support websocket transport
[ https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858947#comment-16858947 ] Julian Hyde commented on CALCITE-3114: -- What are the performance benefits? Based on [this|https://javaee.github.io/tutorial/websocket.html] it seems that the main benefit is full duplex (i.e. server can initiate communication to the client). I don't think that is very important for JDBC. Like HTTP, it's still over TCP, so I imagine the latency and throughput would be similar to HTTP. > Support websocket transport > --- > > Key: CALCITE-3114 > URL: https://issues.apache.org/jira/browse/CALCITE-3114 > Project: Calcite > Issue Type: Wish >Reporter: Leo Tu >Priority: Minor > > it would be greatly improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858915#comment-16858915 ] Julian Hyde commented on CALCITE-3122: -- I'm very excited about this. It makes it possible for people to run Pig scripts on any platform supported by Calcite. I know there are lot of organizations that use Pig but would love some flexibility as to which engine they use. I'm hoping that the contribution comes with a comprehensive set of tests, so that we can continue to maintain it. Maybe 1,000 isn't necessary, but enough to test all common combinations of Pig's features. The latest branch is [master|https://github.com/apache/calcite/commits/master], as always. If you are able to rebase onto a relatively recent version of master then we can start reviewing, and consider for inclusion in 1.21. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858907#comment-16858907 ] Haisheng Yuan commented on CALCITE-3122: The latest release is https://github.com/apache/calcite/tree/calcite-1.19.0 But we will have 1.20.0 rc0 next week. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858903#comment-16858903 ] Khai Tran commented on CALCITE-3122: Thanks a lot for your support, [~hyuan]. BTW, our internal repo is branched from Calcite-14, do you know what is the latest branch I can rebase my changes to before creating the pull request? > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858893#comment-16858893 ] Khai Tran edited comment on CALCITE-3122 at 6/7/19 6:43 PM: Yes, I extended the RelBuider to create logical operators out of Pig logical plans. No physical operators generated. It's a complete solution with several thousands lines of code. was (Author: khaitran): Yes, I extended the RelBuider to create logical operators out of Pig logical plans. No physical operator generated. It's a complete solution with several thousands lines of code. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858893#comment-16858893 ] Khai Tran edited comment on CALCITE-3122 at 6/7/19 6:43 PM: Yes, I extended the RelBuider to create logical operators out of Pig logical plans. No physical operator generated. It's a complete solution with several thousands lines of code. was (Author: khaitran): Yes, I extended the RelBuider to create logical operators out of Pig logical plans. Not physical operator generated. It's a complete solution with several thousands lines of code. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858896#comment-16858896 ] Haisheng Yuan commented on CALCITE-3122: Great. Look forward to your patch. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858893#comment-16858893 ] Khai Tran commented on CALCITE-3122: Yes, I extended the RelBuider to create logical operators out of Pig logical plans. Not physical operator generated. It's a complete solution with several thousands lines of code. > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3122) Converte Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3122: --- Summary: Converte Pig Latin scripts into Calcite logical plan (was: Contribution request: converting Pig Latin scripts into Calcite logical plan ) > Converte Pig Latin scripts into Calcite logical plan > - > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3122) Convert Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan updated CALCITE-3122: --- Summary: Convert Pig Latin scripts into Calcite logical plan (was: Converte Pig Latin scripts into Calcite logical plan ) > Convert Pig Latin scripts into Calcite logical plan > > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3122) Contribution request: converting Pig Latin scripts into Calcite logical plan
[ https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1685#comment-1685 ] Haisheng Yuan commented on CALCITE-3122: Will the Pig RelBuilder build logical operator instead of physical operator? I remember I made a change, all except pig test passed, because the pig relbuilder builds physical operator directly through the pig script, which annoys me a lot. > Contribution request: converting Pig Latin scripts into Calcite logical plan > - > > Key: CALCITE-3122 > URL: https://issues.apache.org/jira/browse/CALCITE-3122 > Project: Calcite > Issue Type: New Feature > Components: core, piglet >Reporter: Khai Tran >Priority: Major > > We create an internal Calcite repo at LinkedIn and develop APIs to parse any > Pig Latin scripts into Calcite logical plan. The code was tested in nearly > ~1000 Pig scripts written at LinkedIn. > Changes: > 1. piglet: main conversion code live there, include: > * APIs to convert any Pig scripts into RelNode plans or SQL statements > * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan > (DAGs) > * Convert Pig schemas into RelDatatype > * Traverse through Pig expression plan and convert Pig expressions into > RexNodes > * Map some basic Pig UDFs to Calcite SQL operators > * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both > Java and Python > * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes > to RelNodes > * Have an optimizer rule to optimize Pig group/cogroup into Aggregate > operators > 2. core: > * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL > * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858884#comment-16858884
]
Haisheng Yuan commented on CALCITE-3121:
This is stack trace:
{code:java}
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:71)
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225)
at
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:132)
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225)
at
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:71)
at GeneratedMetadataHandler_RowCount.getRowCount_$(Unknown Source)
at GeneratedMetadataHandler_RowCount.getRowCount(Unknown Source)
at
org.apache.calcite.rel.metadata.RelMetadataQuery.getRowCount(RelMetadataQuery.java:225)
at
org.apache.calcite.rel.metadata.RelMdRowCount.getRowCount(RelMdRowCount.java:132)
{code}
Cyclic metadata fetching.
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3122) Contribution request: converting Pig Latin scripts into Calcite logical plan
Khai Tran created CALCITE-3122: -- Summary: Contribution request: converting Pig Latin scripts into Calcite logical plan Key: CALCITE-3122 URL: https://issues.apache.org/jira/browse/CALCITE-3122 Project: Calcite Issue Type: New Feature Components: core, piglet Reporter: Khai Tran We create an internal Calcite repo at LinkedIn and develop APIs to parse any Pig Latin scripts into Calcite logical plan. The code was tested in nearly ~1000 Pig scripts written at LinkedIn. Changes: 1. piglet: main conversion code live there, include: * APIs to convert any Pig scripts into RelNode plans or SQL statements * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan (DAGs) * Convert Pig schemas into RelDatatype * Traverse through Pig expression plan and convert Pig expressions into RexNodes * Map some basic Pig UDFs to Calcite SQL operators * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both Java and Python * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes to RelNodes * Have an optimizer rule to optimize Pig group/cogroup into Aggregate operators 2. core: * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL * Other minor changes in a few other classes to make Pig to Calcite works -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858874#comment-16858874
]
Haisheng Yuan commented on CALCITE-3121:
The issue can be reproduced by the following query:
{code:java}
String sql = "select n.n_regionkey from ( select * from "
+ "( select * from sales.customer) t) n where n.n_nationkey >1 ";
{code}
There is nothing wrong with the fix in CALCITE-2798.
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haisheng Yuan updated CALCITE-3121:
---
Comment: was deleted
(was: The initial planBefore is
{code:java}
LogicalProject(N_REGIONKEY=[ITEM($0, 'N_REGIONKEY')])
LogicalFilter(condition=[>(ITEM($0, 'N_NATIONKEY'), 1)])
LogicalProject(**=[$0])
LogicalTableScan(table=[[CATALOG, SALES, CUSTOMER]])
{code}
It seems that LogicalProject(**=[$0]) is not expanded properly.
)
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858870#comment-16858870
]
Haisheng Yuan commented on CALCITE-3121:
The initial planBefore is
{code:java}
LogicalProject(N_REGIONKEY=[ITEM($0, 'N_REGIONKEY')])
LogicalFilter(condition=[>(ITEM($0, 'N_NATIONKEY'), 1)])
LogicalProject(**=[$0])
LogicalTableScan(table=[[CATALOG, SALES, CUSTOMER]])
{code}
It seems that LogicalProject(**=[$0]) is not expanded properly.
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
Bohdan Kazydub created CALCITE-3121:
---
Summary: VolcanoPlanner hangs due to removing ORDER BY from
sub-query
Key: CALCITE-3121
URL: https://issues.apache.org/jira/browse/CALCITE-3121
Project: Calcite
Issue Type: Bug
Reporter: Bohdan Kazydub
After the fix for CALCITE-2798 some queries hang during planning in
VolcanoPlanner (similar issue was reported in CALCITE-2223).
Here is a test case which should be added to the {{RelOptRulesTest}} class:
{code:java}
@Test public void testSubQueryWithOrderByHang() {
String sql = "select n.n_regionkey from ( select * from "
+ "( select * from sales.customer) t order by t.n_regionkey) n where
n.n_nationkey >1 ";
VolcanoPlanner planner = new VolcanoPlanner(null, null);
planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
Tester dynamicTester = createDynamicTester().withDecorrelation(true)
.withClusterFactory(
relOptCluster -> RelOptCluster.create(planner,
relOptCluster.getRexBuilder()));
RelRoot root = dynamicTester.convertSqlToRel(sql);
String planBefore = NL + RelOptUtil.toString(root.rel);
getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
PushProjector.ExprCondition exprCondition = expr -> {
if (expr instanceof RexCall) {
RexCall call = (RexCall) expr;
return
"item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
}
return false;
};
RuleSet ruleSet =
RuleSets.ofList(
FilterProjectTransposeRule.INSTANCE,
FilterMergeRule.INSTANCE,
ProjectMergeRule.INSTANCE,
new ProjectFilterTransposeRule(Project.class, Filter .class,
RelFactories.LOGICAL_BUILDER, exprCondition),
EnumerableRules.ENUMERABLE_PROJECT_RULE,
EnumerableRules.ENUMERABLE_FILTER_RULE,
EnumerableRules.ENUMERABLE_SORT_RULE,
EnumerableRules.ENUMERABLE_LIMIT_RULE,
EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
Program program = Programs.of(ruleSet);
RelTraitSet toTraits =
root.rel.getCluster().traitSet()
.replace(0, EnumerableConvention.INSTANCE);
RelNode relAfter = program.run(planner, root.rel, toTraits,
Collections.emptyList(), Collections.emptyList());
String planAfter = NL + RelOptUtil.toString(relAfter);
getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
}
{code}
Please note that if {{LIMIT }} is added to the sub-query with order by (so
it is not removed due to the fix for CALCITE-2798) the test succeeds.
Though the issue with hanging is more general, I think that if it wouldn't be
fixed, the fix for CALCITE-2798 should be reverted to reduce cases when planner
may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2804) Druid adapter: Cast does not work when casting to timestamp
[
https://issues.apache.org/jira/browse/CALCITE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858810#comment-16858810
]
Hongze Zhang commented on CALCITE-2804:
---
[~michaelmior], [~zabetak] - thanks and sorry I am currently not able to run
the integration test VMs on my PC, so I can test it by my own at this time.
By the way, Stamatis, do you have a chance to see what's the difference between
"expected" and "actual"? I failed using text compare tools to find some clues.
I have a feeling that the issue might not be a major one and we can just modify
expected query text to match the actual.
> Druid adapter: Cast does not work when casting to timestamp
> ---
>
> Key: CALCITE-2804
> URL: https://issues.apache.org/jira/browse/CALCITE-2804
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Justin Szeluga
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When trying to execute a simple statement of
> {code:java}
> select cast("__time" as timestamp) from my_datasource limit 10;{code}
> the query fails with a 500 error
>
> {code:java}
> Caused by: java.io.IOException: Server returned HTTP response code: 500 for
> URL: http://localhost:58082/druid/v2/?pretty
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
> at org.apache.calcite.runtime.HttpUtils.executeMethod(HttpUtils.java:137)
> at org.apache.calcite.runtime.HttpUtils.post(HttpUtils.java:110)
> at
> org.apache.calcite.adapter.druid.DruidConnectionImpl.request(DruidConnectionImpl.java:109)
> ... 29 more
> {code}
> because the JSON generated is incorrect. Here is the generated JSON
>
>
> {code:java}
> {
> "queryType": "scan",
> "dataSource": "my_datasource",
> "intervals": [
> "1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"
> ],
> "virtualColumns": [
> {
> "type": "expression",
> "name": "vc",
> "expression":
> "timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'','UTC')",
> "outputType": "LONG"
> }
> ],
> "columns": [
> "vc"
> ],
> "resultFormat": "compactedList",
> "limit": 10
> }
> {code}
> There are two problems. -First, the "__time" dimension is not formatted
> correctly in timestamp_format. The escaped quotes need to be removed.-
> Also, the call to timestamp_parse is incorrect. If a timezone is going to be
> provided then a date time format needs to be provided as well. Currently, it
> is only empty string
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
[ https://issues.apache.org/jira/browse/CALCITE-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858796#comment-16858796 ] Ruben Quesada Lopez commented on CALCITE-3119: -- [~julianhyde], I agree. PR available. > Deprecate Linq4j CorrelateJoinType (in favor of JoinType) > - > > Key: CALCITE-3119 > URL: https://issues.apache.org/jira/browse/CALCITE-3119 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a > result, we have right now two enums for (almost) the same purpose: > CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, > SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all > types of join, mirroring core's JoinRelType). > The idea is to flag CorrelateJoinType as deprecated, and replace all its > usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
[ https://issues.apache.org/jira/browse/CALCITE-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3119: Labels: pull-request-available (was: ) > Deprecate Linq4j CorrelateJoinType (in favor of JoinType) > - > > Key: CALCITE-3119 > URL: https://issues.apache.org/jira/browse/CALCITE-3119 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > > CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a > result, we have right now two enums for (almost) the same purpose: > CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, > SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all > types of join, mirroring core's JoinRelType). > The idea is to flag CorrelateJoinType as deprecated, and replace all its > usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
[ https://issues.apache.org/jira/browse/CALCITE-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858792#comment-16858792 ] Julian Hyde commented on CALCITE-3119: -- I think we should do this before 1.20. There aren't many uses. > Deprecate Linq4j CorrelateJoinType (in favor of JoinType) > - > > Key: CALCITE-3119 > URL: https://issues.apache.org/jira/browse/CALCITE-3119 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Fix For: 1.20.0 > > > CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a > result, we have right now two enums for (almost) the same purpose: > CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, > SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all > types of join, mirroring core's JoinRelType). > The idea is to flag CorrelateJoinType as deprecated, and replace all its > usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
[ https://issues.apache.org/jira/browse/CALCITE-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruben Quesada Lopez updated CALCITE-3119: - Description: CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a result, we have right now two enums for (almost) the same purpose: CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all types of join, mirroring core's JoinRelType). The idea is to flag CorrelateJoinType as deprecated, and replace all its usages by JoinType. was: CALCITE-2968 introduced in Linq4j a new (more general) JoinType. As a result, we have right now two enums for (almost) the same purpose: CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all types of join, mirroring core's JoinRelType). The idea is to flag CorrelateJoinType as deprecated, and replace all its usages by JoinType. > Deprecate Linq4j CorrelateJoinType (in favor of JoinType) > - > > Key: CALCITE-3119 > URL: https://issues.apache.org/jira/browse/CALCITE-3119 > Project: Calcite > Issue Type: Task >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Fix For: 1.20.0 > > > CALCITE-2968 introduced in Linq4j a new (more general) enum JoinType. As a > result, we have right now two enums for (almost) the same purpose: > CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, > SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all > types of join, mirroring core's JoinRelType). > The idea is to flag CorrelateJoinType as deprecated, and replace all its > usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3120) MongoAdapterTest.testGroupByAvgSumCount fails due the order of sort query
[
https://issues.apache.org/jira/browse/CALCITE-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858788#comment-16858788
]
Stamatis Zampetakis commented on CALCITE-3120:
--
I have the impression that performing the sort on state before or after the
projection is not really a problem so the verification of the test should be
made differently. On the other hand I am not familiar at all with Mongo
pipelines so any tips are appreciated. Maybe [~sereda] has some inspiration :)
> MongoAdapterTest.testGroupByAvgSumCount fails due the order of sort query
> -
>
> Key: CALCITE-3120
> URL: https://issues.apache.org/jira/browse/CALCITE-3120
> Project: Calcite
> Issue Type: Bug
> Components: mongodb-adapter
>Affects Versions: 1.19.0
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
>
> The MongoAdapterTest.testGroupByAvgSumCount fails because $sort operator does
> not appear at the middle of the query but in the end.
> {noformat}
> org.junit.ComparisonFailure:
> expected and actual Mongo queries (pipelines) do not match
> expected:<...$project" : {
> "[POP" : "$pop",
> "STATE" : "$state"
> }
> }
> {
> "$group" : {
> "_id" : "$STATE",
> "_1" : {
> "$sum" : "$POP"
> },
> "_2" : {
> "$sum" : {
> "$cond" : [{
> "$eq" : ["POP", null]
> }, 0, 1]
> }
> }
> }
> }
> {
> "$project" : {
> "STATE" : "$_id",
> "_1" : "$_1",
> "_2" : "$_2"
> }
> }
> {
> "$sort" : {
> "STATE" : 1
> }
> }
> {
> "$project" : {
> "STATE" : 1,
> "A" : {
> "$divide" : [{
> "$cond" : [{
> "$eq" : ["$_2", {
> "$literal" : 0
> }]
> }, null, "$_1"]
> }, "$_2"]
> },
> "S" : {
> "$cond" : [{
> "$eq" : ["$_2", {
> "$literal" : 0
> }]
> }, null, "$_1"]
> },
> "C" : "$_2"]
> }
> }> but was:<...$project" : {
> "[STATE" : "$state",
> "POP" : "$pop"
> }
> }
> {
> "$group" : {
> "_id" : "$STATE",
> "_1" : {
> "$sum" : "$POP"
> },
> "_2" : {
> "$sum" : {
> "$cond" : [{
> "$eq" : ["POP", null]
> }, 0, 1]
> }
> }
> }
> }
> {
> "$project" : {
> "STATE" : "$_id",
> "_1" : "$_1",
> "_2" : "$_2"
> }
> }
> {
> "$project" : {
> "STATE" : 1,
> "A" : {
> "$divide" : [{
> "$cond" : [{
> "$eq" : ["$_2", {
> "$literal" : 0
> }]
> }, null, "$_1"]
> }, "$_2"]
> },
> "S" : {
> "$cond" : [{
> "$eq" : ["$_2", {
> "$literal" : 0
> }]
> }, null, "$_1"]
> },
> "C" : "$_2"
> }
> }
> {
> "$sort" : {
> "STATE" : 1]
> }
> }>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3120) MongoAdapterTest.testGroupByAvgSumCount fails due the order of sort query
Stamatis Zampetakis created CALCITE-3120:
Summary: MongoAdapterTest.testGroupByAvgSumCount fails due the
order of sort query
Key: CALCITE-3120
URL: https://issues.apache.org/jira/browse/CALCITE-3120
Project: Calcite
Issue Type: Bug
Components: mongodb-adapter
Affects Versions: 1.19.0
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
The MongoAdapterTest.testGroupByAvgSumCount fails because $sort operator does
not appear at the middle of the query but in the end.
{noformat}
org.junit.ComparisonFailure:
expected and actual Mongo queries (pipelines) do not match
expected:<...$project" : {
"[POP" : "$pop",
"STATE" : "$state"
}
}
{
"$group" : {
"_id" : "$STATE",
"_1" : {
"$sum" : "$POP"
},
"_2" : {
"$sum" : {
"$cond" : [{
"$eq" : ["POP", null]
}, 0, 1]
}
}
}
}
{
"$project" : {
"STATE" : "$_id",
"_1" : "$_1",
"_2" : "$_2"
}
}
{
"$sort" : {
"STATE" : 1
}
}
{
"$project" : {
"STATE" : 1,
"A" : {
"$divide" : [{
"$cond" : [{
"$eq" : ["$_2", {
"$literal" : 0
}]
}, null, "$_1"]
}, "$_2"]
},
"S" : {
"$cond" : [{
"$eq" : ["$_2", {
"$literal" : 0
}]
}, null, "$_1"]
},
"C" : "$_2"]
}
}> but was:<...$project" : {
"[STATE" : "$state",
"POP" : "$pop"
}
}
{
"$group" : {
"_id" : "$STATE",
"_1" : {
"$sum" : "$POP"
},
"_2" : {
"$sum" : {
"$cond" : [{
"$eq" : ["POP", null]
}, 0, 1]
}
}
}
}
{
"$project" : {
"STATE" : "$_id",
"_1" : "$_1",
"_2" : "$_2"
}
}
{
"$project" : {
"STATE" : 1,
"A" : {
"$divide" : [{
"$cond" : [{
"$eq" : ["$_2", {
"$literal" : 0
}]
}, null, "$_1"]
}, "$_2"]
},
"S" : {
"$cond" : [{
"$eq" : ["$_2", {
"$literal" : 0
}]
}, null, "$_1"]
},
"C" : "$_2"
}
}
{
"$sort" : {
"STATE" : 1]
}
}>
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2804) Druid adapter: Cast does not work when casting to timestamp
[
https://issues.apache.org/jira/browse/CALCITE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858741#comment-16858741
]
Michael Mior commented on CALCITE-2804:
---
Unfortunately, I'm not really a Druid user so I can't say. Any thoughts
[~zhztheplayer]?
> Druid adapter: Cast does not work when casting to timestamp
> ---
>
> Key: CALCITE-2804
> URL: https://issues.apache.org/jira/browse/CALCITE-2804
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Justin Szeluga
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When trying to execute a simple statement of
> {code:java}
> select cast("__time" as timestamp) from my_datasource limit 10;{code}
> the query fails with a 500 error
>
> {code:java}
> Caused by: java.io.IOException: Server returned HTTP response code: 500 for
> URL: http://localhost:58082/druid/v2/?pretty
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
> at org.apache.calcite.runtime.HttpUtils.executeMethod(HttpUtils.java:137)
> at org.apache.calcite.runtime.HttpUtils.post(HttpUtils.java:110)
> at
> org.apache.calcite.adapter.druid.DruidConnectionImpl.request(DruidConnectionImpl.java:109)
> ... 29 more
> {code}
> because the JSON generated is incorrect. Here is the generated JSON
>
>
> {code:java}
> {
> "queryType": "scan",
> "dataSource": "my_datasource",
> "intervals": [
> "1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"
> ],
> "virtualColumns": [
> {
> "type": "expression",
> "name": "vc",
> "expression":
> "timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'','UTC')",
> "outputType": "LONG"
> }
> ],
> "columns": [
> "vc"
> ],
> "resultFormat": "compactedList",
> "limit": 10
> }
> {code}
> There are two problems. -First, the "__time" dimension is not formatted
> correctly in timestamp_format. The escaped quotes need to be removed.-
> Also, the call to timestamp_parse is incorrect. If a timezone is going to be
> provided then a date time format needs to be provided as well. Currently, it
> is only empty string
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2804) Druid adapter: Cast does not work when casting to timestamp
[
https://issues.apache.org/jira/browse/CALCITE-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858737#comment-16858737
]
Stamatis Zampetakis commented on CALCITE-2804:
--
Hi [~jszeluga], [~michaelmior],
I just run the integration tests, and the test introduced by this PR
({{DruidAdapterIT#testCastToTimestamp}}) fails with the following message:
{noformat}
java.lang.AssertionError:
Expected: a string containing
"{\"queryType\":\"scan\",\"dataSource\":\"foodmart\",\"intervals\":[\"1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z\"]\",\"virtualColumns\":[{\"type\":\"expression\",\"name\":\"vc\",\"expression\":\"timestamp_parse(timestamp_format(\"__time\",\"-MM-dd\u0027T\u0027HH:mm:ss.SSS\u0027Z\u0027\",\"America/New_York\"),\"-MM-dd\u0027T\u0027HH:mm:ss.SSS\u0027Z\u0027\",\"UTC\")\",\"outputType\":\"LONG\"}],\"columns\":[\"vc\"],\"resultFormat\":\"compactedList\"}"
but: was
"{"queryType":"scan","dataSource":"foodmart","intervals":["1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z"],"virtualColumns":[{"type":"expression","name":"vc","expression":"timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','UTC')","outputType":"LONG"}],"columns":["vc"],"resultFormat":"compactedList"}"
Expected :a string containing
"{\"queryType\":\"scan\",\"dataSource\":\"foodmart\",\"intervals\":[\"1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z\"]\",\"virtualColumns\":[{\"type\":\"expression\",\"name\":\"vc\",\"expression\":\"timestamp_parse(timestamp_format(\"__time\",\"-MM-dd\u0027T\u0027HH:mm:ss.SSS\u0027Z\u0027\",\"America/New_York\"),\"-MM-dd\u0027T\u0027HH:mm:ss.SSS\u0027Z\u0027\",\"UTC\")\",\"outputType\":\"LONG\"}],\"columns\":[\"vc\"],\"resultFormat\":\"compactedList\"}"
Actual
:"{"queryType":"scan","dataSource":"foodmart","intervals":["1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z"],"virtualColumns":[{"type":"expression","name":"vc","expression":"timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','UTC')","outputType":"LONG"}],"columns":["vc"],"resultFormat":"compactedList"}"
{noformat}
Do you have any ideas why? I don't know if it is relevant but the current vm
uses druid version 0.14.0-incubating.
> Druid adapter: Cast does not work when casting to timestamp
> ---
>
> Key: CALCITE-2804
> URL: https://issues.apache.org/jira/browse/CALCITE-2804
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Justin Szeluga
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> When trying to execute a simple statement of
> {code:java}
> select cast("__time" as timestamp) from my_datasource limit 10;{code}
> the query fails with a 500 error
>
> {code:java}
> Caused by: java.io.IOException: Server returned HTTP response code: 500 for
> URL: http://localhost:58082/druid/v2/?pretty
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
> at org.apache.calcite.runtime.HttpUtils.executeMethod(HttpUtils.java:137)
> at org.apache.calcite.runtime.HttpUtils.post(HttpUtils.java:110)
> at
> org.apache.calcite.adapter.druid.DruidConnectionImpl.request(DruidConnectionImpl.java:109)
> ... 29 more
> {code}
> because the JSON generated is incorrect. Here is the generated JSON
>
>
> {code:java}
> {
> "queryType": "scan",
> "dataSource": "my_datasource",
> "intervals": [
> "1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z"
> ],
> "virtualColumns": [
> {
> "type": "expression",
> "name": "vc",
> "expression":
> "timestamp_parse(timestamp_format(\"__time\",'-MM-dd\\u0027T\\u0027HH:mm:ss.SSS\\u0027Z\\u0027','America/New_York'),'','UTC')",
> "outputType": "LONG"
> }
> ],
> "columns": [
> "vc"
> ],
> "resultFormat": "compactedList",
> "limit": 10
> }
> {code}
> There are two problems. -First, the "__time" dimension is not formatted
> correctly in timestamp_format. The escaped quotes need to be removed.-
> Also, the call to timestamp_parse is incorrect. If a timezone is going to be
> provided then a date time format needs to be provided as well. Currently, it
> is only empty string
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3119) Deprecate Linq4j CorrelateJoinType (in favor of JoinType)
Ruben Quesada Lopez created CALCITE-3119: Summary: Deprecate Linq4j CorrelateJoinType (in favor of JoinType) Key: CALCITE-3119 URL: https://issues.apache.org/jira/browse/CALCITE-3119 Project: Calcite Issue Type: Task Reporter: Ruben Quesada Lopez Assignee: Ruben Quesada Lopez Fix For: 1.20.0 CALCITE-2968 introduced in Linq4j a new (more general) JoinType. As a result, we have right now two enums for (almost) the same purpose: CorrelateJoinType (used for Correlation, thus defining only INNER, LEFT, SEMI, ANTI) and the new JoinType (can be used for any purpose, defining all types of join, mirroring core's JoinRelType). The idea is to flag CorrelateJoinType as deprecated, and replace all its usages by JoinType. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3063) Babel parse should parse PostgreSQL posix regular expressions
[
https://issues.apache.org/jira/browse/CALCITE-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Mior updated CALCITE-3063:
--
Fix Version/s: (was: 1.20.0)
> Babel parse should parse PostgreSQL posix regular expressions
> -
>
> Key: CALCITE-3063
> URL: https://issues.apache.org/jira/browse/CALCITE-3063
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Affects Versions: 1.19.0
>Reporter: Muhammad Gelbana
>Assignee: Muhammad Gelbana
>Priority: Major
> Labels: postgresql, pull-request-available,
> pull-requests-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Quoting from the referenced link below, posix operators are:
> ||Operator||Description||Example||
> |{{~}}|Matches regular expression, case sensitive|{{'thomas' ~ '.*thomas.*'}}|
> |{{~*}}|Matches regular expression, case insensitive|{{'thomas' ~*
> '.*Thomas.*'}}|
> |{{!~}}|Does not match regular expression, case sensitive|{{'thomas' !~
> '.*Thomas.*'}}|
> |{{!~*}}|Does not match regular expression, case insensitive|{{'thomas' !~*
> '.*vadim.*'}}|
>
> +Reference:+
> https://www.postgresql.org/docs/11/functions-matching.html#FUNCTIONS-POSIX-REGEXP
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3063) Babel parse should parse PostgreSQL posix regular expressions
[
https://issues.apache.org/jira/browse/CALCITE-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-3063:
Labels: postgresql pull-request-available pull-requests-available (was:
postgresql pull-requests-available)
> Babel parse should parse PostgreSQL posix regular expressions
> -
>
> Key: CALCITE-3063
> URL: https://issues.apache.org/jira/browse/CALCITE-3063
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Affects Versions: 1.19.0
>Reporter: Muhammad Gelbana
>Assignee: Muhammad Gelbana
>Priority: Major
> Labels: postgresql, pull-request-available,
> pull-requests-available
> Fix For: 1.20.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Quoting from the referenced link below, posix operators are:
> ||Operator||Description||Example||
> |{{~}}|Matches regular expression, case sensitive|{{'thomas' ~ '.*thomas.*'}}|
> |{{~*}}|Matches regular expression, case insensitive|{{'thomas' ~*
> '.*Thomas.*'}}|
> |{{!~}}|Does not match regular expression, case sensitive|{{'thomas' !~
> '.*Thomas.*'}}|
> |{{!~*}}|Does not match regular expression, case insensitive|{{'thomas' !~*
> '.*vadim.*'}}|
>
> +Reference:+
> https://www.postgresql.org/docs/11/functions-matching.html#FUNCTIONS-POSIX-REGEXP
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-1703) Query with BETWEEN or AGGREGATE functions on TIMESTAMP column throws ClassCastException
[
https://issues.apache.org/jira/browse/CALCITE-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Mior updated CALCITE-1703:
--
Fix Version/s: (was: 1.20.0)
> Query with BETWEEN or AGGREGATE functions on TIMESTAMP column throws
> ClassCastException
> ---
>
> Key: CALCITE-1703
> URL: https://issues.apache.org/jira/browse/CALCITE-1703
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.11.0
> Environment: Any
>Reporter: Gangadhar Kairi
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Getting below error while executing the query on timestamp columns with
> BETWEEN or AGGREGATE function operator.
> {noformat}
> java.sql.SQLException: Error while executing SQL "select * from "DATE" where
> "JOINTIMES" < '2017-03-16 18:04:07.034'": java.sql.Timestamp cannot be cast
> to java.lang.Long
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
> at
> org.apache.calcite.test.CsvTest.testTimestampGTCheck(CsvTest.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: java.lang.ClassCastException: java.sql.Timestamp cannot be cast to
> java.lang.Long
> at Baz$1$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:664)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:89)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:196)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> {noformat}
> I could able to reproduce this issues in CsvTests.
>
> Fire a test with query "select * from \"DATE\" where \"JOINTIMES\" BETWEEN '"
> + from + "' and '" + to + "'" you will see test failing with exception.
> Root cause and fix:
> Timestamp datatype needs to be handled in the same way as the Date datatype.
> In {{RexToLixTranslator.convert()}} method and in
> {{PhysTypeImpl.fieldReference()}} method.
--
This message
[jira] [Updated] (CALCITE-2921) nullif(null,y) throws exception in verification
[
https://issues.apache.org/jira/browse/CALCITE-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Mior updated CALCITE-2921:
--
Fix Version/s: (was: 1.20.0)
> nullif(null,y) throws exception in verification
>
>
> Key: CALCITE-2921
> URL: https://issues.apache.org/jira/browse/CALCITE-2921
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently calcite will translate "nullif(x,y)" to a case-when expression,just
> like "case when x = y then null else x".So when "x" is null literal,a
> exception throws out as follow:
> {code:java}
> ELSE clause or at least one THEN clause must be non-NULL
> {code}
> I have test in mysql,"nullif(null,y) works well.So I think we should allow
> this usage of "nullif".
> There are two ways to fix this issue:
> 1) Skip the check for "foundNotNull" in SqlCaseOperator#checkOperandTypes:
>
> {code:java}
> if (!foundNotNull) {
> // according to the sql standard we can not have all of the THEN
> // statements and the ELSE returning null
> if (throwOnFailure) {
> throw callBinding.newError(RESOURCE.mustNotNullInElse());
> }
> return false;
> }{code}
> However, as the comment says, we cannot have all of the THEN and ELSE
> returning null.
> 2) Disable the translation from nullif to case-when and keep "nullif" as it
> is.
> Any suggestion is welcomed,Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2968) New AntiJoin relational expression
[ https://issues.apache.org/jira/browse/CALCITE-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858723#comment-16858723 ] Michael Mior commented on CALCITE-2968: --- Fixed in [be2b979|https://github.com/apache/calcite/commit/be2b97905548a5b24067c8636567b364529332cc] > New AntiJoin relational expression > -- > > Key: CALCITE-2968 > URL: https://issues.apache.org/jira/browse/CALCITE-2968 > Project: Calcite > Issue Type: New Feature >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > The goal of this ticket is to follow the discussion started in CALCITE-2920 > (and its [PR|https://github.com/apache/calcite/pull/1110]). > With the implementation of CALCITE-2920, RelBuilder has now a method to > create an AntiJoin. However, since, for the moment, there is no "AntiJoin" > expression, the only possibility to build an AntiJoin is using a > LogicalCorrelate with SemiJoinType.ANTI, and that's exactly what this method > does, for now. > The idea would be to "generalize" this with the creation of a new AntiJoin > (a.k.a. AntiSemiJoin) relational expression. With the implementation of > CALCITE-2969 (re-design of join-like expressions), it seems clear that the > new AntiJoin expression will be a *LogicalJoin with JoinType=ANTI*. Also, it > will have to be verified that: > - this new AntiJoin works fine in EnumerableConvention with all types of join > condition (equi-join, partial equi-join, non equi-join) > - this new AntiJoin can be correctly transformed into a Correlate (type ANTI) > via JoinToCorrelateRule -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-2968) New AntiJoin relational expression
[ https://issues.apache.org/jira/browse/CALCITE-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Mior resolved CALCITE-2968. --- Resolution: Fixed > New AntiJoin relational expression > -- > > Key: CALCITE-2968 > URL: https://issues.apache.org/jira/browse/CALCITE-2968 > Project: Calcite > Issue Type: New Feature >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > The goal of this ticket is to follow the discussion started in CALCITE-2920 > (and its [PR|https://github.com/apache/calcite/pull/1110]). > With the implementation of CALCITE-2920, RelBuilder has now a method to > create an AntiJoin. However, since, for the moment, there is no "AntiJoin" > expression, the only possibility to build an AntiJoin is using a > LogicalCorrelate with SemiJoinType.ANTI, and that's exactly what this method > does, for now. > The idea would be to "generalize" this with the creation of a new AntiJoin > (a.k.a. AntiSemiJoin) relational expression. With the implementation of > CALCITE-2969 (re-design of join-like expressions), it seems clear that the > new AntiJoin expression will be a *LogicalJoin with JoinType=ANTI*. Also, it > will have to be verified that: > - this new AntiJoin works fine in EnumerableConvention with all types of join > condition (equi-join, partial equi-join, non equi-join) > - this new AntiJoin can be correctly transformed into a Correlate (type ANTI) > via JoinToCorrelateRule -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2736) ReduceExpressionsRule never reduces dynamic expressions but this should be configurable
[ https://issues.apache.org/jira/browse/CALCITE-2736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Mior updated CALCITE-2736: -- Fix Version/s: (was: 1.20.0) > ReduceExpressionsRule never reduces dynamic expressions but this should be > configurable > --- > > Key: CALCITE-2736 > URL: https://issues.apache.org/jira/browse/CALCITE-2736 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Jacques Nadeau >Assignee: Jacques Nadeau >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > There are situations where it is helpful to reduce dynamic SqlCalls. Right > now, the ReduceExpressionsRule always avoids doing this. We should enhance > the rule so this can be configurable depending on where in planning the rule > is used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-1581) UDTF like in hive
[
https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Mior updated CALCITE-1581:
--
Fix Version/s: (was: 1.20.0)
> UDTF like in hive
> -
>
> Key: CALCITE-1581
> URL: https://issues.apache.org/jira/browse/CALCITE-1581
> Project: Calcite
> Issue Type: New Feature
>Reporter: Xiaoyong Deng
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available, udtf
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Support one row in and multi-column/multi-row out(one-to-many mapping), just
> like udtf in hive.
> The query would like this:
> {code}
> select
> func(c0, c1) as (f0, f1, f2)
> from table_name;
> {code}
> c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2843) Babel parser should parse PostgreSQL-style '::' cast operator
[
https://issues.apache.org/jira/browse/CALCITE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Mior updated CALCITE-2843:
--
Fix Version/s: (was: 1.20.0)
> Babel parser should parse PostgreSQL-style '::' cast operator
> -
>
> Key: CALCITE-2843
> URL: https://issues.apache.org/jira/browse/CALCITE-2843
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Affects Versions: 1.18.0
>Reporter: Muhammad Gelbana
>Assignee: Muhammad Gelbana
>Priority: Major
> Labels: postgresql, pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> *Code to reproduce the problem*
> {code:java}
> public static void main(String[] args) throws Exception {
> Config parserConfig =
> configBuilder().setConformance(SqlConformanceEnum.BABEL).setParserFactory(SqlBabelParserImpl.FACTORY).build();
> FrameworkConfig frameworkConfig =
> Frameworks.newConfigBuilder().parserConfig(parserConfig).build();
> Planner planner = Frameworks.getPlanner(frameworkConfig);
> String pg = "SELECT 'array_in'::regproc, typtype FROM pg_catalog.pg_type";
> planner.parse(pg);
> }{code}
>
> *Thrown exception*
> {noformat}
> Exception in thread "main" org.apache.calcite.sql.parser.SqlParseException:
> Encountered ":" at line 1, column 18.
> Was expecting one of:
>
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "FROM" ...
> "," ...
> ...
> ...
> ...
> ...
> ...
> ...
> "." ...
> "IN" ...
> "<" ...
> "<=" ...
> ">" ...
> ">=" ...
> "=" ...
> "<>" ...
> "!=" ...
> "+" ...
> "-" ...
> "*" ...
> "/" ...
> "%" ...
> "||" ...
> "[" ...
> "UNION" ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
>
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.convertException(SqlBabelParserImpl.java:354)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.normalizeException(SqlBabelParserImpl.java:142)
> at
> org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:156)
> at org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:181)
> at org.apache.calcite.prepare.PlannerImpl.parse(PlannerImpl.java:174)
> at org.apache.calcite.tools.Planner.parse(Planner.java:50)
> at com.lab.calcite.App2.main(App2.java:23)
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered
> ":" at line 1, column 18.
> Was expecting one of:
>
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "FROM" ...
> "," ...
> ...
> ...
> ...
> ...
> ...
> ...
> "." ...
> "IN" ...
> "<" ...
> "<=" ...
> ">" ...
> ">=" ...
> "=" ...
> "<>" ...
> "!=" ...
> "+" ...
> "-" ...
> "*" ...
> "/" ...
> "%" ...
> "||" ...
> "[" ...
> "UNION" ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
>
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.generateParseException(SqlBabelParserImpl.java:31191)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.jj_consume_token(SqlBabelParserImpl.java:31008)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.SqlStmtEof(SqlBabelParserImpl.java:877)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.parseSqlStmtEof(SqlBabelParserImpl.java:198)
> at
> org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:148)
> ... 4 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-2764) RelCompositeTrait#satisfies does not work for non-trivial traits
[
https://issues.apache.org/jira/browse/CALCITE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Mior updated CALCITE-2764:
--
Fix Version/s: (was: 1.20.0)
> RelCompositeTrait#satisfies does not work for non-trivial traits
>
>
> Key: CALCITE-2764
> URL: https://issues.apache.org/jira/browse/CALCITE-2764
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.18.0
>Reporter: Vladimir Sitnikov
>Assignee: Vladimir Sitnikov
>Priority: Major
>
> {noformat}java.lang.AssertionError: [[0, 1], [1]].satisfies([[0, 1], [1]])
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.calcite.plan.RelTraitTest.assertSatisfies(RelTraitTest.java:81)
> at
> org.apache.calcite.plan.RelTraitTest.compositeSatisfies(RelTraitTest.java:77)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2721) Support parsing of DOT + MEMBER_FUNCTION
[
https://issues.apache.org/jira/browse/CALCITE-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858715#comment-16858715
]
Michael Mior commented on CALCITE-2721:
---
Fixed in
[4513a72|https://github.com/apache/calcite/commit/4513a7293c7ce2307ecdb8129a9167165fc59ab5]
> Support parsing of DOT + MEMBER_FUNCTION
>
>
> Key: CALCITE-2721
> URL: https://issues.apache.org/jira/browse/CALCITE-2721
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Rong Rong
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Attachments: 6_17_Method_invocation.png
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Support parsing of + parsing Similar to
> {panel:title=Example}
> {{SELECT t.complex_col}}*{{.member_func()}}* {{FROM tbl;}}
> {panel}
> For more details please see: [Member
> Function|https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_8001.htm#i2126584]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-2721) Support parsing of DOT + MEMBER_FUNCTION
[
https://issues.apache.org/jira/browse/CALCITE-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Mior resolved CALCITE-2721.
---
Resolution: Fixed
> Support parsing of DOT + MEMBER_FUNCTION
>
>
> Key: CALCITE-2721
> URL: https://issues.apache.org/jira/browse/CALCITE-2721
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: Rong Rong
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Attachments: 6_17_Method_invocation.png
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Support parsing of + parsing Similar to
> {panel:title=Example}
> {{SELECT t.complex_col}}*{{.member_func()}}* {{FROM tbl;}}
> {panel}
> For more details please see: [Member
> Function|https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_8001.htm#i2126584]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated CALCITE-3118: -- Description: In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, looking for parent operand match), we want to check that the matched parent indeed has the previously matched child RelNode as a child with the expected ordinal. However, there is a bug in this check. As a result, some incorrect parent is not skipped as expected and matched incorrectly. See unit test included in PR for a case that triggers this bug, where a second child RelNode get matched as first child of the parent RelNode. (was: In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, looking for parent operand match), we want to check that the matched parent indeed has the previously matched child RelNode as a child with the expected ordinal. However, there is a bug in this check. As a result, some incorrect parent is not skipped as expected and matched incorrectly. See unit test included in PR for a case that triggers this bug, where the same RelNode get matched for two operands at the same time. ) > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where a second child > RelNode get matched as first child of the parent RelNode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858702#comment-16858702
]
Vladimir Sitnikov commented on CALCITE-3118:
{quote}At least I hope you agree with the one line fix...{quote}
I've no idea.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where the same RelNode get
> matched for two operands at the same time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858699#comment-16858699 ] Botong Huang commented on CALCITE-3118: --- bq. Or it might happen you could rewrite the test-case in easy to understand way. The unit test is indeed involved because of the tricky condition of the bug. However I don't know if there's an easier way to trigger it... At least I hope you agree with the one line fix... > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858696#comment-16858696 ] Vladimir Sitnikov commented on CALCITE-3118: I'm sorry I have no much time left here. Hopefully someone else might pick it up. Or it might happen you could rewrite the test-case in easy to understand way. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858178#comment-16858178 ] Botong Huang edited comment on CALCITE-3118 at 6/7/19 2:24 PM: --- Agree in general. But in this test case, the rule in this unit test (AssertOperandsDifferentRule) is to match a parent and its first and second child, it is wrong to return a match with a parent relNode and its second child twice. was (Author: botong): Agree in general. But in this test case, the rule in this unit test (AssertOperandsDifferentRule) is to match a parent and its first and second child, it is wrong to return a match with a parent relNode and both its second child. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858693#comment-16858693
]
Botong Huang edited comment on CALCITE-3118 at 6/7/19 2:20 PM:
---
{quote}It is not wrong provided the children are identical
{quote}
I think you missed my point. The structure the _AssertOperandsDifferentRule_ is
looking for is:
A
/ \
B C
where *B is A's first child*. What the match returned was:
Rel1
/ \
Rel2.1 Rel2.2
where Rel2.1 is *NOT* Rel1's first child.
was (Author: botong):
{quote}It is not wrong provided the children are identical
{quote}
I think you missed my point. The structure the _AssertOperandsDifferentRule_ is
looking for is:
A
/ \
B C
where *B is A's first child*. What the match returned was:
Rel1
/ \
Rel2.1 Rel2.2
where Rel2.1 is *NOT* A's first child.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where the same RelNode get
> matched for two operands at the same time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858693#comment-16858693
]
Botong Huang edited comment on CALCITE-3118 at 6/7/19 2:19 PM:
---
{quote}It is not wrong provided the children are identical
{quote}
I think you missed my point. The structure the _AssertOperandsDifferentRule_ is
looking for is:
A
/ \
B C
where *B is A's first child*. What the match returned was:
Rel1
/ \
Rel2.1 Rel2.2
where Rel2.1 is *NOT* A's first child.
was (Author: botong):
bq. It is not wrong provided the children are identical
I think you missed my point. The structure the _AssertOperandsDifferentRule_ is
looking for is:
A
/ \
B C
where *B is A's first child*. What the match returned was:
Rel1
/ \
Rel2.1 Rel2.2
where Rel2.1 is *NOT* A's first child.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where the same RelNode get
> matched for two operands at the same time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858693#comment-16858693 ] Botong Huang commented on CALCITE-3118: --- bq. It is not wrong provided the children are identical I think you missed my point. The structure the _AssertOperandsDifferentRule_ is looking for is: A / \ B C where *B is A's first child*. What the match returned was: Rel1 / \ Rel2.1 Rel2.2 where Rel2.1 is *NOT* A's first child. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858683#comment-16858683
]
Vladimir Sitnikov commented on CALCITE-3118:
{quote}it is wrong to return a match with a parent relNode and its second
children twice.{quote}
It is not wrong provided the children are identical
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where the same RelNode get
> matched for two operands at the same time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858653#comment-16858653 ] Botong Huang edited comment on CALCITE-3118 at 6/7/19 1:39 PM: --- Thanks [~danny0405] for reviewing! Having two relNodes of the same class in two RelSubsets in the same RelSet (thus the equivalent line) is the necessary condition to trigger this bug. However, using convention in the unit test is not a must. It's merely to produce the two subSets. Any two traits that are different would suffice. Also as in AssertOperandsDifferentRule, it is not trying to match any convention. We are simply trying to match a parent with its first and second children being PhysLeafRel.class, it is wrong to return a match with a parent relNode and its second children twice. was (Author: botong): Thanks [~danny0405] for reviewing! Having two relNodes of the same class in two RelSubsets in the same RelSet (thus the equivalent line) is the necessary condition to trigger this bug. However, using convention in the unit test is not a must. It's merely to produce the two subSets. Any two traits that are different would suffice. Also as in AssertOperandsDifferentRule, it is not trying to match any convention. We are simply trying to match a parent with its first and second children being PhysLeafRel.class, it is wrong to return a match with a parent relNode and both its second children. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858053#comment-16858053 ] Botong Huang edited comment on CALCITE-3118 at 6/7/19 1:38 PM: --- Sure, supposedly this rule should only match once, with left child operand == leftPhy (PHYS, label = a) and right child operand == rightPhy (PHYS_2, label == b). However, without the fix, when rightPhy gets generated, it triggers two match attempts. The first one is expected as above. In the second one, rightPhy (PHYS_2, label == b) matches the left child operand, then we attempt to match the PhyBiRel as parent operand. Supposedly this parent should not match, because rightPhy is PhyBiRel's second child, but rightPhy's matched left child operand is parent operand's first child. Because of the bug, it didn't skip as expected and still proceed with the matching. After parent is matched, then it attempt to match the right child operand, and again matched the rightPhy (PHYS_2, label == b), as expected because it is PhyBiRel's second child. Overall as a result, both child operand end up matching the same RelNode rightPhy (PHYS_2, label == b). To recap, AssertOperandsDifferentRule is trying to match a parent with its first and second children being PhysLeafRel.class, it is wrong to return a match with a parent relNode and both its second children. Hope this is clearer now. was (Author: botong): Sure, supposedly this rule should only match once, with left child operand == leftPhy (PHYS, label = a) and right child operand == rightPhy (PHYS_2, label == b). However, without the fix, when rightPhy gets generated, it triggers two match attempts. The first one is expected as above. In the second one, rightPhy (PHYS_2, label == b) matches the left child operand, then we attempt to match the PhyBiRel as parent operand. Supposedly this parent should not match, because rightPhy is PhyBiRel's second child, but rightPhy's matched left child operand is parent operand's first child. Because of the bug, it didn't skip as expected and still proceed with the matching. After parent is matched, then it attempt to match the right child operand, and again matched the rightPhy (PHYS_2, label == b), as expected because it is PhyBiRel's second child. Overall as a result, both child operand end up matching the same RelNode rightPhy (PHYS_2, label == b). Hope this is clearer now. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858053#comment-16858053 ] Botong Huang edited comment on CALCITE-3118 at 6/7/19 1:39 PM: --- Sure, supposedly this rule should only match once, with left child operand == leftPhy (PHYS, label = a) and right child operand == rightPhy (PHYS_2, label == b). However, without the fix, when rightPhy gets generated, it triggers two match attempts. The first one is expected as above. In the second one, rightPhy (PHYS_2, label == b) matches the left child operand, then we attempt to match the PhyBiRel as parent operand. Supposedly this parent should not match, because rightPhy is PhyBiRel's second child, but rightPhy's matched left child operand is parent operand's first child. Because of the bug, it didn't skip as expected and still proceed with the matching. After parent is matched, then it attempt to match the right child operand, and again matched the rightPhy (PHYS_2, label == b), as expected because it is PhyBiRel's second child. Overall as a result, both child operand end up matching the same RelNode rightPhy (PHYS_2, label == b). To recap, AssertOperandsDifferentRule is trying to match a parent with its first and second children being PhysLeafRel.class, it is wrong to return a match with a parent relNode and its second children twice. Hope this is clearer now. was (Author: botong): Sure, supposedly this rule should only match once, with left child operand == leftPhy (PHYS, label = a) and right child operand == rightPhy (PHYS_2, label == b). However, without the fix, when rightPhy gets generated, it triggers two match attempts. The first one is expected as above. In the second one, rightPhy (PHYS_2, label == b) matches the left child operand, then we attempt to match the PhyBiRel as parent operand. Supposedly this parent should not match, because rightPhy is PhyBiRel's second child, but rightPhy's matched left child operand is parent operand's first child. Because of the bug, it didn't skip as expected and still proceed with the matching. After parent is matched, then it attempt to match the right child operand, and again matched the rightPhy (PHYS_2, label == b), as expected because it is PhyBiRel's second child. Overall as a result, both child operand end up matching the same RelNode rightPhy (PHYS_2, label == b). To recap, AssertOperandsDifferentRule is trying to match a parent with its first and second children being PhysLeafRel.class, it is wrong to return a match with a parent relNode and both its second children. Hope this is clearer now. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858653#comment-16858653 ] Botong Huang commented on CALCITE-3118: --- Thanks [~danny0405] for reviewing! Having two relNodes of the same class in two RelSubsets in the same RelSet (thus the equivalent line) is the necessary condition to trigger this bug. However, using convention in the unit test is not a must. It's merely to produce the two subSets. Any two traits that are different would suffice. Also as in AssertOperandsDifferentRule, it is not trying to match any convention. We are simply trying to match a parent with its first and second children being PhysLeafRel.class, it is wrong to return a match with a parent relNode and both its second children. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858178#comment-16858178 ] Botong Huang edited comment on CALCITE-3118 at 6/7/19 1:27 PM: --- Agree in general. But in this test case, the rule in this unit test (AssertOperandsDifferentRule) is to match a parent and its first and second child, it is wrong to return a match with a parent relNode and both its second child. was (Author: botong): Agree in general. But in this test case, the rule in this unit test (AssertOperandsDifferentRule) is to match a parent and its first and second child, it is wrong to return a match with a relNode and both its second child. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858325#comment-16858325
]
Danny Chan commented on CALCITE-3118:
-
I don't know why you make {{leftPhy}} and {{rightPhy}} equivalent, If they
were, the behavior is correct without your fix, because RelOptRule only matches
the RelNode class, not the convention, if you really want to match specific
convention, add a predicate on it :)
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where the same RelNode get
> matched for two operands at the same time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
