[jira] [Commented] (CALCITE-3117) Distinguish Calcite's public APIs
[ https://issues.apache.org/jira/browse/CALCITE-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858282#comment-16858282 ] Danny Chan commented on CALCITE-3117: - The parser and sql node should be split out of the calcite-core, user usually only want to parse their sql and get a AST. > Distinguish Calcite's public APIs > - > > Key: CALCITE-3117 > URL: https://issues.apache.org/jira/browse/CALCITE-3117 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > > The goal of this issue is to identify and mark Calcite's public APIs. This > can have bring various benefits to the project: > * provide more flexibility to developers; > * speed-up reviews; > * enhance backward compatibility; > * facilitate the life of end-users who will be exposed to fewer classes and > interfaces. > Let's list here packages, interfaces, and classes that should be part of the > public API. As a first step let's focus exclusively on completing the list > and not argue about others proposals. It will be easier to solve > disagreements afterwards as soon as we have a rather complete list. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2829) Use consistent types when processing ranges
[ https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858261#comment-16858261 ] Danny Chan edited comment on CALCITE-2829 at 6/7/19 2:40 AM: - Sure, the more test cases the better, i would link this issue with CALCITE-2302, thanks for your test cases. was (Author: danny0405): Sure, the more test cases the better, i would link this issue with CALCITE-2302 > Use consistent types when processing ranges > --- > > Key: CALCITE-2829 > URL: https://issues.apache.org/jira/browse/CALCITE-2829 > Project: Calcite > Issue Type: Bug >Reporter: Siddharth Teotia >Assignee: Juhwan Kim >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Range expressions like = 'literal' AND < 'literal' trigger > ClassCastException as literal are implicitly casted differently between =/<> > operators and other comparison operators. Apply the same casting rules for > comparison to =/<> calls when processing ranges, so that all the terms have > the same type for literals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2829) Use consistent types when processing ranges
[ https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858261#comment-16858261 ] Danny Chan commented on CALCITE-2829: - Sure, the more test cases the better, i would link this issue with CALCITE-2302 > Use consistent types when processing ranges > --- > > Key: CALCITE-2829 > URL: https://issues.apache.org/jira/browse/CALCITE-2829 > Project: Calcite > Issue Type: Bug >Reporter: Siddharth Teotia >Assignee: Juhwan Kim >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Range expressions like = 'literal' AND < 'literal' trigger > ClassCastException as literal are implicitly casted differently between =/<> > operators and other comparison operators. Apply the same casting rules for > comparison to =/<> calls when processing ranges, so that all the terms have > the same type for literals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3080) support user-specified row schema in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858246#comment-16858246 ] Xu Mingmin commented on CALCITE-3080: - Ensure the timestamp field in CALCITE-3073 always exist. > support user-specified row schema in KafkaAdapter > - > > Key: CALCITE-3080 > URL: https://issues.apache.org/jira/browse/CALCITE-3080 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Xu Mingmin >Priority: Major > > Currently rowSchema is defined by extending `KafkaRowConverter` only. Feature > in this task would support user-defined row schema when creating a table. > > Here's one proposed interface in > https://github.com/apache/calcite/pull/1127/files#r283913361 > {code} > interface KafkaRowConverter { > RelDataType rowDataType(); > Object[] toRow(ConsumerRecord message); > } > {code} > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858245#comment-16858245 ] Xu Mingmin commented on CALCITE-3073: - {{MSG_TIMESTAMP}} is good, but be aware that these columns are only a default schema, no guarantee to exist as users may define a total different schema in {{KafkaRowConverter}}. Once the name is decided, we need a constrain in https://issues.apache.org/jira/browse/CALCITE-3080. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. > In other words, if the app runs at first time and user want to consume the > past data, user must set the value of > '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. > {quote}{{auto.offset.reset:What to do when there is no initial offset in > Kafka or if the current offset does not exist any more on the server (e.g. > because that data has been deleted):}} > * earliest: automatically reset the offset to the earliest offset > * latest: automatically reset the offset to the latest offset > * none: throw exception to the consumer if no previous offset is found for > the consumer's group > * anything else: throw exception to the consumer.{quote} > for example, suppose data in Kafka is retained for 7 days and you just want > to read from the data of yesterday, if you could not control the start > timestamp, you can only read from the earliest offset, it's very inefficient. > If supporting to consume from special timestamp in KafkaAdapter will be a > good idea for some cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858187#comment-16858187 ] Matt Wang commented on CALCITE-3073: [~mingmxu] Using {color:#707070}*MSG_TIMESTAMP*{color} filed I think it's a better idea, when user use KafkaAdapter, they know these field is already defined: {quote}| MSG_PARTITION | MSG_TIMESTAMP | MSG_OFFSET | MSG_KEY_BYTES | MSG_VALUE_BYTES {quote} > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. > In other words, if the app runs at first time and user want to consume the > past data, user must set the value of > '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. > {quote}{{auto.offset.reset:What to do when there is no initial offset in > Kafka or if the current offset does not exist any more on the server (e.g. > because that data has been deleted):}} > * earliest: automatically reset the offset to the earliest offset > * latest: automatically reset the offset to the latest offset > * none: throw exception to the consumer if no previous offset is found for > the consumer's group > * anything else: throw exception to the consumer.{quote} > for example, suppose data in Kafka is retained for 7 days and you just want > to read from the data of yesterday, if you could not control the start > timestamp, you can only read from the earliest offset, it's very inefficient. > If supporting to consume from special timestamp in KafkaAdapter will be a > good idea for some cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858184#comment-16858184 ] Matt Wang commented on CALCITE-3073: [~zhztheplayer] I have modified the description of the JIRA. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. > In other words, if the app runs at first time and user want to consume the > past data, user must set the value of > '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. > {quote}{{auto.offset.reset:What to do when there is no initial offset in > Kafka or if the current offset does not exist any more on the server (e.g. > because that data has been deleted):}} > * earliest: automatically reset the offset to the earliest offset > * latest: automatically reset the offset to the latest offset > * none: throw exception to the consumer if no previous offset is found for > the consumer's group > * anything else: throw exception to the consumer.{quote} > for example, suppose data in Kafka is retained for 7 days and you just want > to read from the data of yesterday, if you could not control the start > timestamp, you can only read from the earliest offset, it's very inefficient. > If supporting to consume from special timestamp in KafkaAdapter will be a > good idea for some cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated CALCITE-3073: --- Description: Currently the KafkaAdapter consumes data from default offset(latest/earliest/last_offset) and runs forever. In other words, if the app runs at first time and user want to consume the past data, user must set the value of '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. {quote}{{auto.offset.reset:What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted):}} * earliest: automatically reset the offset to the earliest offset * latest: automatically reset the offset to the latest offset * none: throw exception to the consumer if no previous offset is found for the consumer's group * anything else: throw exception to the consumer.{quote} for example, suppose data in Kafka is retained for 7 days and you just want to read from the data of yesterday, if you could not control the start timestamp, you can only read from the earliest offset, it's very inefficient. If supporting to consume from special timestamp in KafkaAdapter will be a good idea for some cases. was: Currently the KafkaAdapter consumes data from default offset(latest/earliest/last_offset) and runs forever. In other words, if the app runs at first time and user want to consume the past data, user must set the value of '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. {quote}{{auto.offset.reset:What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted):}} * earliest: automatically reset the offset to the earliest offset * latest: automatically reset the offset to the latest offset * none: throw exception to the consumer if no previous offset is found for the consumer's group * anything else: throw exception to the consumer.{quote} for example, suppose data in Kafka is retained for 7 days and you just want to read from the data of yesterday, if you could not control the start timestamp, you can read from the earliest offset, it's very inefficient. If supporting to consume from special timestamp will be a good idea for some cases. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. > In other words, if the app runs at first time and user want to consume the > past data, user must set the value of > '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. > {quote}{{auto.offset.reset:What to do when there is no initial offset in > Kafka or if the current offset does not exist any more on the server (e.g. > because that data has been deleted):}} > * earliest: automatically reset the offset to the earliest offset > * latest: automatically reset the offset to the latest offset > * none: throw exception to the consumer if no previous offset is found for > the consumer's group > * anything else: throw exception to the consumer.{quote} > for example, suppose data in Kafka is retained for 7 days and you just want > to read from the data of yesterday, if you could not control the start > timestamp, you can only read from the earliest offset, it's very inefficient. > If supporting to consume from special timestamp in KafkaAdapter will be a > good idea for some cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated CALCITE-3073: --- Description: Currently the KafkaAdapter consumes data from default offset(latest/earliest/last_offset) and runs forever. In other words, if the app runs at first time and user want to consume the past data, user must set the value of '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. {quote}{{auto.offset.reset:What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server (e.g. because that data has been deleted):}} * earliest: automatically reset the offset to the earliest offset * latest: automatically reset the offset to the latest offset * none: throw exception to the consumer if no previous offset is found for the consumer's group * anything else: throw exception to the consumer.{quote} for example, suppose data in Kafka is retained for 7 days and you just want to read from the data of yesterday, if you could not control the start timestamp, you can read from the earliest offset, it's very inefficient. If supporting to consume from special timestamp will be a good idea for some cases. was:Currently the KafkaAdapter consumes data from default offset(latest/earliest/last_offset) and runs forever. With support of from/to timestamp/offset, it's more flexible for users to query a specific range. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. > In other words, if the app runs at first time and user want to consume the > past data, user must set the value of > '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*. > {quote}{{auto.offset.reset:What to do when there is no initial offset in > Kafka or if the current offset does not exist any more on the server (e.g. > because that data has been deleted):}} > * earliest: automatically reset the offset to the earliest offset > * latest: automatically reset the offset to the latest offset > * none: throw exception to the consumer if no previous offset is found for > the consumer's group > * anything else: throw exception to the consumer.{quote} > for example, suppose data in Kafka is retained for 7 days and you just want > to read from the data of yesterday, if you could not control the start > timestamp, you can read from the earliest offset, it's very inefficient. If > supporting to consume from special timestamp will be a good idea for some > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858178#comment-16858178 ] Botong Huang edited comment on CALCITE-3118 at 6/6/19 11:29 PM: Agree in general. But in this test case, the rule in this unit test (AssertOperandsDifferentRule) is to match a parent and its first and second child, it is wrong to return a match with a relNode and both its second child. was (Author: botong): Agree in general. But in this test case, the rule (AssertOperandsDifferentRule) is to match a parent and its first and second child, it is wrong to return a match with a relNode and both its second child. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858178#comment-16858178 ] Botong Huang commented on CALCITE-3118: --- Agree in general. But in this test case, the rule (AssertOperandsDifferentRule) is to match a parent and its first and second child, it is wrong to return a match with a relNode and both its second child. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3116) Upgrade to Avatica 1.15
[ https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis resolved CALCITE-3116. -- Resolution: Fixed Fixed in [614b4350f558a4fabaafc06fef9689ec4c267994|https://github.com/apache/calcite/commit/614b4350f558a4fabaafc06fef9689ec4c267994]! > Upgrade to Avatica 1.15 > --- > > Key: CALCITE-3116 > URL: https://issues.apache.org/jira/browse/CALCITE-3116 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858128#comment-16858128 ] Stamatis Zampetakis commented on CALCITE-3115: -- Reviewing and testing now! > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2829) Use consistent types when processing ranges
[ https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858092#comment-16858092 ] Juhwan Kim commented on CALCITE-2829: - [~danny0405]. I agree with your comment. Implicit type conversion would fix this issue, and I also think that it is a much better idea. I uploaded my PR with a simple test case. Do you mind adding that to your PR? Then, I think we could resolve this ticket along with CALCITE-2302. > Use consistent types when processing ranges > --- > > Key: CALCITE-2829 > URL: https://issues.apache.org/jira/browse/CALCITE-2829 > Project: Calcite > Issue Type: Bug >Reporter: Siddharth Teotia >Assignee: Juhwan Kim >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Range expressions like = 'literal' AND < 'literal' trigger > ClassCastException as literal are implicitly casted differently between =/<> > operators and other comparison operators. Apply the same casting rules for > comparison to =/<> calls when processing ranges, so that all the terms have > the same type for literals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858067#comment-16858067 ] Vladimir Sitnikov commented on CALCITE-3118: [~botong], as far as I understand, there's nothing wrong with matching the same rel for both operands in the gneeric case. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858056#comment-16858056 ] Hongze Zhang commented on CALCITE-3073: --- A naive question: Then what should be implemented for this issue? By reading the issue description: {quote}With support of from/to timestamp/offset, it's more flexible for users to query a specific range. {quote} And the comment in PR: {quote}Given a query like SELECT * FROM KAFKA_TABLE WHERE KAFKA_TIMESTAMP > CURRENT_TIMESTAMP-1, KafkaAdapter would take the filter clause KAFKA_TIMESTAMP > CURRENT_TIMESTAMP-1 directly. {quote} Didn't we already support SQL like {{SELECT STREAM * FROM KAFKA_TABLE WHERE MSG_TIMESTAMP > CURRENT_TIMESTAMP - 1}} in Kafka adaptor (Maybe tweaking on data types is needed)? Or the topic is more about performance improvement? > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858053#comment-16858053 ] Botong Huang commented on CALCITE-3118: --- Sure, supposedly this rule should only match once, with left child operand == leftPhy (PHYS, label = a) and right child operand == rightPhy (PHYS_2, label == b). However, without the fix, when rightPhy gets generated, it triggers two match attempts. The first one is expected as above. In the second one, rightPhy (PHYS_2, label == b) matches the left child operand, then we attempt to match the PhyBiRel as parent operand. Supposedly this parent should not match, because rightPhy is PhyBiRel's second child, but rightPhy's matched left child operand is parent operand's first child. Because of the bug, it didn't skip as expected and still proceed with the matching. After parent is matched, then it attempt to match the right child operand, and again matched the rightPhy (PHYS_2, label == b), as expected because it is PhyBiRel's second child. Overall as a result, both child operand end up matching the same RelNode rightPhy (PHYS_2, label == b). Hope this is clearer now. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2826) Add versions of create() for LogicalSort and LogicalExchange that take cluster and trait set as input
[ https://issues.apache.org/jira/browse/CALCITE-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858052#comment-16858052 ] Juhwan Kim commented on CALCITE-2826: - Can someone review this pull request? > Add versions of create() for LogicalSort and LogicalExchange that take > cluster and trait set as input > - > > Key: CALCITE-2826 > URL: https://issues.apache.org/jira/browse/CALCITE-2826 > Project: Calcite > Issue Type: Task > Components: core >Reporter: Siddharth Teotia >Assignee: Juhwan Kim >Priority: Minor > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Add versions of create() for LogicalSort and LogicalExchange that take > cluster and trait set as input. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858043#comment-16858043 ] Vladimir Sitnikov commented on CALCITE-3118: [~botong], I reverted the fix, and the test you add fails as follows: {noformat}java.lang.AssertionError: rel#4:PhysLeafRel.PHYS_2(label=b) should be different from rel#4:PhysLeafRel.PHYS_2(label=b) at org.apache.calcite.plan.volcano.PlannerTests$AssertOperandsDifferentRule.onMatch(PlannerTests.java:278) at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208) at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631) at org.apache.calcite.plan.volcano.VolcanoPlannerTest.testMatchedOperandsDifferent(VolcanoPlannerTest.java:154) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) {noformat} Can you please clarify why {{rel#4:PhysLeafRel.PHYS_2(label=b)}} should be different from {{rel#4:PhysLeafRel.PHYS_2(label=b)}} ? > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858038#comment-16858038 ] Botong Huang commented on CALCITE-3118: --- New version pushed, can you take another look and see if it is better now? > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858027#comment-16858027 ] Vladimir Sitnikov commented on CALCITE-3118: {quote} is unit test required for this kind of fix{quote} It is. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858020#comment-16858020 ] Botong Huang commented on CALCITE-3118: --- Sure, let me try make the unit test more readable. On a side note, is unit test required for this kind of fix? > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858005#comment-16858005 ] Vladimir Sitnikov commented on CALCITE-3118: {quote} Without the one line fix, this assert will fail because{quote} What I mean is it is not clear what and why the test is trying to test. The thing is: you see an error. It might be obvious for you. However committers would have to support the test code should it fail for a random reason. That means, for committers (even for those who are not super familiar with Volcano) the code/test should be more or less understandable. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857997#comment-16857997 ] Botong Huang edited comment on CALCITE-3118 at 6/6/19 7:08 PM: --- Thanks [~vladimirsitnikov] for reviewing. The real check in the unit test is the assert inside TwoChildrenSameSetMatchRule. Without the one line fix, this assert will fail because it ends up matching the same PhysLeafRel for both child operands. I can add some comments in the unit test in needed. was (Author: botong): Thanks [~vladimirsitnikov] for reviewing. The real check in the unit test is the assert inside TwoChildrenSameSetMatchRule. Without the one line fix, this assert will fail because it ends up matching the same PhysLeafRel as both child operand. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857997#comment-16857997 ] Botong Huang commented on CALCITE-3118: --- Thanks [~vladimirsitnikov] for reviewing. The real check in the unit test is the assert inside TwoChildrenSameSetMatchRule. Without the one line fix, this assert will fail because it ends up matching the same PhysLeafRel as both child operand. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Sitnikov updated CALCITE-3118: --- Labels: pull-request-available (was: ) > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > Labels: pull-request-available > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[ https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857994#comment-16857994 ] Vladimir Sitnikov commented on CALCITE-3118: Frankly speaking, I have scanned through the added unit test, and its obscure what the test is doing and what it is asserting. Even though I have fixed bugs near VolcanoRuleCall, and the change looks simple, I can't commit this PR because I don't understand it. Sorry. > VolcanoRuleCall match parent child ordinal not properly checked > --- > > Key: CALCITE-3118 > URL: https://issues.apache.org/jira/browse/CALCITE-3118 > Project: Calcite > Issue Type: Bug >Reporter: Botong Huang >Priority: Major > > In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, > looking for parent operand match), we want to check that the matched parent > indeed has the previously matched child RelNode as a child with the expected > ordinal. However, there is a bug in this check. As a result, some incorrect > parent is not skipped as expected and matched incorrectly. See unit test > included in PR for a case that triggers this bug, where the same RelNode get > matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (CALCITE-3108) Babel parser should parse the PostgreSQL TEXT type
[ https://issues.apache.org/jira/browse/CALCITE-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Gelbana closed CALCITE-3108. - Resolution: Won't Fix Users can define their own types but the problem seems with the returned column type name. The metadata for the returned resultset for that query using calcite won't return the user-defiend type alias for the column type name, it will return whatever the user-define type was configured to act as (ex: {{VARCHAR}}). *Example:* The following code prints {{TEXT}} as expected {code:java} Connection connection = // A connection to a real PostgreSQL DB PreparedStatement statement = connection.prepareStatement("SELECT column1::TEXT FROM (VALUES (1, 2, 3)) a"); System.out.println(statement.executeQuery().getMetaData().getColumnTypeName(1)); {code} But the following code prints {{VARCHAR}} while I It should print {{TEXT}} instead {code:java} Connection connection = DriverManager.getConnection("jdbc:calcite:", info); connection.unwrap(CalciteConnection.class).getRootSchema().unwrap(CalciteSchema.class).add("TEXT", new RelProtoDataType() { @Override public RelDataType apply(RelDataTypeFactory factory) { return factory.createTypeWithNullability(factory.createSqlType(SqlTypeName.VARCHAR), false); } }); PreparedStatement statement = connection.prepareStatement("SELECT CAST(EXPR$0 AS text) FROM (VALUES (1, 2, 3)) a"); System.out.println(statement.executeQuery().getMetaData().getColumnTypeName(1)); {code} > Babel parser should parse the PostgreSQL TEXT type > -- > > Key: CALCITE-3108 > URL: https://issues.apache.org/jira/browse/CALCITE-3108 > Project: Calcite > Issue Type: Bug > Components: babel, core >Affects Versions: 1.19.0 >Reporter: Muhammad Gelbana >Assignee: Muhammad Gelbana >Priority: Minor > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Casting to PostgreSQL TEXT (ie. VARCHAR) isn't currently supported. The > following query would fail in parsing and execution. > {code:sql}SELECT CAST(EXPR$0 AS text) FROM (VALUES (1, 2, 3)){code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857871#comment-16857871 ] Xu Mingmin commented on CALCITE-3073: - Either field name is good to me, I do want to avoid the configuration in files. \{{MSG_TIMESTAMP}} is the name in the default converter, users may have their own. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857860#comment-16857860 ] Matt Wang commented on CALCITE-3073: [~zhztheplayer] We can use `MSG_TIMESTAMP` field to define the end timestamp(I almost forgot it), it's not necessary to define `KAFKA_TIMESTAMP` reserved keyword. Just setting the start timestamp can satisfy the case of querying a specific range. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857848#comment-16857848 ] Hongze Zhang commented on CALCITE-3073: --- I'am a little confused with the two columns, the new {{KAFKA_TIMESTAMP}} and the existing {{MSG_TIMESTAMP}} (sorry I'm not a Kafka expert :)). If we have to provide a different timestamp column, hopefully we can make them well documented so that users know which one they should use. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
Botong Huang created CALCITE-3118: - Summary: VolcanoRuleCall match parent child ordinal not properly checked Key: CALCITE-3118 URL: https://issues.apache.org/jira/browse/CALCITE-3118 Project: Calcite Issue Type: Bug Reporter: Botong Huang In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, looking for parent operand match), we want to check that the matched parent indeed has the previously matched child RelNode as a child with the expected ordinal. However, there is a bug in this check. As a result, some incorrect parent is not skipped as expected and matched incorrectly. See unit test included in PR for a case that triggers this bug, where the same RelNode get matched for two operands at the same time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857838#comment-16857838 ] Matt Wang commented on CALCITE-3073: [~zhztheplayer] I have updated the JIRA title, now the PR just support to consume form the special timestamp. Now we can set the start timestamp in `modern.json` file, then we can read from the special timestamp. I do not know it's necessary to support the end timestamp, because the data in Kafka is a streaming data and it is producing all the time. But like [~mingmxu] suggested, if we add a special reserved keyword(`KAFKA_TIMESTAMP`) in KafkaAdapter, we can query like `SELECT * FROM KAFKA_TABLE WHERE KAFKA_TIMESTAMP > CURRENT_TIMESTAMP-1`, it will make easy to use, but I'm not sure is a good idea. > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2906) Record DOT extension does not distinguish between simple and compound identifiers
[ https://issues.apache.org/jira/browse/CALCITE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857835#comment-16857835 ] Hongze Zhang commented on CALCITE-2906: --- Marked fix version as 1.21.0. > Record DOT extension does not distinguish between simple and compound > identifiers > - > > Key: CALCITE-2906 > URL: https://issues.apache.org/jira/browse/CALCITE-2906 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Rong Rong >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > > Currently the DOT extensions only supports Simple Identifiers: > For example: > * *{{tbl.foo(0).col.bar}}* is parsed as: *{{((`TBL`.`FOO`(0).`COL`).`BAR`)}}* > However, this is not the case in regular Expression field, for example: > * *{{tbl.col.bar}}* is parsed as: *{{`TBL`.`COL`.`BAR`}}* > Parser should support both compound identifier, and simple identifier chain > after a record type SqlNode. > Similarly, parsing should also be consistent for other types of expression, > for example > * *{{tbl.arr[0].foo.bar}}* is currently parsed as: > *{{((`TBL`.`ARR`[0].`FOO`).`BAR`)}}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2906) Record DOT extension does not distinguish between simple and compound identifiers
[ https://issues.apache.org/jira/browse/CALCITE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongze Zhang updated CALCITE-2906: -- Fix Version/s: 1.21.0 > Record DOT extension does not distinguish between simple and compound > identifiers > - > > Key: CALCITE-2906 > URL: https://issues.apache.org/jira/browse/CALCITE-2906 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Rong Rong >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > > Currently the DOT extensions only supports Simple Identifiers: > For example: > * *{{tbl.foo(0).col.bar}}* is parsed as: *{{((`TBL`.`FOO`(0).`COL`).`BAR`)}}* > However, this is not the case in regular Expression field, for example: > * *{{tbl.col.bar}}* is parsed as: *{{`TBL`.`COL`.`BAR`}}* > Parser should support both compound identifier, and simple identifier chain > after a record type SqlNode. > Similarly, parsing should also be consistent for other types of expression, > for example > * *{{tbl.arr[0].foo.bar}}* is currently parsed as: > *{{((`TBL`.`ARR`[0].`FOO`).`BAR`)}}* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3073) Support from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated CALCITE-3073: --- Summary: Support from timestamp in KafkaAdapter (was: Support both from timestamp in KafkaAdapter) > Support from timestamp in KafkaAdapter > -- > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated CALCITE-3073: --- Summary: Support to consume from timestamp in KafkaAdapter (was: Support from timestamp in KafkaAdapter) > Support to consume from timestamp in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3073) Support both from timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang updated CALCITE-3073: --- Summary: Support both from timestamp in KafkaAdapter (was: Support both from/to timestamp in KafkaAdapter) > Support both from timestamp in KafkaAdapter > --- > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3114) Support websocket transport
[ https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857785#comment-16857785 ] Leo Tu edited comment on CALCITE-3114 at 6/6/19 3:19 PM: - Currently, the client interacts with the server using the Http Request-Response mode. If you can provide Client and Server interaction using websocket's frame to implement Request-Response that will be similar to the traditional JDBC Driver TCP/IP. (1) Websocket client <-> (2) Websocket server <-> (3) Database Server Transportation between: (1) & (2) Websocket Protocol (Connection/Statement/ResultSet Serializable) (2) & (3) TCP/IP vender's JDBC driver (Connection/Statement/ResultSet) was (Author: leotu): Currently, the client interacts with the server using the Http Request-Response mode. If you can provide Client and Server interaction using websocket's frame to implement Request-Response that will be similar to the traditional JDBC Driver TCP/IP. (1) Websocket client <-> (2) Websocket server <-> (3) Database Server Transportation between: (1) & (2) Websocket Protocol (Connection/Statement/ResultSet Serializable) (2) & (3) TPC/IP (Connection/Statement/ResultSet) > Support websocket transport > --- > > Key: CALCITE-3114 > URL: https://issues.apache.org/jira/browse/CALCITE-3114 > Project: Calcite > Issue Type: Wish >Reporter: Leo Tu >Priority: Minor > > it would be greatly improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3114) Support websocket transport
[ https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857785#comment-16857785 ] Leo Tu edited comment on CALCITE-3114 at 6/6/19 3:17 PM: - Currently, the client interacts with the server using the Http Request-Response mode. If you can provide Client and Server interaction using websocket's frame to implement Request-Response that will be similar to the traditional JDBC Driver TCP/IP. (1) Websocket client <-> (2) Websocket server <-> (3) Database Server Transportation between: (1) & (2) Websocket Protocol (Connection/Statement/ResultSet Serializable) (2) & (3) TPC/IP (Connection/Statement/ResultSet) was (Author: leotu): Currently, the client interacts with the server using the Http Request-Response mode. If you can provide Client and Server interaction using websocket's frame to implement Request-Response that will be similar to the traditional JDBC Driver TCP/IP. (1) Websocket client <> (2) Websocket server <> (3) Database Server (1) & (2) is Websocket Protocal (Connection/Statement/ResultSet) (2) & (3) TPC/IP (Connection/Statement/ResultSet) > Support websocket transport > --- > > Key: CALCITE-3114 > URL: https://issues.apache.org/jira/browse/CALCITE-3114 > Project: Calcite > Issue Type: Wish >Reporter: Leo Tu >Priority: Minor > > it would be greatly improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3117) Distinguish Calcite's public APIs
[ https://issues.apache.org/jira/browse/CALCITE-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857750#comment-16857750 ] Ivan Grgurina commented on CALCITE-3117: RelOptRule and Program should definitely be considered public APIs. > Distinguish Calcite's public APIs > - > > Key: CALCITE-3117 > URL: https://issues.apache.org/jira/browse/CALCITE-3117 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > > The goal of this issue is to identify and mark Calcite's public APIs. This > can have bring various benefits to the project: > * provide more flexibility to developers; > * speed-up reviews; > * enhance backward compatibility; > * facilitate the life of end-users who will be exposed to fewer classes and > interfaces. > Let's list here packages, interfaces, and classes that should be part of the > public API. As a first step let's focus exclusively on completing the list > and not argue about others proposals. It will be easier to solve > disagreements afterwards as soon as we have a rather complete list. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3116) Upgrade to Avatica 1.15
[ https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Mior updated CALCITE-3116: -- Fix Version/s: 1.20.0 > Upgrade to Avatica 1.15 > --- > > Key: CALCITE-3116 > URL: https://issues.apache.org/jira/browse/CALCITE-3116 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3116) Upgrade to Avatica 1.15
[ https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857745#comment-16857745 ] Michael Mior commented on CALCITE-3116: --- Sounds reasonable to me. It would be great if this could be tested and merged today. > Upgrade to Avatica 1.15 > --- > > Key: CALCITE-3116 > URL: https://issues.apache.org/jira/browse/CALCITE-3116 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3117) Distinguish Calcite's public APIs
Stamatis Zampetakis created CALCITE-3117: Summary: Distinguish Calcite's public APIs Key: CALCITE-3117 URL: https://issues.apache.org/jira/browse/CALCITE-3117 Project: Calcite Issue Type: Task Reporter: Stamatis Zampetakis Assignee: Stamatis Zampetakis The goal of this issue is to identify and mark Calcite's public APIs. This can have bring various benefits to the project: * provide more flexibility to developers; * speed-up reviews; * enhance backward compatibility; * facilitate the life of end-users who will be exposed to fewer classes and interfaces. Let's list here packages, interfaces, and classes that should be part of the public API. As a first step let's focus exclusively on completing the list and not argue about others proposals. It will be easier to solve disagreements afterwards as soon as we have a rather complete list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3116) Upgrade to Avatica 1.15
[ https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857728#comment-16857728 ] Stamatis Zampetakis commented on CALCITE-3116: -- Hey [~michaelmior], this is kind of trivial can we get this into 1.20? > Upgrade to Avatica 1.15 > --- > > Key: CALCITE-3116 > URL: https://issues.apache.org/jira/browse/CALCITE-3116 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3073) Support both from/to timestamp in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Mingmin updated CALCITE-3073: Summary: Support both from/to timestamp in KafkaAdapter (was: Support both from/to timestamp/offset in KafkaAdapter) > Support both from/to timestamp in KafkaAdapter > -- > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3116) Upgrade to Avatica 1.15
[ https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3116: Labels: pull-request-available (was: ) > Upgrade to Avatica 1.15 > --- > > Key: CALCITE-3116 > URL: https://issues.apache.org/jira/browse/CALCITE-3116 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3088) Multi-groupsets query failed due to data type preservation check
[ https://issues.apache.org/jira/browse/CALCITE-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3088: Labels: pull-request-available (was: ) > Multi-groupsets query failed due to data type preservation check > > > Key: CALCITE-3088 > URL: https://issues.apache.org/jira/browse/CALCITE-3088 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Feng Zhu >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > > We encounter the proble in our environment. It can be reproduced by the > modification on _testCube_ in *SqlToRelConverterTest*. > {code:java} > @Test public void testCube() { > // cube(a, b) is {(a, b), (a), (b), ()} > final String sql = "select a+1\n" > + "from (values (1, 2, 3, 4)) as t(a, b, c, d)\n" > + "group by cube(a, b)"; > sql(sql).ok(); > } > {code} > This query will fail with following exception: > {code:java} > java.lang.AssertionError: Conversion to relational algebra failed to preserve > datatypes: > validated type: > RecordType(INTEGER NOT NULL EXPR$0) NOT NULL > converted type: > RecordType(INTEGER EXPR$0) NOT NULL > rel: > LogicalProject(EXPR$0=[+($0, 1)]) > LogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {1}, {}]]) > LogicalProject(A=[$0], B=[$1]) > LogicalValues(tuples=[[{ 1, 2, 3, 4 }]]) > at > org.apache.calcite.sql2rel.SqlToRelConverter.checkConvertedType(SqlToRelConverter.java:406) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:523) > at > org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:610) > at > org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:723) > at > org.apache.calcite.test.SqlToRelConverterTest$Sql.convertsTo(SqlToRelConverterTest.java:3073) > at > org.apache.calcite.test.SqlToRelConverterTest$Sql.ok(SqlToRelConverterTest.java:3065) > at > org.apache.calcite.test.SqlToRelConverterTest.testCube(SqlToRelConverterTest.java:474) > {code} > The exception stacktrace is very clear. In validation phase, SqlValidator > does not take such case into consideration, and consequently the type of > {color:#FF}EXPR$0(a+1){color} cannot be inferred as nullable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3116) Upgrade to Avatica 1.15
Stamatis Zampetakis created CALCITE-3116: Summary: Upgrade to Avatica 1.15 Key: CALCITE-3116 URL: https://issues.apache.org/jira/browse/CALCITE-3116 Project: Calcite Issue Type: Task Reporter: Stamatis Zampetakis Assignee: Stamatis Zampetakis -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3073) Support both from/to timestamp/offset in KafkaAdapter
[ https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857627#comment-16857627 ] Hongze Zhang commented on CALCITE-3073: --- > Agree, let's focus on timestamp support here. Should we update JIRA title accordingly? > Support both from/to timestamp/offset in KafkaAdapter > - > > Key: CALCITE-3073 > URL: https://issues.apache.org/jira/browse/CALCITE-3073 > Project: Calcite > Issue Type: Improvement > Components: kafka-adapter >Reporter: Xu Mingmin >Assignee: Matt Wang >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Currently the KafkaAdapter consumes data from default > offset(latest/earliest/last_offset) and runs forever. With support of from/to > timestamp/offset, it's more flexible for users to query a specific range. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857566#comment-16857566 ] TANG Wen-hui edited comment on CALCITE-3115 at 6/6/19 11:48 AM: Once JdbcAggregateRule(JDBC.test, RelFactories.LOGICAL_BUILDER) was added to ruleset, JdbcAggregateRule(JDBC.test2, RelFactories.LOGICAL_BUILDER) cannot be added to same planner's ruleset, since they have same description. And the hashCode() of RelOptRule are: {code:java} public int hashCode() { // Conventionally, hashCode() and equals() should use the same // criteria, whereas here we only look at the description. This is // okay, because the planner requires all rule instances to have // distinct descriptions. return description.hashCode(); } {code} (JDBC.test and JDBC.test2 are JdbcConvention.) So I have changed the description of JdbcRules. was (Author: winipanda): Once JdbcAggregateRule(JDBC.test, RelFactories.LOGICAL_BUILDER) was added to ruleset, JdbcAggregateRule(JDBC.test2, RelFactories.LOGICAL_BUILDER) cannot be added to same planner's ruleset, since they have same description. And the hashCode() of RelOptRule are: public int hashCode() { // Conventionally, hashCode() and equals() should use the same // criteria, whereas here we only look at the description. This is // okay, because the planner requires all rule instances to have // distinct descriptions. return description.hashCode(); } (JDBC.test and JDBC.test2 are JdbcConvention.) So I have changed the description of JdbcRules. > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3115: Labels: pull-request-available (was: ) > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857566#comment-16857566 ] TANG Wen-hui commented on CALCITE-3115: --- Once JdbcAggregateRule(JDBC.test, RelFactories.LOGICAL_BUILDER) was added to ruleset, JdbcAggregateRule(JDBC.test2, RelFactories.LOGICAL_BUILDER) cannot be added to same planner's ruleset, since they have same description. And the hashCode() of RelOptRule are: public int hashCode() { // Conventionally, hashCode() and equals() should use the same // criteria, whereas here we only look at the description. This is // okay, because the planner requires all rule instances to have // distinct descriptions. return description.hashCode(); } (JDBC.test and JDBC.test2 are JdbcConvention.) So I have changed the description of JdbcRules. > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TANG Wen-hui updated CALCITE-3115: -- Description: When we use Calcite via JDBC to run a sql which involves two difference jdbc schema: {code:java} select * from (select "a",max("b") as max_b, sum("c") as sum_c from "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from "test2"."temp2" group by "a","b","c" {code} the sql get a plan like that: {code:java} EnumerableUnion(all=[false]) JdbcToEnumerableConverter JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) JdbcSort(fetch=[10]) JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 100))]) JdbcTableScan(table=[[test, temp]]) EnumerableAggregate(group=[{0, 1, 2}]) JdbcToEnumerableConverter JdbcTableScan(table=[[test2, temp2]]) {code} And the EnumerableAggregate for table test2.temp2 cannot be converted to JdbcAggregate. was: When we use Calcite via JDBC to run a sql which involves two difference jdbc schema: {code:java} select * from (select "a",max("b") as max_b, sum("c") as sum_c from "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from "test2"."temp2" group by "a","b","c" {code} the sql get a plan like that: {code:java} EnumerableUnion(all=[false]) JdbcToEnumerableConverter JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) JdbcSort(fetch=[10]) JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 100))]) JdbcTableScan(table=[[test, temp]]) EnumerableAggregate(group=[{0, 1, 2}]) JdbcToEnumerableConverter JdbcTableScan(table=[[test2, temp2]]) {code} And the EnumerableAggregate for table test2.temp2 cannot be convert to JdbcAggregate. > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-2460) Implement TO_BASE64 and FROM_BASE64 functions
[ https://issues.apache.org/jira/browse/CALCITE-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TANG Wen-hui reassigned CALCITE-2460: - Assignee: TANG Wen-hui > Implement TO_BASE64 and FROM_BASE64 functions > - > > Key: CALCITE-2460 > URL: https://issues.apache.org/jira/browse/CALCITE-2460 > Project: Calcite > Issue Type: Improvement >Reporter: vinoyang >Assignee: TANG Wen-hui >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > refer to mysql TO_BASE64 function : > [https://dev.mysql.com/doc/refman/5.6/en/string-functions.html#function_to-base64] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-2460) Implement TO_BASE64 and FROM_BASE64 functions
[ https://issues.apache.org/jira/browse/CALCITE-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TANG Wen-hui updated CALCITE-2460: -- Summary: Implement TO_BASE64 and FROM_BASE64 functions (was: Add implementation of To_Base64 to SqlFunctions) > Implement TO_BASE64 and FROM_BASE64 functions > - > > Key: CALCITE-2460 > URL: https://issues.apache.org/jira/browse/CALCITE-2460 > Project: Calcite > Issue Type: Improvement >Reporter: vinoyang >Priority: Minor > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > refer to mysql TO_BASE64 function : > [https://dev.mysql.com/doc/refman/5.6/en/string-functions.html#function_to-base64] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
TANG Wen-hui created CALCITE-3115: - Summary: Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet. Key: CALCITE-3115 URL: https://issues.apache.org/jira/browse/CALCITE-3115 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.19.0 Reporter: TANG Wen-hui Assignee: TANG Wen-hui When we use Calcite via JDBC to run a sql which involves two difference jdbc schema: {code:java} select * from (select "a",max("b") as max_b, sum("c") as sum_c from "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from "test2"."temp2" group by "a","b","c" {code} the sql get a plan like that: {code:java} EnumerableUnion(all=[false]) JdbcToEnumerableConverter JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) JdbcSort(fetch=[10]) JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 100))]) JdbcTableScan(table=[[test, temp]]) EnumerableAggregate(group=[{0, 1, 2}]) JdbcToEnumerableConverter JdbcTableScan(table=[[test2, temp2]]) {code} And the EnumerableAggregate for table test2.temp2 cannot be convert to JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3114) Support websocket transport
[ https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857429#comment-16857429 ] Danny Chan commented on CALCITE-3114: - What do you mean by websocket transport ? > Support websocket transport > --- > > Key: CALCITE-3114 > URL: https://issues.apache.org/jira/browse/CALCITE-3114 > Project: Calcite > Issue Type: Wish >Reporter: Leo Tu >Priority: Minor > > it would be greatly improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3114) Support websocket transport
Leo Tu created CALCITE-3114: --- Summary: Support websocket transport Key: CALCITE-3114 URL: https://issues.apache.org/jira/browse/CALCITE-3114 Project: Calcite Issue Type: Wish Reporter: Leo Tu it would be greatly improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857366#comment-16857366 ] Danny Chan commented on CALCITE-2280: - This is actually what we have to do in Flink custom parser, for computation engines like Flink, one big topic is the Hive compatibility, for example, if we want to add "partition" grammar like {code:java} insert into T partition(col1=vl) select from S; {code} What we have to do is to copy the whole java cc parse block for insert statement and add our custom parse logic. We also have to add our SqlConformance Hive, but the SqlConformanceEnum is enum, so i also have to hard code the conformance in the parser template. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3113) Equivalent MutableAggregates with different row types fail with AssertionError
Feng Zhu created CALCITE-3113: - Summary: Equivalent MutableAggregates with different row types fail with AssertionError Key: CALCITE-3113 URL: https://issues.apache.org/jira/browse/CALCITE-3113 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.19.0 Reporter: Feng Zhu Add test case in MaterializationTest: {code:java} @Test public void testAggregateAlias() { checkMaterialize( "select count(*) as c from \"emps\" group by \"empid\"", "select count(*) + 1 as c from \"emps\" group by \"empid\""); } {code} It fails due to different rowtype. {code:java} java.lang.AssertionError at org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:504) at org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:465) at org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56) at org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200) at org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72) at org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:347) {code} However, according to MutableAggregate's hashCode implementation, this materialization can be reused, i.e., queryDescedant=targetDescendant. {code:java} queryDescendant: RecordType(JavaType(int) empid, BIGINT $f1) = Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()]) Project(projects: [$0]) Scan(table: [hr, emps]) targetDescendant: RecordType(JavaType(int) empid, BIGINT C) = Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()]) Project(projects: [$0]) Scan(table: [hr, emps]) {code} So, how can we align them? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3087) AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its Mapping breaks ordering
[ https://issues.apache.org/jira/browse/CALCITE-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857333#comment-16857333 ] Feng Zhu commented on CALCITE-3087: --- Therefore, when a posList generates IdentifyMapping or (partially IdentifyMapping + \{i->j, j->i}), i.e., getSource(i)==getTarget(i), *MutableRels.createProject* can be OK. {code:java} posList={0,1,2} mapping={0->0, 1->1, 2->2} OK posList={1,0} mapping={0->1, 1->0} OK posList={1,0,2} mapping={0->1, 1->0, 2->2} OK posList={1,2,0} mapping={0->1, 1->2, 2->0} NO {code} Moreover, I have checked all code and found that: *MutableRels.createProject* is only called in unifyAggregates under condition (_query.{color:#660e7a}groupSets{color}.equals(target.{color:#660e7a}groupSets{color})_) It seems that the mapping _{color:#FF}isIdentify{color}_ from this entrance, and consequently I can't figure out a test case for the logic inconsistency. > AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its > Mapping breaks ordering > --- > > Key: CALCITE-3087 > URL: https://issues.apache.org/jira/browse/CALCITE-3087 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Haisheng Yuan >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Add the following test to MaterializationTest: > {code:java} > @Test public void testAggregateGroupSetsRollUp0() { > checkMaterialize( > "select \"empid\", \"deptno\", count(*) as c, sum(\"salary\") as s > from \"emps\" " > + "group by \"empid\", \"deptno\"", > "select count(*) + 1 as c, \"deptno\" from \"emps\" group by > cube(\"deptno\", \"empid\")", > HR_FKUK_MODEL, > CalciteAssert.checkResultContains( > "EnumerableCalc(expr#0..2=[{inputs}], expr#3=[1], " > + "expr#4=[+($t2, $t3)], C=[$t4], deptno=[$t1])\n" > + " EnumerableAggregate(group=[{0, 1}], groups=[[{0, 1}, > {0}, {1}, {}]], agg#0=[$SUM0($2)])\n" > + "EnumerableTableScan(table=[[hr, m0]])")); > } > {code} > We got error: > {code:java} > Caused by: java.lang.IllegalArgumentException: must be sorted: [{0, 1}, {1}, > {0}, {}] > at > org.apache.calcite.rel.core.Aggregate$Group.induce(Aggregate.java:472) > at > org.apache.calcite.rel.mutable.MutableAggregate.getGroupType(MutableAggregate.java:83) > at > org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1281) > at > org.apache.calcite.plan.SubstitutionVisitor$AggregateOnProjectToAggregateUnifyRule.apply(SubstitutionVisitor.java:1345) > at > org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:531) > at > org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:466) > at > org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56) > at > org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200) > at > org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:348) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:584) > {code} > The reason is that Mappings.apply2 doesn't sort. -- This message was sent by Atlassian JIRA (v7.6.3#76005)