[jira] [Commented] (CALCITE-3117) Distinguish Calcite's public APIs

2019-06-06 Thread Danny Chan (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858282#comment-16858282
 ] 

Danny Chan commented on CALCITE-3117:
-

The parser and sql node should be split out of the calcite-core, user usually 
only want to parse their sql and get a AST.

> Distinguish Calcite's public APIs
> -
>
> Key: CALCITE-3117
> URL: https://issues.apache.org/jira/browse/CALCITE-3117
> Project: Calcite
>  Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Minor
>
> The goal of this issue is to identify and mark Calcite's public APIs. This 
> can have bring various benefits to the project:
>  * provide more flexibility to developers;
>  * speed-up reviews;
>  * enhance backward compatibility;
>  * facilitate the life of end-users who will be exposed to fewer classes and 
> interfaces.
> Let's list here packages, interfaces, and classes that should be part of the 
> public API. As a first step let's focus exclusively on completing the list 
> and not argue about others proposals. It will be easier to solve 
> disagreements afterwards as soon as we have a rather complete list.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (CALCITE-2829) Use consistent types when processing ranges

2019-06-06 Thread Danny Chan (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858261#comment-16858261
 ] 

Danny Chan edited comment on CALCITE-2829 at 6/7/19 2:40 AM:
-

Sure, the more test cases the better, i would link this issue with 
CALCITE-2302, thanks for your test cases.


was (Author: danny0405):
Sure, the more test cases the better, i would link this issue with CALCITE-2302

> Use consistent types when processing ranges
> ---
>
> Key: CALCITE-2829
> URL: https://issues.apache.org/jira/browse/CALCITE-2829
> Project: Calcite
>  Issue Type: Bug
>Reporter: Siddharth Teotia
>Assignee: Juhwan Kim
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Range expressions like  = 'literal' AND  < 'literal' trigger 
> ClassCastException as literal are implicitly casted differently between =/<> 
> operators and other comparison operators. Apply the same casting rules for 
> comparison to =/<> calls when processing ranges, so that all the terms have 
> the same type for literals.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-2829) Use consistent types when processing ranges

2019-06-06 Thread Danny Chan (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858261#comment-16858261
 ] 

Danny Chan commented on CALCITE-2829:
-

Sure, the more test cases the better, i would link this issue with CALCITE-2302

> Use consistent types when processing ranges
> ---
>
> Key: CALCITE-2829
> URL: https://issues.apache.org/jira/browse/CALCITE-2829
> Project: Calcite
>  Issue Type: Bug
>Reporter: Siddharth Teotia
>Assignee: Juhwan Kim
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Range expressions like  = 'literal' AND  < 'literal' trigger 
> ClassCastException as literal are implicitly casted differently between =/<> 
> operators and other comparison operators. Apply the same casting rules for 
> comparison to =/<> calls when processing ranges, so that all the terms have 
> the same type for literals.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3080) support user-specified row schema in KafkaAdapter

2019-06-06 Thread Xu Mingmin (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858246#comment-16858246
 ] 

Xu Mingmin commented on CALCITE-3080:
-

Ensure the timestamp field in CALCITE-3073 always exist.

> support user-specified row schema in KafkaAdapter
> -
>
> Key: CALCITE-3080
> URL: https://issues.apache.org/jira/browse/CALCITE-3080
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Xu Mingmin
>Priority: Major
>
> Currently rowSchema is defined by extending `KafkaRowConverter` only. Feature 
> in this task would support user-defined row schema when creating a table.
>  
> Here's one proposed interface in 
> https://github.com/apache/calcite/pull/1127/files#r283913361
> {code}
> interface KafkaRowConverter {
> RelDataType rowDataType();
> Object[] toRow(ConsumerRecord message);
> }
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Xu Mingmin (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858245#comment-16858245
 ] 

Xu Mingmin commented on CALCITE-3073:
-

{{MSG_TIMESTAMP}} is good, but be aware that these columns are only a default 
schema, no guarantee to exist as users may define a total different schema in 
{{KafkaRowConverter}}. Once the name is decided, we need a constrain in 
https://issues.apache.org/jira/browse/CALCITE-3080.

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever.
> In other words, if the app runs at first time and user want to consume the 
> past data, user must set the value of 
> '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*.
> {quote}{{auto.offset.reset:What to do when there is no initial offset in 
> Kafka or if the current offset does not exist any more on the server (e.g. 
> because that data has been deleted):}}
>  * earliest: automatically reset the offset to the earliest offset
>  * latest: automatically reset the offset to the latest offset
>  * none: throw exception to the consumer if no previous offset is found for 
> the consumer's group
>  * anything else: throw exception to the consumer.{quote}
> for example, suppose data in Kafka is retained for 7 days and you just want 
> to read from the data of yesterday, if you could not control the start 
> timestamp, you can only read from the earliest offset, it's very inefficient. 
> If supporting to consume from special timestamp in KafkaAdapter will be a 
> good idea for some cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858187#comment-16858187
 ] 

Matt Wang commented on CALCITE-3073:


[~mingmxu] Using {color:#707070}*MSG_TIMESTAMP*{color} filed I think it's a 
better idea, when user use KafkaAdapter, they know these field is already 
defined:
{quote}| MSG_PARTITION | MSG_TIMESTAMP | MSG_OFFSET | MSG_KEY_BYTES |           
              MSG_VALUE_BYTES 
{quote}
 

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever.
> In other words, if the app runs at first time and user want to consume the 
> past data, user must set the value of 
> '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*.
> {quote}{{auto.offset.reset:What to do when there is no initial offset in 
> Kafka or if the current offset does not exist any more on the server (e.g. 
> because that data has been deleted):}}
>  * earliest: automatically reset the offset to the earliest offset
>  * latest: automatically reset the offset to the latest offset
>  * none: throw exception to the consumer if no previous offset is found for 
> the consumer's group
>  * anything else: throw exception to the consumer.{quote}
> for example, suppose data in Kafka is retained for 7 days and you just want 
> to read from the data of yesterday, if you could not control the start 
> timestamp, you can only read from the earliest offset, it's very inefficient. 
> If supporting to consume from special timestamp in KafkaAdapter will be a 
> good idea for some cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858184#comment-16858184
 ] 

Matt Wang commented on CALCITE-3073:


[~zhztheplayer] I have modified the description of the JIRA.

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever.
> In other words, if the app runs at first time and user want to consume the 
> past data, user must set the value of 
> '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*.
> {quote}{{auto.offset.reset:What to do when there is no initial offset in 
> Kafka or if the current offset does not exist any more on the server (e.g. 
> because that data has been deleted):}}
>  * earliest: automatically reset the offset to the earliest offset
>  * latest: automatically reset the offset to the latest offset
>  * none: throw exception to the consumer if no previous offset is found for 
> the consumer's group
>  * anything else: throw exception to the consumer.{quote}
> for example, suppose data in Kafka is retained for 7 days and you just want 
> to read from the data of yesterday, if you could not control the start 
> timestamp, you can only read from the earliest offset, it's very inefficient. 
> If supporting to consume from special timestamp in KafkaAdapter will be a 
> good idea for some cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Wang updated CALCITE-3073:
---
Description: 
Currently the KafkaAdapter consumes data from default 
offset(latest/earliest/last_offset) and runs forever.

In other words, if the app runs at first time and user want to consume the past 
data, user must set the value of '{color:#707070}*auto.offset.reset'*{color} 
parameter to *earliest*.
{quote}{{auto.offset.reset:What to do when there is no initial offset in Kafka 
or if the current offset does not exist any more on the server (e.g. because 
that data has been deleted):}}
 * earliest: automatically reset the offset to the earliest offset
 * latest: automatically reset the offset to the latest offset
 * none: throw exception to the consumer if no previous offset is found for the 
consumer's group
 * anything else: throw exception to the consumer.{quote}
for example, suppose data in Kafka is retained for 7 days and you just want to 
read from the data of yesterday, if you could not control the start timestamp, 
you can only read from the earliest offset, it's very inefficient. If 
supporting to consume from special timestamp in KafkaAdapter will be a good 
idea for some cases.

  was:
Currently the KafkaAdapter consumes data from default 
offset(latest/earliest/last_offset) and runs forever.

In other words, if the app runs at first time and user want to consume the past 
data, user must set the value of '{color:#707070}*auto.offset.reset'*{color} 
parameter to *earliest*.
{quote}{{auto.offset.reset:What to do when there is no initial offset in Kafka 
or if the current offset does not exist any more on the server (e.g. because 
that data has been deleted):}}
 * earliest: automatically reset the offset to the earliest offset
 * latest: automatically reset the offset to the latest offset
 * none: throw exception to the consumer if no previous offset is found for the 
consumer's group
 * anything else: throw exception to the consumer.{quote}
for example, suppose data in Kafka is retained for 7 days and you just want to 
read from the data of yesterday, if you could not control the start timestamp, 
you can read from the earliest offset, it's very inefficient. If supporting to 
consume from special timestamp will be a good idea for some cases.


> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever.
> In other words, if the app runs at first time and user want to consume the 
> past data, user must set the value of 
> '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*.
> {quote}{{auto.offset.reset:What to do when there is no initial offset in 
> Kafka or if the current offset does not exist any more on the server (e.g. 
> because that data has been deleted):}}
>  * earliest: automatically reset the offset to the earliest offset
>  * latest: automatically reset the offset to the latest offset
>  * none: throw exception to the consumer if no previous offset is found for 
> the consumer's group
>  * anything else: throw exception to the consumer.{quote}
> for example, suppose data in Kafka is retained for 7 days and you just want 
> to read from the data of yesterday, if you could not control the start 
> timestamp, you can only read from the earliest offset, it's very inefficient. 
> If supporting to consume from special timestamp in KafkaAdapter will be a 
> good idea for some cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Wang updated CALCITE-3073:
---
Description: 
Currently the KafkaAdapter consumes data from default 
offset(latest/earliest/last_offset) and runs forever.

In other words, if the app runs at first time and user want to consume the past 
data, user must set the value of '{color:#707070}*auto.offset.reset'*{color} 
parameter to *earliest*.
{quote}{{auto.offset.reset:What to do when there is no initial offset in Kafka 
or if the current offset does not exist any more on the server (e.g. because 
that data has been deleted):}}
 * earliest: automatically reset the offset to the earliest offset
 * latest: automatically reset the offset to the latest offset
 * none: throw exception to the consumer if no previous offset is found for the 
consumer's group
 * anything else: throw exception to the consumer.{quote}
for example, suppose data in Kafka is retained for 7 days and you just want to 
read from the data of yesterday, if you could not control the start timestamp, 
you can read from the earliest offset, it's very inefficient. If supporting to 
consume from special timestamp will be a good idea for some cases.

  was:Currently the KafkaAdapter consumes data from default 
offset(latest/earliest/last_offset) and runs forever. With support of from/to 
timestamp/offset, it's more flexible for users to query a specific range.


> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever.
> In other words, if the app runs at first time and user want to consume the 
> past data, user must set the value of 
> '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*.
> {quote}{{auto.offset.reset:What to do when there is no initial offset in 
> Kafka or if the current offset does not exist any more on the server (e.g. 
> because that data has been deleted):}}
>  * earliest: automatically reset the offset to the earliest offset
>  * latest: automatically reset the offset to the latest offset
>  * none: throw exception to the consumer if no previous offset is found for 
> the consumer's group
>  * anything else: throw exception to the consumer.{quote}
> for example, suppose data in Kafka is retained for 7 days and you just want 
> to read from the data of yesterday, if you could not control the start 
> timestamp, you can read from the earliest offset, it's very inefficient. If 
> supporting to consume from special timestamp will be a good idea for some 
> cases.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858178#comment-16858178
 ] 

Botong Huang edited comment on CALCITE-3118 at 6/6/19 11:29 PM:


Agree in general. But in this test case, the rule in this unit test 
(AssertOperandsDifferentRule) is to match a parent and its first and second 
child, it is wrong to return a match with a relNode and both its second child. 


was (Author: botong):
Agree in general. But in this test case, the rule (AssertOperandsDifferentRule) 
is to match a parent and its first and second child, it is wrong to return a 
match with a relNode and both its second child. 

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858178#comment-16858178
 ] 

Botong Huang commented on CALCITE-3118:
---

Agree in general. But in this test case, the rule (AssertOperandsDifferentRule) 
is to match a parent and its first and second child, it is wrong to return a 
match with a relNode and both its second child. 

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (CALCITE-3116) Upgrade to Avatica 1.15

2019-06-06 Thread Stamatis Zampetakis (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stamatis Zampetakis resolved CALCITE-3116.
--
Resolution: Fixed

Fixed in 
[614b4350f558a4fabaafc06fef9689ec4c267994|https://github.com/apache/calcite/commit/614b4350f558a4fabaafc06fef9689ec4c267994]!

> Upgrade to Avatica 1.15
> ---
>
> Key: CALCITE-3116
> URL: https://issues.apache.org/jira/browse/CALCITE-3116
> Project: Calcite
>  Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.20.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.

2019-06-06 Thread Stamatis Zampetakis (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858128#comment-16858128
 ] 

Stamatis Zampetakis commented on CALCITE-3115:
--

Reviewing and testing now!

> Cannot add JdbcRules which have different JdbcConvention to same 
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc 
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
> "a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
> "test2"."temp2" group by "a","b","c" 
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
>   JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
>   JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
>   JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
> 100))])
>   JdbcTableScan(table=[[test, temp]])
>   EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
>   JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to 
> JdbcAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-2829) Use consistent types when processing ranges

2019-06-06 Thread Juhwan Kim (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858092#comment-16858092
 ] 

Juhwan Kim commented on CALCITE-2829:
-

[~danny0405]. I agree with your comment. Implicit type conversion would fix 
this issue, and I also think that it is a much better idea. I uploaded my PR 
with a simple test case. Do you mind adding that to your PR? Then, I think we 
could resolve this ticket along with CALCITE-2302.

> Use consistent types when processing ranges
> ---
>
> Key: CALCITE-2829
> URL: https://issues.apache.org/jira/browse/CALCITE-2829
> Project: Calcite
>  Issue Type: Bug
>Reporter: Siddharth Teotia
>Assignee: Juhwan Kim
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Range expressions like  = 'literal' AND  < 'literal' trigger 
> ClassCastException as literal are implicitly casted differently between =/<> 
> operators and other comparison operators. Apply the same casting rules for 
> comparison to =/<> calls when processing ranges, so that all the terms have 
> the same type for literals.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Vladimir Sitnikov (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858067#comment-16858067
 ] 

Vladimir Sitnikov commented on CALCITE-3118:


[~botong], as far as I understand, there's nothing wrong with matching the same 
rel for both operands in the gneeric case.

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Hongze Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858056#comment-16858056
 ] 

Hongze Zhang commented on CALCITE-3073:
---

A naive question: Then what should be implemented for this issue? By reading 
the issue description:
{quote}With support of from/to timestamp/offset, it's more flexible for users 
to query a specific range.
{quote}
And the comment in PR:
{quote}Given a query like SELECT * FROM KAFKA_TABLE WHERE KAFKA_TIMESTAMP > 
CURRENT_TIMESTAMP-1, KafkaAdapter would take the filter clause KAFKA_TIMESTAMP 
> CURRENT_TIMESTAMP-1 directly.
{quote}
Didn't we already support SQL like {{SELECT STREAM * FROM KAFKA_TABLE WHERE 
MSG_TIMESTAMP > CURRENT_TIMESTAMP - 1}} in Kafka adaptor (Maybe tweaking on 
data types is needed)? Or the topic is more about performance improvement?

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858053#comment-16858053
 ] 

Botong Huang commented on CALCITE-3118:
---

Sure, supposedly this rule should only match once, with left child operand == 
leftPhy (PHYS, label = a) and right child operand == rightPhy (PHYS_2, label == 
b). 

However, without the fix, when rightPhy gets generated, it triggers two match 
attempts. The first one is expected as above. In the second one, rightPhy 
(PHYS_2, label == b) matches the left child operand, then we attempt to match 
the PhyBiRel as parent operand. 

Supposedly this parent should not match, because rightPhy is PhyBiRel's second 
child, but rightPhy's matched left child operand is parent operand's first 
child. Because of the bug, it didn't skip as expected and still proceed with 
the matching. After parent is matched, then it attempt to match the right child 
operand, and again matched the rightPhy (PHYS_2, label == b), as expected 
because it is PhyBiRel's second child. Overall as a result, both child operand 
end up matching the same RelNode rightPhy (PHYS_2, label == b). 

Hope this is clearer now. 

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-2826) Add versions of create() for LogicalSort and LogicalExchange that take cluster and trait set as input

2019-06-06 Thread Juhwan Kim (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858052#comment-16858052
 ] 

Juhwan Kim commented on CALCITE-2826:
-

Can someone review this pull request?

> Add versions of create() for LogicalSort and LogicalExchange that take 
> cluster and trait set as input
> -
>
> Key: CALCITE-2826
> URL: https://issues.apache.org/jira/browse/CALCITE-2826
> Project: Calcite
>  Issue Type: Task
>  Components: core
>Reporter: Siddharth Teotia
>Assignee: Juhwan Kim
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Add versions of create() for LogicalSort and LogicalExchange that take 
> cluster and trait set as input.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Vladimir Sitnikov (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858043#comment-16858043
 ] 

Vladimir Sitnikov commented on CALCITE-3118:


[~botong],
I reverted the fix, and the test you add fails as follows:

{noformat}java.lang.AssertionError: rel#4:PhysLeafRel.PHYS_2(label=b) should be 
different from rel#4:PhysLeafRel.PHYS_2(label=b)

at 
org.apache.calcite.plan.volcano.PlannerTests$AssertOperandsDifferentRule.onMatch(PlannerTests.java:278)
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
at 
org.apache.calcite.plan.volcano.VolcanoPlannerTest.testMatchedOperandsDifferent(VolcanoPlannerTest.java:154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at 
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at 
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
{noformat}

Can you please clarify why {{rel#4:PhysLeafRel.PHYS_2(label=b)}} should be 
different from {{rel#4:PhysLeafRel.PHYS_2(label=b)}} ?


> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858038#comment-16858038
 ] 

Botong Huang commented on CALCITE-3118:
---

New version pushed, can you take another look and see if it is better now? 

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Vladimir Sitnikov (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858027#comment-16858027
 ] 

Vladimir Sitnikov commented on CALCITE-3118:


{quote} is unit test required for this kind of fix{quote}
It is.

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858020#comment-16858020
 ] 

Botong Huang commented on CALCITE-3118:
---

Sure, let me try make the unit test more readable. On a side note, is unit test 
required for this kind of fix? 

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Vladimir Sitnikov (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858005#comment-16858005
 ] 

Vladimir Sitnikov commented on CALCITE-3118:


{quote} Without the one line fix, this assert will fail because{quote}
What I mean is it is not clear what and why the test is trying to test.

The thing is: you see an error. It might be obvious for you.

However committers would have to support the test code should it fail for a 
random reason. That means, for committers (even for those who are not super 
familiar with Volcano) the code/test should be more or less understandable.

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857997#comment-16857997
 ] 

Botong Huang edited comment on CALCITE-3118 at 6/6/19 7:08 PM:
---

Thanks [~vladimirsitnikov] for reviewing. The real check in the unit test is 
the assert inside TwoChildrenSameSetMatchRule. Without the one line fix, this 
assert will fail because it ends up matching the same PhysLeafRel for both 
child operands. I can add some comments in the unit test in needed.


was (Author: botong):
Thanks [~vladimirsitnikov] for reviewing. The real check in the unit test is 
the assert inside TwoChildrenSameSetMatchRule. Without the one line fix, this 
assert will fail because it ends up matching the same PhysLeafRel as both child 
operand. 

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857997#comment-16857997
 ] 

Botong Huang commented on CALCITE-3118:
---

Thanks [~vladimirsitnikov] for reviewing. The real check in the unit test is 
the assert inside TwoChildrenSameSetMatchRule. Without the one line fix, this 
assert will fail because it ends up matching the same PhysLeafRel as both child 
operand. 

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Vladimir Sitnikov (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Sitnikov updated CALCITE-3118:
---
Labels: pull-request-available  (was: )

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>  Labels: pull-request-available
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Vladimir Sitnikov (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857994#comment-16857994
 ] 

Vladimir Sitnikov commented on CALCITE-3118:


Frankly speaking, I have scanned through the added unit test, and its obscure 
what the test is doing and what it is asserting.

Even though I have fixed bugs near VolcanoRuleCall, and the change looks 
simple, I can't commit this PR because I don't understand it. Sorry.

> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
>  Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
> looking for parent operand match), we want to check that the matched parent 
> indeed has the previously matched child RelNode as a child with the expected 
> ordinal. However, there is a bug in this check. As a result, some incorrect 
> parent is not skipped as expected and matched incorrectly. See unit test 
> included in PR for a case that triggers this bug, where the same RelNode get 
> matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Closed] (CALCITE-3108) Babel parser should parse the PostgreSQL TEXT type

2019-06-06 Thread Muhammad Gelbana (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Muhammad Gelbana closed CALCITE-3108.
-
Resolution: Won't Fix

Users can define their own types but the problem seems with the returned column 
type name.

The metadata for the returned resultset for that query using calcite won't 
return the user-defiend type alias for the column type name, it will return 
whatever the user-define type was configured to act as (ex: {{VARCHAR}}).

*Example:*
 The following code prints {{TEXT}} as expected
{code:java}
Connection connection = // A connection to a real PostgreSQL DB
PreparedStatement statement = connection.prepareStatement("SELECT column1::TEXT 
FROM (VALUES (1, 2, 3)) a");
System.out.println(statement.executeQuery().getMetaData().getColumnTypeName(1));
{code}
But the following code prints {{VARCHAR}} while I It should print {{TEXT}} 
instead
{code:java}
Connection connection = DriverManager.getConnection("jdbc:calcite:", info);
connection.unwrap(CalciteConnection.class).getRootSchema().unwrap(CalciteSchema.class).add("TEXT",
 new RelProtoDataType() {
@Override
public RelDataType apply(RelDataTypeFactory factory) {
 return 
factory.createTypeWithNullability(factory.createSqlType(SqlTypeName.VARCHAR), 
false);
}
});

PreparedStatement statement = connection.prepareStatement("SELECT CAST(EXPR$0 
AS text) FROM (VALUES (1, 2, 3)) a");
System.out.println(statement.executeQuery().getMetaData().getColumnTypeName(1));
{code}

> Babel parser should parse the PostgreSQL TEXT type
> --
>
> Key: CALCITE-3108
> URL: https://issues.apache.org/jira/browse/CALCITE-3108
> Project: Calcite
>  Issue Type: Bug
>  Components: babel, core
>Affects Versions: 1.19.0
>Reporter: Muhammad Gelbana
>Assignee: Muhammad Gelbana
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.20.0
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Casting to PostgreSQL TEXT (ie. VARCHAR) isn't currently supported. The 
> following query would fail in parsing and execution.
> {code:sql}SELECT CAST(EXPR$0 AS text) FROM (VALUES (1, 2, 3)){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Xu Mingmin (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857871#comment-16857871
 ] 

Xu Mingmin commented on CALCITE-3073:
-

Either field name is good to me, I do want to avoid the configuration in files. 
\{{MSG_TIMESTAMP}} is the name in the default converter, users may have their 
own.

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857860#comment-16857860
 ] 

Matt Wang commented on CALCITE-3073:


[~zhztheplayer]  We can use `MSG_TIMESTAMP` field to define the end 
timestamp(I almost forgot it), it's not necessary to define `KAFKA_TIMESTAMP` 
reserved keyword. Just setting the start timestamp can satisfy the case of 
querying a specific range.

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Hongze Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857848#comment-16857848
 ] 

Hongze Zhang commented on CALCITE-3073:
---

I'am a little confused with the two columns, the new {{KAFKA_TIMESTAMP}} and 
the existing {{MSG_TIMESTAMP}} (sorry I'm not a Kafka expert :)). If we have to 
provide a different timestamp column, hopefully we can make them well 
documented so that users know which one they should use.

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked

2019-06-06 Thread Botong Huang (JIRA)
Botong Huang created CALCITE-3118:
-

 Summary: VolcanoRuleCall match parent child ordinal not properly 
checked
 Key: CALCITE-3118
 URL: https://issues.apache.org/jira/browse/CALCITE-3118
 Project: Calcite
  Issue Type: Bug
Reporter: Botong Huang


In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched, 
looking for parent operand match), we want to check that the matched parent 
indeed has the previously matched child RelNode as a child with the expected 
ordinal. However, there is a bug in this check. As a result, some incorrect 
parent is not skipped as expected and matched incorrectly. See unit test 
included in PR for a case that triggers this bug, where the same RelNode get 
matched for two operands at the same time. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857838#comment-16857838
 ] 

Matt Wang commented on CALCITE-3073:


[~zhztheplayer] I have updated the JIRA title, now the PR just support to 
consume form the special timestamp.

Now we can set the start timestamp in `modern.json` file, then we can read from 
the special timestamp. I do not know it's necessary to support the end 
timestamp, because the data in Kafka is a streaming data and it is producing 
all the time.

But like [~mingmxu] suggested, if we add a special reserved 
keyword(`KAFKA_TIMESTAMP`) in KafkaAdapter, we can query like `SELECT * FROM 
KAFKA_TABLE WHERE KAFKA_TIMESTAMP > CURRENT_TIMESTAMP-1`, it will make easy to 
use, but I'm not sure is a good idea.

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-2906) Record DOT extension does not distinguish between simple and compound identifiers

2019-06-06 Thread Hongze Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857835#comment-16857835
 ] 

Hongze Zhang commented on CALCITE-2906:
---

Marked fix version as 1.21.0.

> Record DOT extension does not distinguish between simple and compound 
> identifiers
> -
>
> Key: CALCITE-2906
> URL: https://issues.apache.org/jira/browse/CALCITE-2906
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Rong Rong
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> Currently the DOT extensions only supports Simple Identifiers:
> For example:
> * *{{tbl.foo(0).col.bar}}* is parsed as: *{{((`TBL`.`FOO`(0).`COL`).`BAR`)}}*
> However, this is not the case in regular Expression field, for example:
> * *{{tbl.col.bar}}* is parsed as: *{{`TBL`.`COL`.`BAR`}}*
> Parser should support both compound identifier, and simple identifier chain 
> after a record type SqlNode.
> Similarly, parsing should also be consistent for other types of expression, 
> for example
> * *{{tbl.arr[0].foo.bar}}* is currently parsed as: 
> *{{((`TBL`.`ARR`[0].`FOO`).`BAR`)}}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-2906) Record DOT extension does not distinguish between simple and compound identifiers

2019-06-06 Thread Hongze Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hongze Zhang updated CALCITE-2906:
--
Fix Version/s: 1.21.0

> Record DOT extension does not distinguish between simple and compound 
> identifiers
> -
>
> Key: CALCITE-2906
> URL: https://issues.apache.org/jira/browse/CALCITE-2906
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Rong Rong
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> Currently the DOT extensions only supports Simple Identifiers:
> For example:
> * *{{tbl.foo(0).col.bar}}* is parsed as: *{{((`TBL`.`FOO`(0).`COL`).`BAR`)}}*
> However, this is not the case in regular Expression field, for example:
> * *{{tbl.col.bar}}* is parsed as: *{{`TBL`.`COL`.`BAR`}}*
> Parser should support both compound identifier, and simple identifier chain 
> after a record type SqlNode.
> Similarly, parsing should also be consistent for other types of expression, 
> for example
> * *{{tbl.arr[0].foo.bar}}* is currently parsed as: 
> *{{((`TBL`.`ARR`[0].`FOO`).`BAR`)}}*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3073) Support from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Wang updated CALCITE-3073:
---
Summary: Support from timestamp in KafkaAdapter  (was: Support both from 
timestamp in KafkaAdapter)

> Support from timestamp in KafkaAdapter
> --
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3073) Support to consume from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Wang updated CALCITE-3073:
---
Summary: Support to consume from timestamp in KafkaAdapter  (was: Support 
from timestamp in KafkaAdapter)

> Support to consume from timestamp in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3073) Support both from timestamp in KafkaAdapter

2019-06-06 Thread Matt Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Wang updated CALCITE-3073:
---
Summary: Support both from timestamp in KafkaAdapter  (was: Support both 
from/to timestamp in KafkaAdapter)

> Support both from timestamp in KafkaAdapter
> ---
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (CALCITE-3114) Support websocket transport

2019-06-06 Thread Leo Tu (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857785#comment-16857785
 ] 

Leo Tu edited comment on CALCITE-3114 at 6/6/19 3:19 PM:
-

Currently, the client interacts with the server using the Http Request-Response 
mode.

If you can provide Client and Server interaction using websocket's frame to 
implement Request-Response
 that will be similar to the traditional JDBC Driver TCP/IP.

 

(1) Websocket client <-> (2)  

Websocket server <-> (3) Database Server

 

Transportation between:

(1) & (2) Websocket  Protocol (Connection/Statement/ResultSet Serializable)

(2) & (3) TCP/IP vender's  JDBC driver (Connection/Statement/ResultSet)

 

 

 

 


was (Author: leotu):
Currently, the client interacts with the server using the Http Request-Response 
mode.

If you can provide Client and Server interaction using websocket's frame to 
implement Request-Response
 that will be similar to the traditional JDBC Driver TCP/IP.

 

(1) Websocket client <-> (2)  

Websocket server <-> (3) Database Server

 

Transportation between:

(1) & (2) Websocket  Protocol (Connection/Statement/ResultSet Serializable)

(2) & (3) TPC/IP (Connection/Statement/ResultSet)

 

 

 

 

> Support websocket transport
> ---
>
> Key: CALCITE-3114
> URL: https://issues.apache.org/jira/browse/CALCITE-3114
> Project: Calcite
>  Issue Type: Wish
>Reporter: Leo Tu
>Priority: Minor
>
> it would be greatly improve performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (CALCITE-3114) Support websocket transport

2019-06-06 Thread Leo Tu (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857785#comment-16857785
 ] 

Leo Tu edited comment on CALCITE-3114 at 6/6/19 3:17 PM:
-

Currently, the client interacts with the server using the Http Request-Response 
mode.

If you can provide Client and Server interaction using websocket's frame to 
implement Request-Response
 that will be similar to the traditional JDBC Driver TCP/IP.

 

(1) Websocket client <-> (2)  

Websocket server <-> (3) Database Server

 

Transportation between:

(1) & (2) Websocket  Protocol (Connection/Statement/ResultSet Serializable)

(2) & (3) TPC/IP (Connection/Statement/ResultSet)

 

 

 

 


was (Author: leotu):
Currently, the client interacts with the server using the Http Request-Response 
mode.

If you can provide Client and Server interaction using websocket's frame to 
implement Request-Response
that will be similar to the traditional JDBC Driver TCP/IP.

 

                       
(1) 

 

Websocket client <> (2)  

Websocket server <> (3) Database Server

 

(1) & (2) is Websocket Protocal (Connection/Statement/ResultSet)

(2) & (3) TPC/IP (Connection/Statement/ResultSet)

 

 

 

 

> Support websocket transport
> ---
>
> Key: CALCITE-3114
> URL: https://issues.apache.org/jira/browse/CALCITE-3114
> Project: Calcite
>  Issue Type: Wish
>Reporter: Leo Tu
>Priority: Minor
>
> it would be greatly improve performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3117) Distinguish Calcite's public APIs

2019-06-06 Thread Ivan Grgurina (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857750#comment-16857750
 ] 

Ivan Grgurina commented on CALCITE-3117:


RelOptRule and Program should definitely be considered public APIs.

> Distinguish Calcite's public APIs
> -
>
> Key: CALCITE-3117
> URL: https://issues.apache.org/jira/browse/CALCITE-3117
> Project: Calcite
>  Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Minor
>
> The goal of this issue is to identify and mark Calcite's public APIs. This 
> can have bring various benefits to the project:
>  * provide more flexibility to developers;
>  * speed-up reviews;
>  * enhance backward compatibility;
>  * facilitate the life of end-users who will be exposed to fewer classes and 
> interfaces.
> Let's list here packages, interfaces, and classes that should be part of the 
> public API. As a first step let's focus exclusively on completing the list 
> and not argue about others proposals. It will be easier to solve 
> disagreements afterwards as soon as we have a rather complete list.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3116) Upgrade to Avatica 1.15

2019-06-06 Thread Michael Mior (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Mior updated CALCITE-3116:
--
Fix Version/s: 1.20.0

> Upgrade to Avatica 1.15
> ---
>
> Key: CALCITE-3116
> URL: https://issues.apache.org/jira/browse/CALCITE-3116
> Project: Calcite
>  Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.20.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3116) Upgrade to Avatica 1.15

2019-06-06 Thread Michael Mior (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857745#comment-16857745
 ] 

Michael Mior commented on CALCITE-3116:
---

Sounds reasonable to me. It would be great if this could be tested and merged 
today.

> Upgrade to Avatica 1.15
> ---
>
> Key: CALCITE-3116
> URL: https://issues.apache.org/jira/browse/CALCITE-3116
> Project: Calcite
>  Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-3117) Distinguish Calcite's public APIs

2019-06-06 Thread Stamatis Zampetakis (JIRA)
Stamatis Zampetakis created CALCITE-3117:


 Summary: Distinguish Calcite's public APIs
 Key: CALCITE-3117
 URL: https://issues.apache.org/jira/browse/CALCITE-3117
 Project: Calcite
  Issue Type: Task
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis


The goal of this issue is to identify and mark Calcite's public APIs. This can 
have bring various benefits to the project:
 * provide more flexibility to developers;
 * speed-up reviews;
 * enhance backward compatibility;
 * facilitate the life of end-users who will be exposed to fewer classes and 
interfaces.

Let's list here packages, interfaces, and classes that should be part of the 
public API. As a first step let's focus exclusively on completing the list and 
not argue about others proposals. It will be easier to solve disagreements 
afterwards as soon as we have a rather complete list.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3116) Upgrade to Avatica 1.15

2019-06-06 Thread Stamatis Zampetakis (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857728#comment-16857728
 ] 

Stamatis Zampetakis commented on CALCITE-3116:
--

Hey [~michaelmior], this is kind of trivial can we get this into 1.20?

> Upgrade to Avatica 1.15
> ---
>
> Key: CALCITE-3116
> URL: https://issues.apache.org/jira/browse/CALCITE-3116
> Project: Calcite
>  Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3073) Support both from/to timestamp in KafkaAdapter

2019-06-06 Thread Xu Mingmin (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xu Mingmin updated CALCITE-3073:

Summary: Support both from/to timestamp in KafkaAdapter  (was: Support both 
from/to timestamp/offset in KafkaAdapter)

> Support both from/to timestamp in KafkaAdapter
> --
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3116) Upgrade to Avatica 1.15

2019-06-06 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3116:

Labels: pull-request-available  (was: )

> Upgrade to Avatica 1.15
> ---
>
> Key: CALCITE-3116
> URL: https://issues.apache.org/jira/browse/CALCITE-3116
> Project: Calcite
>  Issue Type: Task
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Minor
>  Labels: pull-request-available
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3088) Multi-groupsets query failed due to data type preservation check

2019-06-06 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3088:

Labels: pull-request-available  (was: )

> Multi-groupsets query failed due to data type preservation check
> 
>
> Key: CALCITE-3088
> URL: https://issues.apache.org/jira/browse/CALCITE-3088
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.20.0
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
>  Labels: pull-request-available
>
> We encounter the proble in our environment. It can be reproduced by the 
> modification on _testCube_ in *SqlToRelConverterTest*.
> {code:java}
> @Test public void testCube() {
>   // cube(a, b) is {(a, b), (a), (b), ()}
>   final String sql = "select a+1\n"
>   + "from (values (1, 2, 3, 4)) as t(a, b, c, d)\n"
>   + "group by cube(a, b)";
>   sql(sql).ok();
> }
> {code}
> This query will fail with following exception:
> {code:java}
> java.lang.AssertionError: Conversion to relational algebra failed to preserve 
> datatypes:
> validated type:
> RecordType(INTEGER NOT NULL EXPR$0) NOT NULL
> converted type:
> RecordType(INTEGER EXPR$0) NOT NULL
> rel:
> LogicalProject(EXPR$0=[+($0, 1)])
>   LogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {1}, {}]])
>     LogicalProject(A=[$0], B=[$1])
>   LogicalValues(tuples=[[{ 1, 2, 3, 4 }]])
>     at 
> org.apache.calcite.sql2rel.SqlToRelConverter.checkConvertedType(SqlToRelConverter.java:406)
>     at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:523)
>     at 
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:610)
>     at 
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.assertConvertsTo(SqlToRelTestBase.java:723)
>     at 
> org.apache.calcite.test.SqlToRelConverterTest$Sql.convertsTo(SqlToRelConverterTest.java:3073)
>     at 
> org.apache.calcite.test.SqlToRelConverterTest$Sql.ok(SqlToRelConverterTest.java:3065)
>     at 
> org.apache.calcite.test.SqlToRelConverterTest.testCube(SqlToRelConverterTest.java:474)
> {code}
> The exception stacktrace is very clear. In validation phase, SqlValidator 
> does not take such case into consideration, and consequently the type of 
> {color:#FF}EXPR$0(a+1){color} cannot be inferred as nullable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-3116) Upgrade to Avatica 1.15

2019-06-06 Thread Stamatis Zampetakis (JIRA)
Stamatis Zampetakis created CALCITE-3116:


 Summary: Upgrade to Avatica 1.15
 Key: CALCITE-3116
 URL: https://issues.apache.org/jira/browse/CALCITE-3116
 Project: Calcite
  Issue Type: Task
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3073) Support both from/to timestamp/offset in KafkaAdapter

2019-06-06 Thread Hongze Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857627#comment-16857627
 ] 

Hongze Zhang commented on CALCITE-3073:
---

> Agree, let's focus on timestamp support here.
Should we update JIRA title accordingly?

> Support both from/to timestamp/offset in KafkaAdapter
> -
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
>  Issue Type: Improvement
>  Components: kafka-adapter
>Reporter: Xu Mingmin
>Assignee: Matt Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default 
> offset(latest/earliest/last_offset) and runs forever. With support of from/to 
> timestamp/offset, it's more flexible for users to query a specific range.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.

2019-06-06 Thread TANG Wen-hui (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857566#comment-16857566
 ] 

TANG Wen-hui edited comment on CALCITE-3115 at 6/6/19 11:48 AM:


Once JdbcAggregateRule(JDBC.test, RelFactories.LOGICAL_BUILDER) was added to 
ruleset, JdbcAggregateRule(JDBC.test2, RelFactories.LOGICAL_BUILDER) cannot be 
added to same planner's ruleset, since they have same description. And the 
hashCode() of RelOptRule are:

 
{code:java}
public int hashCode()
{ // Conventionally, hashCode() and equals() should use the same 
// criteria, whereas here we only look at the description. This is 
// okay, because the planner requires all rule instances to have 
// distinct descriptions. 
return description.hashCode(); 
}
{code}
 

(JDBC.test and JDBC.test2 are JdbcConvention.) 

So I have changed the description of JdbcRules. 


was (Author: winipanda):
Once JdbcAggregateRule(JDBC.test, RelFactories.LOGICAL_BUILDER) was added to 
ruleset, JdbcAggregateRule(JDBC.test2, RelFactories.LOGICAL_BUILDER) cannot be 
added to same planner's ruleset, since they have same description. And the 
hashCode() of RelOptRule are:

public int hashCode() {
 // Conventionally, hashCode() and equals() should use the same
 // criteria, whereas here we only look at the description. This is
 // okay, because the planner requires all rule instances to have
 // distinct descriptions.
 return description.hashCode();
}

(JDBC.test and JDBC.test2 are JdbcConvention.) 

So I have changed the description of JdbcRules. 

> Cannot add JdbcRules which have different JdbcConvention to same 
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc 
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
> "a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
> "test2"."temp2" group by "a","b","c" 
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
>   JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
>   JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
>   JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
> 100))])
>   JdbcTableScan(table=[[test, temp]])
>   EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
>   JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to 
> JdbcAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.

2019-06-06 Thread ASF GitHub Bot (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-3115:

Labels: pull-request-available  (was: )

> Cannot add JdbcRules which have different JdbcConvention to same 
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
>  Labels: pull-request-available
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc 
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
> "a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
> "test2"."temp2" group by "a","b","c" 
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
>   JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
>   JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
>   JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
> 100))])
>   JdbcTableScan(table=[[test, temp]])
>   EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
>   JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to 
> JdbcAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.

2019-06-06 Thread TANG Wen-hui (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857566#comment-16857566
 ] 

TANG Wen-hui commented on CALCITE-3115:
---

Once JdbcAggregateRule(JDBC.test, RelFactories.LOGICAL_BUILDER) was added to 
ruleset, JdbcAggregateRule(JDBC.test2, RelFactories.LOGICAL_BUILDER) cannot be 
added to same planner's ruleset, since they have same description. And the 
hashCode() of RelOptRule are:

public int hashCode() {
 // Conventionally, hashCode() and equals() should use the same
 // criteria, whereas here we only look at the description. This is
 // okay, because the planner requires all rule instances to have
 // distinct descriptions.
 return description.hashCode();
}

(JDBC.test and JDBC.test2 are JdbcConvention.) 

So I have changed the description of JdbcRules. 

> Cannot add JdbcRules which have different JdbcConvention to same 
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc 
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
> "a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
> "test2"."temp2" group by "a","b","c" 
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
>   JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
>   JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
>   JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
> 100))])
>   JdbcTableScan(table=[[test, temp]])
>   EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
>   JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to 
> JdbcAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.

2019-06-06 Thread TANG Wen-hui (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TANG Wen-hui updated CALCITE-3115:
--
Description: 
When we use Calcite via JDBC to run a sql which involves two difference jdbc 
schema:
{code:java}
select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
"test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
"a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
"test2"."temp2" group by "a","b","c" 
{code}
the sql get a plan like that:
{code:java}
EnumerableUnion(all=[false])
  JdbcToEnumerableConverter
JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
  JdbcSort(fetch=[10])
JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
  JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
100))])
  JdbcTableScan(table=[[test, temp]])
  EnumerableAggregate(group=[{0, 1, 2}])
JdbcToEnumerableConverter
  JdbcTableScan(table=[[test2, temp2]])
{code}
And the EnumerableAggregate for table test2.temp2 cannot be converted to 
JdbcAggregate.

  was:
When we use Calcite via JDBC to run a sql which involves two difference jdbc 
schema:

{code:java}
select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
"test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
"a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
"test2"."temp2" group by "a","b","c" 
{code}
the sql get a plan like that:

{code:java}
EnumerableUnion(all=[false])
  JdbcToEnumerableConverter
JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
  JdbcSort(fetch=[10])
JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
  JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
100))])
  JdbcTableScan(table=[[test, temp]])
  EnumerableAggregate(group=[{0, 1, 2}])
JdbcToEnumerableConverter
  JdbcTableScan(table=[[test2, temp2]])
{code}
And the EnumerableAggregate for table test2.temp2 cannot be convert to 
JdbcAggregate.


> Cannot add JdbcRules which have different JdbcConvention to same 
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: TANG Wen-hui
>Priority: Major
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc 
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
> "a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
> "test2"."temp2" group by "a","b","c" 
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
>   JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
>   JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
>   JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
> 100))])
>   JdbcTableScan(table=[[test, temp]])
>   EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
>   JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to 
> JdbcAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (CALCITE-2460) Implement TO_BASE64 and FROM_BASE64 functions

2019-06-06 Thread TANG Wen-hui (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TANG Wen-hui reassigned CALCITE-2460:
-

Assignee: TANG Wen-hui

> Implement TO_BASE64 and FROM_BASE64 functions
> -
>
> Key: CALCITE-2460
> URL: https://issues.apache.org/jira/browse/CALCITE-2460
> Project: Calcite
>  Issue Type: Improvement
>Reporter: vinoyang
>Assignee: TANG Wen-hui
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> refer to mysql TO_BASE64 function : 
> [https://dev.mysql.com/doc/refman/5.6/en/string-functions.html#function_to-base64]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (CALCITE-2460) Implement TO_BASE64 and FROM_BASE64 functions

2019-06-06 Thread TANG Wen-hui (JIRA)


 [ 
https://issues.apache.org/jira/browse/CALCITE-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

TANG Wen-hui updated CALCITE-2460:
--
Summary: Implement TO_BASE64 and FROM_BASE64 functions  (was: Add 
implementation of To_Base64 to SqlFunctions)

> Implement TO_BASE64 and FROM_BASE64 functions
> -
>
> Key: CALCITE-2460
> URL: https://issues.apache.org/jira/browse/CALCITE-2460
> Project: Calcite
>  Issue Type: Improvement
>Reporter: vinoyang
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> refer to mysql TO_BASE64 function : 
> [https://dev.mysql.com/doc/refman/5.6/en/string-functions.html#function_to-base64]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.

2019-06-06 Thread TANG Wen-hui (JIRA)
TANG Wen-hui created CALCITE-3115:
-

 Summary: Cannot add JdbcRules which have different JdbcConvention 
to same VolcanoPlanner's RuleSet.
 Key: CALCITE-3115
 URL: https://issues.apache.org/jira/browse/CALCITE-3115
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.19.0
Reporter: TANG Wen-hui
Assignee: TANG Wen-hui


When we use Calcite via JDBC to run a sql which involves two difference jdbc 
schema:

{code:java}
select * from (select "a",max("b") as max_b, sum("c") as sum_c from 
"test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having 
"a" > 100 and max("b") < 20 limit 10) t  union select "a", "b","c" from 
"test2"."temp2" group by "a","b","c" 
{code}
the sql get a plan like that:

{code:java}
EnumerableUnion(all=[false])
  JdbcToEnumerableConverter
JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
  JdbcSort(fetch=[10])
JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
  JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, 
100))])
  JdbcTableScan(table=[[test, temp]])
  EnumerableAggregate(group=[{0, 1, 2}])
JdbcToEnumerableConverter
  JdbcTableScan(table=[[test2, temp2]])
{code}
And the EnumerableAggregate for table test2.temp2 cannot be convert to 
JdbcAggregate.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3114) Support websocket transport

2019-06-06 Thread Danny Chan (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857429#comment-16857429
 ] 

Danny Chan commented on CALCITE-3114:
-

What do you mean by websocket transport ?

> Support websocket transport
> ---
>
> Key: CALCITE-3114
> URL: https://issues.apache.org/jira/browse/CALCITE-3114
> Project: Calcite
>  Issue Type: Wish
>Reporter: Leo Tu
>Priority: Minor
>
> it would be greatly improve performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-3114) Support websocket transport

2019-06-06 Thread Leo Tu (JIRA)
Leo Tu created CALCITE-3114:
---

 Summary: Support websocket transport
 Key: CALCITE-3114
 URL: https://issues.apache.org/jira/browse/CALCITE-3114
 Project: Calcite
  Issue Type: Wish
Reporter: Leo Tu


it would be greatly improve performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects

2019-06-06 Thread Danny Chan (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857366#comment-16857366
 ] 

Danny Chan commented on CALCITE-2280:
-

This is actually what we have to do in Flink custom parser, for computation 
engines like Flink, one big topic is the Hive compatibility, for example, if we 
want to add "partition" grammar like
{code:java}
insert into T partition(col1=vl) select from S;
{code}
What we have to do is to copy the whole java cc parse block for insert 
statement and add our custom parse logic.

We also have to add our SqlConformance Hive, but the SqlConformanceEnum is 
enum, so i also have to hard code the conformance in the parser template.

 

> Liberal "babel" parser that accepts all SQL dialects
> 
>
> Key: CALCITE-2280
> URL: https://issues.apache.org/jira/browse/CALCITE-2280
> Project: Calcite
>  Issue Type: Bug
>  Components: babel
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.17.0
>
>
> Create a parser that accepts all SQL dialects.
> It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. 
> If you have preferred dialects, please let us know in the comments section. 
> (If you're willing to work on a particular dialect, even better!)
> We would do this in a new module, inheriting and extending the parser in the 
> same way that the DDL parser in the "server" module does.
> This would be a messy and difficult project, because we would have to comply 
> with the rules of each parser (and its set of built-in functions) rather than 
> writing the rules as we would like them to be. That's why I would keep it out 
> of the core parser. But it would also have large benefits.
> This would be new territory Calcite: as a tool for manipulating/understanding 
> SQL, not (necessarily) for relational algebra or execution.
> Some possible uses:
> * analyze query lineage (what tables and columns are used in a query);
> * translate from one SQL dialect to another (using the JDBC adapter to 
> generate SQL in the target dialect);
> * a "deep" compatibility mode (much more comprehensive than the current 
> compatibility mode) where Calcite could pretend to be, say, Oracle;
> * SQL parser as a service: a REST call gives a SQL query, and returns a JSON 
> or XML document with the parse tree.
> If you can think of interesting uses, please discuss in the comments.
> There are similarities with Uber's 
> [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can 
> collaborate, or make use of their test cases.
> We will need a lot of sample queries. If you are able to contribute sample 
> queries for particular dialects, please discuss in the comments section. It 
> would be good if the sample queries are based on a familiar schema (e.g. 
> scott or foodmart) but we can be flexible about this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (CALCITE-3113) Equivalent MutableAggregates with different row types fail with AssertionError

2019-06-06 Thread Feng Zhu (JIRA)
Feng Zhu created CALCITE-3113:
-

 Summary: Equivalent MutableAggregates with different row types 
fail with AssertionError
 Key: CALCITE-3113
 URL: https://issues.apache.org/jira/browse/CALCITE-3113
 Project: Calcite
  Issue Type: Bug
  Components: core
Affects Versions: 1.19.0
Reporter: Feng Zhu


Add test case in MaterializationTest: 
{code:java}
@Test public void testAggregateAlias() {
  checkMaterialize(
  "select count(*) as c from \"emps\" group by \"empid\"",
  "select count(*) + 1 as c from \"emps\" group by \"empid\"");
}
{code}
 It fails due to different rowtype.
{code:java}
java.lang.AssertionError
    at 
org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:504)
    at 
org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:465)
    at 
org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56)
    at 
org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200)
    at 
org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72)
    at 
org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:347)
{code}
However, according to MutableAggregate's hashCode implementation, this 
materialization can be reused, i.e., queryDescedant=targetDescendant.
{code:java}
queryDescendant: RecordType(JavaType(int) empid, BIGINT $f1)
=
Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()])
  Project(projects: [$0])
    Scan(table: [hr, emps])

targetDescendant: RecordType(JavaType(int) empid, BIGINT C)
=
Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()])
  Project(projects: [$0])
    Scan(table: [hr, emps])
{code}
So, how can we align them?

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (CALCITE-3087) AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its Mapping breaks ordering

2019-06-06 Thread Feng Zhu (JIRA)


[ 
https://issues.apache.org/jira/browse/CALCITE-3087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857333#comment-16857333
 ] 

Feng Zhu commented on CALCITE-3087:
---

Therefore, when a posList generates IdentifyMapping or (partially 
IdentifyMapping + \{i->j, j->i}), i.e., getSource(i)==getTarget(i), 
*MutableRels.createProject* can be OK.
{code:java}
posList={0,1,2} mapping={0->0, 1->1, 2->2}  OK
posList={1,0} mapping={0->1, 1->0}  OK
posList={1,0,2} mapping={0->1, 1->0, 2->2} OK
posList={1,2,0} mapping={0->1, 1->2, 2->0} NO
{code}
Moreover, I have checked all code and found that:

*MutableRels.createProject* is only called in unifyAggregates

under condition 
(_query.{color:#660e7a}groupSets{color}.equals(target.{color:#660e7a}groupSets{color})_)

It seems that the mapping _{color:#FF}isIdentify{color}_ from this 
entrance, and consequently I can't figure out a test case for the logic 
inconsistency.

> AggregateOnProjectToAggregateUnifyRule ignores Project incorrectly when its 
> Mapping breaks ordering
> ---
>
> Key: CALCITE-3087
> URL: https://issues.apache.org/jira/browse/CALCITE-3087
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Haisheng Yuan
>Assignee: Feng Zhu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Add the following test to MaterializationTest:
> {code:java}
> @Test public void testAggregateGroupSetsRollUp0() {
> checkMaterialize(
> "select \"empid\", \"deptno\", count(*) as c, sum(\"salary\") as s 
> from \"emps\" "
> + "group by \"empid\", \"deptno\"",
> "select count(*) + 1 as c,  \"deptno\" from \"emps\" group by 
> cube(\"deptno\", \"empid\")",
> HR_FKUK_MODEL,
> CalciteAssert.checkResultContains(
> "EnumerableCalc(expr#0..2=[{inputs}], expr#3=[1], "
> + "expr#4=[+($t2, $t3)], C=[$t4], deptno=[$t1])\n"
> + "  EnumerableAggregate(group=[{0, 1}], groups=[[{0, 1}, 
> {0}, {1}, {}]], agg#0=[$SUM0($2)])\n"
> + "EnumerableTableScan(table=[[hr, m0]])"));
>   }
> {code}
> We got error:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: must be sorted: [{0, 1}, {1}, 
> {0}, {}]
>   at 
> org.apache.calcite.rel.core.Aggregate$Group.induce(Aggregate.java:472)
>   at 
> org.apache.calcite.rel.mutable.MutableAggregate.getGroupType(MutableAggregate.java:83)
>   at 
> org.apache.calcite.plan.SubstitutionVisitor.unifyAggregates(SubstitutionVisitor.java:1281)
>   at 
> org.apache.calcite.plan.SubstitutionVisitor$AggregateOnProjectToAggregateUnifyRule.apply(SubstitutionVisitor.java:1345)
>   at 
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:531)
>   at 
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:466)
>   at 
> org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56)
>   at 
> org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200)
>   at 
> org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72)
>   at 
> org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:348)
>   at 
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:584)
> {code}
> The reason is that Mappings.apply2 doesn't sort.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)