[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-08 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118187#comment-16118187
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4373


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-08 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118035#comment-16118035
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on the issue:

https://github.com/apache/flink/pull/4373
  
Thanks for the update @haohui. I think this PR is in a good shape now. I 
will go over the changes again and merge this. Unfortunately, the new findbugs 
plugin causes the build to fail. We have to wait until this is fixed.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-07 Thread Haohui Mai (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117500#comment-16117500
 ] 

Haohui Mai commented on FLINK-6429:
---

bq. In order to change the default behavior of the rule, the only thing we need 
to do is to create a new `ProjectJoinTransposeRule` with a custom 
`PushProjector.ExprCondition` which filters prevent time indicator rex nodes.

That's a good suggestion. I created the rules based on Calcite 1.12 in the 
current PR (hopefully unblock other changes). I'll incorporate the suggestion 
in a later PR.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-07 Thread Haohui Mai (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117328#comment-16117328
 ] 

Haohui Mai commented on FLINK-6429:
---

It's difficult to say this is a pure Calcite bug. Technically SQL allows 
special characters like '-' in the column name while the Table API rejects them.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-04 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115222#comment-16115222
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user wuchong commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r131512787
  
--- Diff: 
flink-libraries/flink-table/src/main/java/org/apache/calcite/rel/rules/PushProjector.java
 ---
@@ -0,0 +1,864 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
--- End diff --

Yes,  you are right. We should make sure it is a Calcite bug and will be 
fixed in the next release. 


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-04 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114154#comment-16114154
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r131346038
  
--- Diff: 
flink-libraries/flink-table/src/main/java/org/apache/calcite/rel/rules/PushProjector.java
 ---
@@ -0,0 +1,864 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
--- End diff --

Can you add a comment here when we can remove this file again 
(corresponding CALCITE- issue?).


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-03 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113850#comment-16113850
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user wuchong commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r131302891
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/stream/sql/JoinTest.scala
 ---
@@ -55,12 +55,13 @@ class JoinTest extends TableTestBase {
   unaryNode(
 "DataStreamCalc",
 streamTableNode(1),
-term("select", "a", "b", "proctime")
+term("select", "a", "b", "-(proctime, 360) AS -",
--- End diff --

I think it's easy to prevent the time indicator push-down in our side. 
Calcite 1.3 changed the `ProjectJoinTransposeRule.INSTANCE` to `new 
ProjectJoinTransposeRule( PushProjector.ExprCondition.TRUE, 
RelFactories.LOGICAL_BUILDER);` instead of `new 
ProjectJoinTransposeRule(PushProjector.ExprCondition.FALSE, 
RelFactories.LOGICAL_BUILDER);`.  

In order to change the default behavior of the rule, the only thing we need 
to do is to create a new `ProjectJoinTransposeRule` with a custom 
`PushProjector.ExprCondition` which filters prevent time indicator rex nodes.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-03 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112482#comment-16112482
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r131094518
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/stream/sql/JoinTest.scala
 ---
@@ -55,12 +55,13 @@ class JoinTest extends TableTestBase {
   unaryNode(
 "DataStreamCalc",
 streamTableNode(1),
-term("select", "a", "b", "proctime")
+term("select", "a", "b", "-(proctime, 360) AS -",
--- End diff --

Yes, you're of course right @twalthr. This is a valid rewrite and I agree 
with you're analysis. We have to prevent the push-down of predicates that 
include time indicator fields. The `DataStreamWindowJoin` needs the offsets to 
compute the correct window boundaries.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-03 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112422#comment-16112422
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r131085265
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/stream/sql/JoinTest.scala
 ---
@@ -55,12 +55,13 @@ class JoinTest extends TableTestBase {
   unaryNode(
 "DataStreamCalc",
 streamTableNode(1),
-term("select", "a", "b", "proctime")
+term("select", "a", "b", "-(proctime, 360) AS -",
--- End diff --

@fhueske Actually, from an optimizer perspective this behavior is correct. 
But we don't want to push down predicates with time indicators. The time 
indicators are not materialized in this case as the materialization happens 
before optimization.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-03 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112399#comment-16112399
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r131079109
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/expressions/time.scala
 ---
@@ -355,26 +355,36 @@ case class TemporalOverlaps(
   rightT: RexNode,
   relBuilder: FlinkRelBuilder)
 : RexNode = {
-// leftT = leftP + leftT if leftT is an interval
-val convLeftT = if (isTimeInterval(leftTemporal.resultType)) {
-relBuilder.call(SqlStdOperatorTable.DATETIME_PLUS, leftP, leftT)
-  } else {
-leftT
-  }
-// rightT = rightP + rightT if rightT is an interval
-val convRightT = if (isTimeInterval(rightTemporal.resultType)) {
-relBuilder.call(SqlStdOperatorTable.DATETIME_PLUS, rightP, rightT)
-  } else {
-rightT
-  }
-// leftT >= rightP
-val leftPred = 
relBuilder.call(SqlStdOperatorTable.GREATER_THAN_OR_EQUAL, convLeftT, rightP)
-// rightT >= leftP
-val rightPred = 
relBuilder.call(SqlStdOperatorTable.GREATER_THAN_OR_EQUAL, convRightT, leftP)
+val convLeftT = convertOverlapsEnd(relBuilder, leftP, leftT, 
leftTemporal.resultType)
--- End diff --

Good point. I'm fine with fixing this with this PR.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110621#comment-16110621
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user dianfu commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r130829080
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/stream/sql/JoinTest.scala
 ---
@@ -55,12 +55,13 @@ class JoinTest extends TableTestBase {
   unaryNode(
 "DataStreamCalc",
 streamTableNode(1),
-term("select", "a", "b", "proctime")
+term("select", "a", "b", "-(proctime, 360) AS -",
--- End diff --

I think this issue may be caused by CALCITE-1753, still investigating the 
changes in that JIRA. 


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110613#comment-16110613
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user fhueske commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r130827249
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/stream/sql/JoinTest.scala
 ---
@@ -55,12 +55,13 @@ class JoinTest extends TableTestBase {
   unaryNode(
 "DataStreamCalc",
 streamTableNode(1),
-term("select", "a", "b", "proctime")
+term("select", "a", "b", "-(proctime, 360) AS -",
--- End diff --

Yes, I'm also curious why this plan would be generated. AFAIK, the 
`DataStreamWindowJoin` is very strict about the join predicates it accepts. It 
needs two range predicates with time indicator attributes on both sides. If the 
computation is pushed into the `DataStreamCalc` the time indicator attribute 
should be materialized and no longer be identified as a time indicator. 


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110442#comment-16110442
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user dianfu commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r130801549
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/stream/sql/JoinTest.scala
 ---
@@ -55,12 +55,13 @@ class JoinTest extends TableTestBase {
   unaryNode(
 "DataStreamCalc",
 streamTableNode(1),
-term("select", "a", "b", "proctime")
+term("select", "a", "b", "-(proctime, 360) AS -",
--- End diff --

In the original test, the calculation of **proctime** is located when doing 
**Join**. While currently, it's pushed down to the **DataStremaCalc**. It seems 
that this is not correct?


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110425#comment-16110425
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user haohui commented on the issue:

https://github.com/apache/flink/pull/4373
  
@dianfu thanks for the investigation! It turns out that it no longer 
requires copying {{PushProjector}} after rebasing to the latest tip. Updated 
the PR to address the issue.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110387#comment-16110387
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user haohui commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r130794757
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/expressions/time.scala
 ---
@@ -355,26 +355,36 @@ case class TemporalOverlaps(
   rightT: RexNode,
   relBuilder: FlinkRelBuilder)
 : RexNode = {
-// leftT = leftP + leftT if leftT is an interval
-val convLeftT = if (isTimeInterval(leftTemporal.resultType)) {
-relBuilder.call(SqlStdOperatorTable.DATETIME_PLUS, leftP, leftT)
-  } else {
-leftT
-  }
-// rightT = rightP + rightT if rightT is an interval
-val convRightT = if (isTimeInterval(rightTemporal.resultType)) {
-relBuilder.call(SqlStdOperatorTable.DATETIME_PLUS, rightP, rightT)
-  } else {
-rightT
-  }
-// leftT >= rightP
-val leftPred = 
relBuilder.call(SqlStdOperatorTable.GREATER_THAN_OR_EQUAL, convLeftT, rightP)
-// rightT >= leftP
-val rightPred = 
relBuilder.call(SqlStdOperatorTable.GREATER_THAN_OR_EQUAL, convRightT, leftP)
+val convLeftT = convertOverlapsEnd(relBuilder, leftP, leftT, 
leftTemporal.resultType)
--- End diff --

The problem is that SQL semantic shifted from Calcite 1.12 to Calcite 1.13. 
Do you think it is a good idea to disable the relevant tests temporarily and 
then bring it back momentarily?


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110343#comment-16110343
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user haohui commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r130789743
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/batch/sql/CorrelateTest.scala
 ---
@@ -43,7 +43,7 @@ class CorrelateTest extends TableTestBase {
 term("invocation", "func1($cor0.c)"),
 term("function", func1.getClass.getCanonicalName),
 term("rowType",
- "RecordType(INTEGER a, BIGINT b, VARCHAR(2147483647) c, 
VARCHAR(2147483647) f0)"),
+ "RecordType(INTEGER a, BIGINT b, VARCHAR(65536) c, 
VARCHAR(65536) f0)"),
--- End diff --

Will do.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110341#comment-16110341
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user haohui commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r130789546
  
--- Diff: 
flink-libraries/flink-table/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java
 ---
@@ -1039,7 +1039,7 @@ public boolean argumentMustBeScalar(int ordinal) {
new SqlCollectionTableOperator("TABLE", SqlModality.RELATION);
 
public static final SqlOverlapsOperator OVERLAPS =
--- End diff --

Do you think it is a good idea to separate it into another PR? It might 
simplify the reviewing process.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-08-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110339#comment-16110339
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user dianfu commented on the issue:

https://github.com/apache/flink/pull/4373
  
I have did some investigation of the test failure of 
**JoinITCase.testJoinWithExpressionPreds** and would like to share my findings 
and solutions for your reference. The cause of this issue is that for preserved 
expressions, in **PushProjector#createProjectRefsAndExprs**, the column names 
corresponding to them will be the operator names of the expressions. For 
example for expression **a - 1** in the test case, the  column corresponding to 
it will be **-**. I think this behavior is not expected and have copied 
**PushProjector** from calcite and made some changes to it (line 507). Please 
refer to 
[here](https://github.com/dianfu/flink/commit/efa9641e0bd395a3679b0d496b60e3d42aa7b832)
 for more information.

For the files copied from calcite, all files can be removed except 
**SqlTimestampAddFunction** and **AuxiliaryConverter**.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-25 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099709#comment-16099709
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r129237531
  
--- Diff: 
flink-libraries/flink-table/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java
 ---
@@ -1869,13 +1869,23 @@ private RexNode convertOver(Blackboard bb, SqlNode 
node) {
// Walk over the tree and apply 'over' to all agg 
functions. This is
// necessary because the returned expression is not 
necessarily a call
// to an agg function. For example, AVG(x) becomes 
SUM(x) / COUNT(x).
+
--- End diff --

I think we can remove this file for now and rely on Calcite 1.13 
implementation. We don't need it anymore.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-25 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099711#comment-16099711
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r129237304
  
--- Diff: 
flink-libraries/flink-table/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java
 ---
@@ -1039,7 +1039,7 @@ public boolean argumentMustBeScalar(int ordinal) {
new SqlCollectionTableOperator("TABLE", SqlModality.RELATION);
 
public static final SqlOverlapsOperator OVERLAPS =
--- End diff --

I think we can remove this file for now and rely on Calcite 1.13 
implementation. We don't need it anymore.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-25 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099710#comment-16099710
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r129240196
  
--- Diff: 
flink-libraries/flink-table/src/test/scala/org/apache/flink/table/api/batch/sql/CorrelateTest.scala
 ---
@@ -43,7 +43,7 @@ class CorrelateTest extends TableTestBase {
 term("invocation", "func1($cor0.c)"),
 term("function", func1.getClass.getCanonicalName),
 term("rowType",
- "RecordType(INTEGER a, BIGINT b, VARCHAR(2147483647) c, 
VARCHAR(2147483647) f0)"),
+ "RecordType(INTEGER a, BIGINT b, VARCHAR(65536) c, 
VARCHAR(65536) f0)"),
--- End diff --

It seems that Calcite has defined an upper limit for strings and ignores 
our size. Can you also update this in `FlinkTypeSystem` and the comment in 
`FlinkTypeFactory#createSqlType`?


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-25 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099708#comment-16099708
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

Github user twalthr commented on a diff in the pull request:

https://github.com/apache/flink/pull/4373#discussion_r129239292
  
--- Diff: 
flink-libraries/flink-table/src/main/scala/org/apache/flink/table/expressions/time.scala
 ---
@@ -355,26 +355,36 @@ case class TemporalOverlaps(
   rightT: RexNode,
   relBuilder: FlinkRelBuilder)
 : RexNode = {
-// leftT = leftP + leftT if leftT is an interval
-val convLeftT = if (isTimeInterval(leftTemporal.resultType)) {
-relBuilder.call(SqlStdOperatorTable.DATETIME_PLUS, leftP, leftT)
-  } else {
-leftT
-  }
-// rightT = rightP + rightT if rightT is an interval
-val convRightT = if (isTimeInterval(rightTemporal.resultType)) {
-relBuilder.call(SqlStdOperatorTable.DATETIME_PLUS, rightP, rightT)
-  } else {
-rightT
-  }
-// leftT >= rightP
-val leftPred = 
relBuilder.call(SqlStdOperatorTable.GREATER_THAN_OR_EQUAL, convLeftT, rightP)
-// rightT >= leftP
-val rightPred = 
relBuilder.call(SqlStdOperatorTable.GREATER_THAN_OR_EQUAL, convRightT, leftP)
+val convLeftT = convertOverlapsEnd(relBuilder, leftP, leftT, 
leftTemporal.resultType)
--- End diff --

Can you open a separate PR for this? It should be part of FLINK-7159. I 
think we can get this in before this PR.


> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-19 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093844#comment-16093844
 ] 

ASF GitHub Bot commented on FLINK-6429:
---

GitHub user haohui opened a pull request:

https://github.com/apache/flink/pull/4373

[FLINK-6429] [table] Bump up Calcite version to 1.13.

This PR bumps upgrades Calcite from 1.12 to 1.13. It includes fixes for 
CALCITE-1884 and FLINK-7159. We can probably separate it to smaller PR as well.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/haohui/flink FLINK-6429

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4373.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4373


commit aa0e69d795f4b1d718bfa9b1ffc0f62f78243b43
Author: Haohui Mai 
Date:   2017-07-19T21:34:37Z

[FLINK-6429] Bump Calcite version to 1.13.




> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-18 Thread Haohui Mai (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091283#comment-16091283
 ] 

Haohui Mai commented on FLINK-6429:
---

Sorry for the delay. There are several issues that need to be addressed in the 
upgrade. I'll work on them first. I also have a WIP that I will upload shortly.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Haohui Mai
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-18 Thread Timo Walther (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16091250#comment-16091250
 ] 

Timo Walther commented on FLINK-6429:
-

[~wheat9] I will start working on this issue.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-11 Thread Haohui Mai (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083277#comment-16083277
 ] 

Haohui Mai commented on FLINK-6429:
---

[~twalthr] that would be great. I found a few issues and I'm adding it to this 
jira

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-11 Thread Timo Walther (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081835#comment-16081835
 ] 

Timo Walther commented on FLINK-6429:
-

[~wheat9] do you want to assign this issue then? I wanted to work on this week, 
but feel free to take it. We should also keep FLINK-6584 in mind.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-11 Thread Haohui Mai (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081820#comment-16081820
 ] 

Haohui Mai commented on FLINK-6429:
---

I have a patch and will upload it shortly.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-11 Thread Dian Fu (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081824#comment-16081824
 ] 

Dian Fu commented on FLINK-6429:


Thanks a lot for the update and looking forward to it. :)

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-07-10 Thread Dian Fu (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081588#comment-16081588
 ] 

Dian Fu commented on FLINK-6429:


Hi [~twalthr], could I know what's the progress of this ticket as FLINK-6935 
requires calcite 1.13.0 and it's currently blocked by this ticket. Thanks a lot.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>Assignee: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-06-30 Thread Timo Walther (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070071#comment-16070071
 ] 

Timo Walther commented on FLINK-6429:
-

I will assign this issue to me and coordinate it with the auxiliary group 
functions issues (see FLINK-6584).

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6429) Bump up Calcite version to 1.13

2017-06-26 Thread Fabian Hueske (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062877#comment-16062877
 ] 

Fabian Hueske commented on FLINK-6429:
--

The vote for Calcite 1.13.0 has passed today and it should be available soon.

> Bump up Calcite version to 1.13
> ---
>
> Key: FLINK-6429
> URL: https://issues.apache.org/jira/browse/FLINK-6429
> Project: Flink
>  Issue Type: Improvement
>  Components: Table API & SQL
>Reporter: Timo Walther
>
> This is an umbrella issue for all tasks that need to be done once Apache 
> Calcite 1.13 is released.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)