[jira] [Resolved] (CALCITE-3608) Promote RelOptUtil.createCastRel to not create new projection if the input rel is already a project

2019-12-18 Thread Danny Chen (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Danny Chen resolved CALCITE-3608.
-
Resolution: Fixed

Fixed in 
[d616a08|https://github.com/apache/calcite/commit/d616a080af2b86feeb3a0b526bb59bef94f94523]
 !

> Promote RelOptUtil.createCastRel to not create new projection if the input 
> rel is already a project
> ---
>
> Key: CALCITE-3608
> URL: https://issues.apache.org/jira/browse/CALCITE-3608
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.21.0
>Reporter: Danny Chen
>Assignee: Danny Chen
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.22.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3603) SqlLateralOperator's unparse add additional keyword 'LATERAL' when the inner operator is SqlSnapshot

2019-12-18 Thread Danny Chen (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999704#comment-16999704
 ] 

Danny Chen commented on CALCITE-3603:
-

It was created directly in the parser [1], which was introduced in CALCITE-1912 
for temporal table join.

[1] 
https://github.com/apache/calcite/blob/a289a75830ff86413bb809f76fd291902142/core/src/main/codegen/templates/Parser.jj#L2027

> SqlLateralOperator's unparse add additional keyword 'LATERAL' when the inner 
> operator is SqlSnapshot
> 
>
> Key: CALCITE-3603
> URL: https://issues.apache.org/jira/browse/CALCITE-3603
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.21.0
>Reporter: Kevin Zhang
>Assignee: Danny Chen
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.22.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> When joining with a dimension table using the following sql
> {code:sql}
> SELECT
>   o.amout, o.currency, r.rate, o.amount * r.rate
> FROM
>   Orders AS o
>   JOIN LatestRates FOR SYSTEM_TIME AS OF o.proctime AS r
>   ON r.currency = o.currency
> {code}
> the unparsed sql is:
> {code:sql}
> SELECT `o`.`amout`, `o`.`currency`, `r`.`rate`, `o`.`amount` * `r`.`rate`
> FROM `Orders` AS `o`
> INNER JOIN LATERAL `LatestRates` FOR SYSTEM_TIME AS OF `o`.`proctime` AS `r` 
> ON `r`.`currency` = `o`.`currency`
> {code}
> which has a syntax error because an additional "LATERAL" is added after 
> "JOIN".
> The problem lies in SqlLateralOperator's unparse method, if the kind of the 
> first operand is SqlSnapshot, we should not write out the operator's name.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (CALCITE-3610) Fix avatica dockerfiles

2019-12-18 Thread Francis Chuang (Jira) 
Francis Chuang created CALCITE-3610:
---

 Summary: Fix avatica dockerfiles
 Key: CALCITE-3610
 URL: https://issues.apache.org/jira/browse/CALCITE-3610
 Project: Calcite
  Issue Type: Task
  Components: avatica
Reporter: Francis Chuang
Assignee: Francis Chuang


The dockerfiles current reference vatica-standalone-server-1.16.0-shaded.jar. 
These jars no longer exist and are now avatica-standalone-server-1.16.0.jar. 
The dockerifles should be updated to point to these new filenames.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (CALCITE-3398) Release avatica 1.16.0

2019-12-18 Thread Francis Chuang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francis Chuang closed CALCITE-3398.
---

Resolved in release Calcite-Avatica 1.16.0

> Release avatica 1.16.0
> --
>
> Key: CALCITE-3398
> URL: https://issues.apache.org/jira/browse/CALCITE-3398
> Project: Calcite
>  Issue Type: Improvement
>  Components: avatica
>Reporter: Francis Chuang
>Assignee: Francis Chuang
>Priority: Major
> Fix For: avatica-1.16.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (CALCITE-3398) Release avatica 1.16.0

2019-12-18 Thread Francis Chuang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francis Chuang resolved CALCITE-3398.
-
Resolution: Fixed

> Release avatica 1.16.0
> --
>
> Key: CALCITE-3398
> URL: https://issues.apache.org/jira/browse/CALCITE-3398
> Project: Calcite
>  Issue Type: Improvement
>  Components: avatica
>Reporter: Francis Chuang
>Assignee: Francis Chuang
>Priority: Major
> Fix For: avatica-1.16.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3563) When resolving method call in calcite runtime, add type check and match mechanism for input arguments

2019-12-18 Thread Julian Hyde (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999496#comment-16999496
 ] 

Julian Hyde commented on CALCITE-3563:
--

bq. shall we make strict check in validation phase before entring into runtime?

I think it is worthwhile validating more strictly when doing code generation 
(i.e. when creating a {{MethodCallExpression}}). We will find problems earlier. 
There will be some work to do - there are a few places where we call overloaded 
methods (e.g. we only have one {{BuiltInMethod.SUBSTRING}} enum constant but 
there are 4 {{substring}} methods in {{SqlFunctions}}).

> When resolving method call in calcite runtime, add type check and match 
> mechanism for input arguments
> -
>
> Key: CALCITE-3563
> URL: https://issues.apache.org/jira/browse/CALCITE-3563
> Project: Calcite
>  Issue Type: Sub-task
>  Components: core
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Sub-task targets on following issue.
>  
> (1)Validation phase allows a wide range of operand types, but the runtime 
> implementation does not cover all cases.
>  For example, _*SqlFunction(MOD)*_ adopts 
> _*OperandTypes.EXACT_NUMERIC_EXACT_NUMERIC*_.
> {code:java}
> @Test public void test0() {
>  final String sql = "SELECT mod(12.5, cast(1 as bigint))";
>  CalciteAssert.that()
>  .query(sql)
>  .returns("EXPR$0=0.5\n");
>  }{code}
> We will get:
> {code:java}
> java.lang.RuntimeException: while resolving method 'mod[class 
> java.math.BigDecimal, long]' in class class 
> org.apache.calcite.runtime.SqlFunctions
>  at org.apache.calcite.linq4j.tree.Types.lookupMethod(Types.java:323)
>  at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:445)
>  at 
> org.apache.calcite.adapter.enumerable.RexImpTable$MethodNameImplementor.implement(RexImpTable.java:2253)
>  at 
> org.apache.calcite.adapter.enumerable.RexImpTable.implementCall(RexImpTable.java:1195){code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread Rui Wang (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999401#comment-16999401
 ] 

Rui Wang commented on CALCITE-3609:
---

I am not aware of Calcite re-release past versions if there are patches to 
those specific versions(and that's too much loads). 

A common solution in open source community is to build a long-term support 
version(LTS) from a common-used and stable version, and accept patches there 
and keep releasing, but I am not aware of it existing in Calcite either.

You probably have to upgrade your calcite version and if there is still a bug, 
submit patch to the master branch and use the next release after it.

> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
> Attachments: calcite-3608.patch
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1]) 
>   HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>  
> after fix RelRecordType isNullable,the plan is
> {code:java}
> HiveProject(id=[$0], info=[$1])
>   HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
> then the result is:
> {code:java}
> 1 {"name":"zhou","age":30} 
> 2 {"name":"yan","age":30} 
> 3 {"name":"chen","age":20} 
> 4 {"name":"li","age":80}
> NULL{"name":null,"age":null}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3603) SqlLateralOperator's unparse add additional keyword 'LATERAL' when the inner operator is SqlSnapshot

2019-12-18 Thread Julian Hyde (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999383#comment-16999383
 ] 

Julian Hyde commented on CALCITE-3603:
--

I still don't understand why SqlLateralOperator is present.

> SqlLateralOperator's unparse add additional keyword 'LATERAL' when the inner 
> operator is SqlSnapshot
> 
>
> Key: CALCITE-3603
> URL: https://issues.apache.org/jira/browse/CALCITE-3603
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.21.0
>Reporter: Kevin Zhang
>Assignee: Danny Chen
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.22.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> When joining with a dimension table using the following sql
> {code:sql}
> SELECT
>   o.amout, o.currency, r.rate, o.amount * r.rate
> FROM
>   Orders AS o
>   JOIN LatestRates FOR SYSTEM_TIME AS OF o.proctime AS r
>   ON r.currency = o.currency
> {code}
> the unparsed sql is:
> {code:sql}
> SELECT `o`.`amout`, `o`.`currency`, `r`.`rate`, `o`.`amount` * `r`.`rate`
> FROM `Orders` AS `o`
> INNER JOIN LATERAL `LatestRates` FOR SYSTEM_TIME AS OF `o`.`proctime` AS `r` 
> ON `r`.`currency` = `o`.`currency`
> {code}
> which has a syntax error because an additional "LATERAL" is added after 
> "JOIN".
> The problem lies in SqlLateralOperator's unparse method, if the kind of the 
> first operand is SqlSnapshot, we should not write out the operator's name.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999179#comment-16999179
 ] 

hezhang commented on CALCITE-3609:
--

We are mainly using hive-2.3.2, hive-3.1.1, they depend on 1.10.0 and 1.16.0 
respectively. so only update code in calcite-1.16.0.:D

> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
> Attachments: calcite-3608.patch
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1]) 
>   HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>  
> after fix RelRecordType isNullable,the plan is
> {code:java}
> HiveProject(id=[$0], info=[$1])
>   HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
> then the result is:
> {code:java}
> 1 {"name":"zhou","age":30} 
> 2 {"name":"yan","age":30} 
> 3 {"name":"chen","age":20} 
> 4 {"name":"li","age":80}
> NULL{"name":null,"age":null}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread Jin Xing (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999168#comment-16999168
 ] 

Jin Xing commented on CALCITE-3609:
---

Seems the patch is not against latest code base.

This issue is already fixed by 
[http://issues.apache.org/jira/browse/CALCITE-2464]

> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
> Attachments: calcite-3608.patch
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1]) 
>   HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>  
> after fix RelRecordType isNullable,the plan is
> {code:java}
> HiveProject(id=[$0], info=[$1])
>   HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
> then the result is:
> {code:java}
> 1 {"name":"zhou","age":30} 
> 2 {"name":"yan","age":30} 
> 3 {"name":"chen","age":20} 
> 4 {"name":"li","age":80}
> NULL{"name":null,"age":null}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hezhang updated CALCITE-3609:
-
Attachment: calcite-3608.patch

> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
> Attachments: calcite-3608.patch
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1]) 
>   HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>  
> after fix RelRecordType isNullable,the plan is
> {code:java}
> HiveProject(id=[$0], info=[$1])
>   HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
> then the result is:
> {code:java}
> 1 {"name":"zhou","age":30} 
> 2 {"name":"yan","age":30} 
> 3 {"name":"chen","age":20} 
> 4 {"name":"li","age":80}
> NULL{"name":null,"age":null}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hezhang updated CALCITE-3609:
-
Description: 
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}
the sql:

 
{code:java}
select * from student where info is not null;
{code}
result:

 
{code:java}
1   {"name":"zhou","age":30}
2   {"name":"yan","age":30}
3   {"name":"chen","age":20}
4   {"name":"li","age":80}
NULLNULL
NULL{"name":null,"age":null}
{code}
cause:calcite Ineffective optimization
{code:java}
HiveProject(id=[$0], info=[$1]) 
  HiveTableScan(table=[[default.student]], table:alias=[student])
{code}
 

after fix RelRecordType isNullable,the plan is
{code:java}
HiveProject(id=[$0], info=[$1])
  HiveFilter(condition=[IS NOT NULL($1)])
HiveTableScan(table=[[default.student]], table:alias=[student])
{code}
then the result is:
{code:java}
1 {"name":"zhou","age":30} 
2 {"name":"yan","age":30} 
3 {"name":"chen","age":20} 
4 {"name":"li","age":80}
NULL{"name":null,"age":null}

{code}
 

  was:
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}
the sql:

 
{code:java}
select * from student where info is not null;
{code}
result:

 
{code:java}
1   {"name":"zhou","age":30}
2   {"name":"yan","age":30}
3   {"name":"chen","age":20}
4   {"name":"li","age":80}
NULLNULL
NULL{"name":null,"age":null}
{code}
cause:calcite Ineffective optimization
{code:java}
HiveProject(id=[$0], info=[$1]) 
  HiveTableScan(table=[[default.student]], table:alias=[student])
{code}
 

after fix RelRecordType isNullable,the plan is

 


> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1]) 
>   HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>  
> after fix RelRecordType isNullable,the plan is
> {code:java}
> HiveProject(id=[$0], info=[$1])
>   HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
> then the result is:
> {code:java}
> 1 {"name":"zhou","age":30} 
> 2 {"name":"yan","age":30} 
> 3 {"name":"chen","age":20} 
> 4 {"name":"li","age":80}
> NULL{"name":null,"age":null}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hezhang updated CALCITE-3609:
-
Description: 
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}
the sql:

 
{code:java}
select * from student where info is not null;
{code}
result:

 
{code:java}
1   {"name":"zhou","age":30}
2   {"name":"yan","age":30}
3   {"name":"chen","age":20}
4   {"name":"li","age":80}
NULLNULL
NULL{"name":null,"age":null}
{code}
cause:calcite Ineffective optimization
{code:java}
HiveProject(id=[$0], info=[$1]) 
  HiveTableScan(table=[[default.student]], table:alias=[student])
{code}
 

after fix RelRecordType isNullable,the plan is

 

  was:
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}
the sql:

 
{code:java}
select * from student where info is not null;
{code}
result:

 
{code:java}
1   {"name":"zhou","age":30}
2   {"name":"yan","age":30}
3   {"name":"chen","age":20}
4   {"name":"li","age":80}
NULLNULL
NULL{"name":null,"age":null}
{code}
cause:calcite Ineffective optimization
{code:java}
HiveProject(id=[$0], info=[$1]) HiveTableScan(table=[[default.student]], 
table:alias=[student])
{code}
 

after fix RelRecordType isNullable,the plan is

 


> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1]) 
>   HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>  
> after fix RelRecordType isNullable,the plan is
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hezhang updated CALCITE-3609:
-
Description: 
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}
the sql:

 
{code:java}
select * from student where info is not null;
{code}
result:

 
{code:java}
1   {"name":"zhou","age":30}
2   {"name":"yan","age":30}
3   {"name":"chen","age":20}
4   {"name":"li","age":80}
NULLNULL
NULL{"name":null,"age":null}
{code}
cause:calcite Ineffective optimization
{code:java}
HiveProject(id=[$0], info=[$1]) HiveTableScan(table=[[default.student]], 
table:alias=[student])
{code}
 

after fix RelRecordType isNullable,the plan is

 

  was:
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}
the sql:

 
{code:java}
select * from student where info is not null;
{code}
result: calcite 

 
{code:java}
1   {"name":"zhou","age":30}
2   {"name":"yan","age":30}
3   {"name":"chen","age":20}
4   {"name":"li","age":80}
NULLNULL
NULL{"name":null,"age":null}
{code}
cause:
{code:java}
{code}
 


> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1]) HiveTableScan(table=[[default.student]], 
> table:alias=[student])
> {code}
>  
> after fix RelRecordType isNullable,the plan is
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hezhang updated CALCITE-3609:
-
Description: 
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}
the sql:

 
{code:java}
select * from student where info is not null;
{code}
result: calcite 

 
{code:java}
1   {"name":"zhou","age":30}
2   {"name":"yan","age":30}
3   {"name":"chen","age":20}
4   {"name":"li","age":80}
NULLNULL
NULL{"name":null,"age":null}
{code}
cause:
{code:java}
{code}
 

  was:
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}

the sql:



> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:
>  
> {code:java}
> select * from student where info is not null;
> {code}
> result: calcite 
>  
> {code:java}
> 1   {"name":"zhou","age":30}
> 2   {"name":"yan","age":30}
> 3   {"name":"chen","age":20}
> 4   {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:
> {code:java}
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hezhang updated CALCITE-3609:
-
Description: 
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}

the sql:


  was:
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}


> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}
> the sql:



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3609) calite for hive struct Ineffective optimization

2019-12-18 Thread hezhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hezhang updated CALCITE-3609:
-
Description: 
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')  
{code}

  was:
student table：
{code:java}
CREATE TABLE `student`(
  `id` int, 
  `info` struct)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
WITH SERDEPROPERTIES ( 
  'colelction.delim'=':', 
  'field.delim'=',', 
  'serialization.format'=',', 
  'serialization.null.format'='NULL')
{code}


> calite for hive struct  Ineffective optimization
> 
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Fix For: 1.16.0
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
>   `id` int, 
>   `info` struct)
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
> WITH SERDEPROPERTIES ( 
>   'colelction.delim'=':', 
>   'field.delim'=',', 
>   'serialization.format'=',', 
>   'serialization.null.format'='NULL')  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (CALCITE-3563) When resolving method call in calcite runtime, add type check and match mechanism for input arguments

2019-12-18 Thread Feng Zhu (Jira) 


 [ 
https://issues.apache.org/jira/browse/CALCITE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Feng Zhu updated CALCITE-3563:
--
Summary: When resolving method call in calcite runtime, add type check and 
match mechanism for input arguments  (was: Convert function operand type to 
match implementation if necessary in runtime)

> When resolving method call in calcite runtime, add type check and match 
> mechanism for input arguments
> -
>
> Key: CALCITE-3563
> URL: https://issues.apache.org/jira/browse/CALCITE-3563
> Project: Calcite
>  Issue Type: Sub-task
>  Components: core
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Sub-task targets on following issue.
>  
> (1)Validation phase allows a wide range of operand types, but the runtime 
> implementation does not cover all cases.
>  For example, _*SqlFunction(MOD)*_ adopts 
> _*OperandTypes.EXACT_NUMERIC_EXACT_NUMERIC*_.
> {code:java}
> @Test public void test0() {
>  final String sql = "SELECT mod(12.5, cast(1 as bigint))";
>  CalciteAssert.that()
>  .query(sql)
>  .returns("EXPR$0=0.5\n");
>  }{code}
> We will get:
> {code:java}
> java.lang.RuntimeException: while resolving method 'mod[class 
> java.math.BigDecimal, long]' in class class 
> org.apache.calcite.runtime.SqlFunctions
>  at org.apache.calcite.linq4j.tree.Types.lookupMethod(Types.java:323)
>  at org.apache.calcite.linq4j.tree.Expressions.call(Expressions.java:445)
>  at 
> org.apache.calcite.adapter.enumerable.RexImpTable$MethodNameImplementor.implement(RexImpTable.java:2253)
>  at 
> org.apache.calcite.adapter.enumerable.RexImpTable.implementCall(RexImpTable.java:1195){code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (CALCITE-3606) batch insert failed

2019-12-18 Thread Danny Chen (Jira) 


[ 
https://issues.apache.org/jira/browse/CALCITE-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999003#comment-16999003
 ] 

Danny Chen commented on CALCITE-3606:
-

Thanks [~caoran] for reporting this, a re-producable test case would help a 
lot, you can add one in JdbcTest or RelOptRulesTest.

> batch insert failed
> ---
>
> Key: CALCITE-3606
> URL: https://issues.apache.org/jira/browse/CALCITE-3606
> Project: Calcite
>  Issue Type: Wish
>  Components: core
>Affects Versions: 1.21.0
>Reporter: Ran Cao
>Priority: Major
>
> when I try to execute sql like (insert into example_table (column1,column2) 
> values (value1,value2),(value1,value2)), it will failed with error message 
> like this: column "EXPR$0" of relation "example_table" does not exist. I find 
> the reason is that when converting SqlNode(insert sql) to 
> RelNode(TableModify), one of the steps is to change the column that stored in 
> RelDataType from the fake column name (like "EXPR$0") to the real column name 
> (like "id"). But when the values part in sql is more than one , the step 
> above-mentioned will skip because the RelNode is instance of  LogicalUnion 
> instead of Project, the code refered to org.apache.calcite.tools.RelBuilder 
> line 1461:
> if (input instanceof Project && fieldNames != null) {
>     // change the column name
> }
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)