[jira] [Commented] (CALCITE-1581) UDTF like in hive
[ https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906962#comment-16906962 ] Stamatis Zampetakis commented on CALCITE-1581: -- Hi [~Chunwei Lei], did you have the chance to review the PR? Will it make it in 1.21.0? > UDTF like in hive > - > > Key: CALCITE-1581 > URL: https://issues.apache.org/jira/browse/CALCITE-1581 > Project: Calcite > Issue Type: New Feature >Reporter: Xiaoyong Deng >Assignee: pengzhiwei >Priority: Major > Labels: pull-request-available, udtf > Fix For: 1.21.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Support one row in and multi-column/multi-row out(one-to-many mapping), just > like udtf in hive. > The query would like this: > {code} > select > func(c0, c1) as (f0, f1, f2) > from table_name; > {code} > c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3249) Substitution#getRexShuttle does not consider RexLiteral
jin xing created CALCITE-3249: - Summary: Substitution#getRexShuttle does not consider RexLiteral Key: CALCITE-3249 URL: https://issues.apache.org/jira/browse/CALCITE-3249 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing Assignee: jin xing -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3249) Substitution#getRexShuttle does not consider RexLiteral
[ https://issues.apache.org/jira/browse/CALCITE-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated CALCITE-3249: -- Description: Current {{Substitution#getRexShuttle}} does not take {{RexLiteral}} into consideration. Thus below query & mv fails matching: {code:java} MV: select deptno, sum(salary), sum(commission), sum(k) from (select deptno, salary, commission, 100 as k from emps) group by deptno Query: select deptno, sum(salary), sum(k) from (select deptno, salary, 100 as k from emps) group by deptno {code} The root cause is that {{ProjectToProjectUnifyRule}} compensates a {{Project}} which contains {{RexLiteral}}, but {{AggregateOnProjectToAggregateUnifyRule}} works only when the {{Project}} in query is a mapping https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L1357 was: Current {{Substitution#getRexShuttle}} does not take {{RexLiteral}} into consideration. Thus below query & mv fails matching: {code:java} MV: select deptno, sum(salary), sum(commission), sum(k) from (select deptno, salary, commission, 100 as k from emps) group by deptno Query: select deptno, sum(salary), sum(k) from (select deptno, salary, 100 as k from emps) group by deptno {code} The root cause is that {{ProjectToProjectUnifyRule}} compensate a {{Project}} which contains {{RexLiteral}}, but {{AggregateOnProjectToAggregateUnifyRule}} works only when the {{Project}} in query is a mapping https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L1357 > Substitution#getRexShuttle does not consider RexLiteral > --- > > Key: CALCITE-3249 > URL: https://issues.apache.org/jira/browse/CALCITE-3249 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: jin xing >Assignee: jin xing >Priority: Minor > > Current {{Substitution#getRexShuttle}} does not take {{RexLiteral}} into > consideration. > Thus below query & mv fails matching: > {code:java} > MV: > select deptno, sum(salary), sum(commission), sum(k) > from > (select deptno, salary, commission, 100 as k > from emps) > group by deptno > Query: > select deptno, sum(salary), sum(k) > from > (select deptno, salary, 100 as k > from emps) > group by deptno > {code} > The root cause is that {{ProjectToProjectUnifyRule}} compensates a > {{Project}} which contains {{RexLiteral}}, but > {{AggregateOnProjectToAggregateUnifyRule}} works only when the {{Project}} in > query is a mapping > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L1357 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3249) Substitution#getRexShuttle does not consider RexLiteral
[ https://issues.apache.org/jira/browse/CALCITE-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated CALCITE-3249: -- Description: Current {{Substitution#getRexShuttle}} does not take {{RexLiteral}} into consideration. Thus below query & mv fails matching: {code:java} MV: select deptno, sum(salary), sum(commission), sum(k) from (select deptno, salary, commission, 100 as k from emps) group by deptno Query: select deptno, sum(salary), sum(k) from (select deptno, salary, 100 as k from emps) group by deptno {code} The root cause is that {{ProjectToProjectUnifyRule}} compensate a {{Project}} which contains {{RexLiteral}}, but {{AggregateOnProjectToAggregateUnifyRule}} works only when the {{Project}} in query is a mapping https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L1357 > Substitution#getRexShuttle does not consider RexLiteral > --- > > Key: CALCITE-3249 > URL: https://issues.apache.org/jira/browse/CALCITE-3249 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: jin xing >Assignee: jin xing >Priority: Minor > > Current {{Substitution#getRexShuttle}} does not take {{RexLiteral}} into > consideration. > Thus below query & mv fails matching: > {code:java} > MV: > select deptno, sum(salary), sum(commission), sum(k) > from > (select deptno, salary, commission, 100 as k > from emps) > group by deptno > Query: > select deptno, sum(salary), sum(k) > from > (select deptno, salary, 100 as k > from emps) > group by deptno > {code} > The root cause is that {{ProjectToProjectUnifyRule}} compensate a {{Project}} > which contains {{RexLiteral}}, but {{AggregateOnProjectToAggregateUnifyRule}} > works only when the {{Project}} in query is a mapping > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L1357 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3249) Substitution#getRexShuttle does not consider RexLiteral
[ https://issues.apache.org/jira/browse/CALCITE-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3249: Labels: pull-request-available (was: ) > Substitution#getRexShuttle does not consider RexLiteral > --- > > Key: CALCITE-3249 > URL: https://issues.apache.org/jira/browse/CALCITE-3249 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: jin xing >Assignee: jin xing >Priority: Minor > Labels: pull-request-available > > Current {{Substitution#getRexShuttle}} does not take {{RexLiteral}} into > consideration. > Thus below query & mv fails matching: > {code:java} > MV: > select deptno, sum(salary), sum(commission), sum(k) > from > (select deptno, salary, commission, 100 as k > from emps) > group by deptno > Query: > select deptno, sum(salary), sum(k) > from > (select deptno, salary, 100 as k > from emps) > group by deptno > {code} > The root cause is that {{ProjectToProjectUnifyRule}} compensates a > {{Project}} which contains {{RexLiteral}}, but > {{AggregateOnProjectToAggregateUnifyRule}} works only when the {{Project}} in > query is a mapping > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/plan/SubstitutionVisitor.java#L1357 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (CALCITE-3238) Support Time Zone suffix of DateTime types for SqlDataTypeSpec
[ https://issues.apache.org/jira/browse/CALCITE-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3238. - Resolution: Fixed Fix Version/s: 1.21.0 Fixed in [f25734a|https://github.com/apache/calcite/commit/f25734a2d27f304a454a310e656ab83a5fa85890] ! > Support Time Zone suffix of DateTime types for SqlDataTypeSpec > -- > > Key: CALCITE-3238 > URL: https://issues.apache.org/jira/browse/CALCITE-3238 > Project: Calcite > Issue Type: Sub-task >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > This is the grammar copied from SQL-2011: > {code:sql} > ::= > DATE > | TIME [] [ time zone> ] > | TIMESTAMP [] > [ ] > ::= > WITH TIME ZONE > | WITHOUT TIME ZONE > ::= > > ::= > > ::= > > {code} > We extend this grammar to support "with local time zone" definition: > {code:sql} > time(0) with local time zone > timestamp(0) with local time zone > {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3250) Support nested collection type for SqlDataTypeSpec
Danny Chan created CALCITE-3250: --- Summary: Support nested collection type for SqlDataTypeSpec Key: CALCITE-3250 URL: https://issues.apache.org/jira/browse/CALCITE-3250 Project: Calcite Issue Type: Sub-task Reporter: Danny Chan Assignee: Danny Chan According to SQL-2011, collection type can be nested: {code:sql} ::= ARRAY [] ::= {code} *Example*: {code:sql} int array array; int multiset array; {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3241) Calcite this is a very strange mistake
[ https://issues.apache.org/jira/browse/CALCITE-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907044#comment-16907044 ] Danny Chan commented on CALCITE-3241: - [~1693294577] Thanks, can you summarize and make an abstract the issue about the error, or even better, can you give a test case in Calcite to reproduce this problem ? It would helps a lot for the contributors who has willingness on this issue. > Calcite this is a very strange mistake > -- > > Key: CALCITE-3241 > URL: https://issues.apache.org/jira/browse/CALCITE-3241 > Project: Calcite > Issue Type: Bug >Reporter: ;ianghaibin >Priority: Major > > 0: jdbc:calcite:model=/root/calcite/apache-ca> select sum("xxx") as a from > "xxx"; > java.lang.NumberFormatException: For input string: "13485468068" > at > ru.yandex.clickhouse.response.ByteFragmentUtils.parseInt(ByteFragmentUtils.java:61) > at > ru.yandex.clickhouse.response.ClickHouseResultSet.getInt(ClickHouseResultSet.java:342) > at > org.apache.commons.dbcp2.DelegatingResultSet.getInt(DelegatingResultSet.java:623) > at > org.apache.commons.dbcp2.DelegatingResultSet.getInt(DelegatingResultSet.java:623) > at Baz$1$1.apply(Unknown Source) > at > org.apache.calcite.runtime.ResultSetEnumerable$ResultSetEnumerator.current(ResultSetEnumerable.java:365) > at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:683) > at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at sqlline.BufferedRows.nextList(BufferedRows.java:110) > at sqlline.BufferedRows.(BufferedRows.java:52) > at sqlline.SqlLine.print(SqlLine.java:1640) > at sqlline.Commands.executeSingleQuery(Commands.java:1017) > at sqlline.Commands.execute(Commands.java:957) > at sqlline.Commands.sql(Commands.java:921) > at sqlline.SqlLine.dispatch(SqlLine.java:717) > at sqlline.SqlLine.begin(SqlLine.java:536) > at sqlline.SqlLine.start(SqlLine.java:266) > at sqlline.SqlLine.main(SqlLine.java:205) > The data I got is integer, and I am getting an error. > However, this is not an error. > select "xx","xx",sum("xx") as xx from "xx" group by "xx","xxx" limit 1; -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3138) Restructuring ROW type fields is not supported
[ https://issues.apache.org/jira/browse/CALCITE-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Guzenko updated CALCITE-3138: -- Description: 1) RelStructuredTypeFlattener.restructureFields(structType) doesn't support ROW type. However, ROW type is flattened by RelStructuredTypeFlattener just like struct. So when user queries one column with complex type ROW, after flattening and restructuring top level project returns reference to first inner primitive field of the requested column. 2) Another finding is related to ITEM expression applied to array of structs column. For example, let's imagine a table with column of type ARRAY>>. When user requests is SQL array_column[1], Calcite generates ITEM($0, 1) , where $0 is ref to array column from Scan and 1 is index literal. Current flattener generates two field acess expressions ITEM($0, 1).a, ITEM($0, 1).b but dont take into account that ITEM($0, 1).b returns struct which also should be flattened. 3) In some cases applying of ITEM after flattenning is no longer possible. For example, consider column with type STRUCT>. User requests column['b'] in query and Calcite creates ITEM($0,'b'). After flattenning Scan is covered by LogicalProject($0.a, $0.b.x, $0.b.y) and the old projection ITEM($0,'b') can't be applied anymore. So now it should be converted to refer only subset of fields ($1,$2) from flattening project. UPDATES IN EXPECTED TEST RESULTS: -- TEST CASE: SqlToRelConverterTest.testNestedColumnType() {code} select empa.home_address.zip from sales.emp_address empa where empa.home_address.city = 'abc' {code} OLD RESULT: {code} LogicalProject(ZIP=[$4]) LogicalFilter(condition=[=($3, 'abc':VARCHAR(20))]) LogicalProject(EMPNO=[$0], STREET=[$1.STREET], CITY=[$1.CITY], ZIP=[$1.ZIP], STATE=[$1.STATE], STREET5=[$2.STREET], CITY6=[$2.CITY], ZIP7=[$2.ZIP], STATE8=[$2.STATE]) LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]]) {code} 1. Above in logical filter, condition references to field $3 which is ZIP=[$1.ZIP] field from previous project, however in original query filtering should be done by CITY field. 2. Also the top level project references to $4 field, which is STATE=[$1.STATE] field from project, but original query requested ZIP field. UPDATED RESULT: {code} LogicalProject(ZIP=[$3]) LogicalFilter(condition=[=($2, 'abc')]) LogicalProject(EMPNO=[$0], STREET=[$1.STREET], CITY=[$1.CITY], ZIP=[$1.ZIP], STATE=[$1.STATE], STREET5=[$2.STREET], CITY6=[$2.CITY], ZIP7=[$2.ZIP], STATE8=[$2.STATE]) LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]]) {code} -- TEST CASE: SqlToRelConverterTest.testStructTypeAlias() {code} select t.r AS myRow from (select row(row(1)) r from dept) t {code} OLD RESULT: {code} LogicalProject(MYROW$$0$$0=[1]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) {code} 1. Inside the subselect of row(row(1)) type of returned column is RecordType(RecordType(INTEGER EXPR$0) EXPR$0), but the top level project uses flattened expression and returns to user literal 1 with type RecordType(INTEGER MYROW$$0$$0), although the type doesn't match type returned by row(row(1)) expression. 2. Also it's suspicious that caller expects returned column to have name 'myRow' but gets 'MYROW$$0$$0'. UPDATED RESULT: {code} LogicalProject(MYROW=[ROW(ROW($0))]) LogicalProject(MYROW$$0$$0=[1]) LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) {code} -- TEST CASE: SqlToRelConverterTest.testFlattenRecords() {code} select employees[1] from dept_nested {code} OLD RESULT: {code} LogicalProject(EXPR$0=[$0]) LogicalProject(EXPR$0$0=[ITEM($3, 1).EMPNO], EXPR$0$1=[ITEM($3, 1).ENAME], EXPR$0$2=[ITEM($3, 1).DETAIL]) LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]]) {code} 1. Given selection of element by index from array, which by definition contain elements of type: {code} STRUCT {code} First problem is that the intermediate project on top of the scan expected to return only flattened types. But last projection expression returns `DETAIL` field of array element which has type STRUCT and contain array field named 'SKILLS'. 2. The top level project returns incorrect flattened EMPNO:INT field from retrieved array element instead of whole array element of type struct. UPDATED RESULT: {code} LogicalProject(EXPR$0=[ROW($0, $1, ROW($2))]) LogicalProject(EXPR$0$0=[ITEM($3, 1).EMPNO], EXPR$0$1=[ITEM($3, 1).ENAME], EXPR$0$2$0=[ITEM($3, 1).DETAIL.SKILLS]) LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]]) {code} Other few updates in RelToSqlConverterStructsTest.java and PlannerTest.java are more explicit and visible in PR.
[jira] [Commented] (CALCITE-3138) Restructuring ROW type fields is not supported
[ https://issues.apache.org/jira/browse/CALCITE-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907054#comment-16907054 ] Igor Guzenko commented on CALCITE-3138: --- Hello [~julianhyde], I've updated description. > Restructuring ROW type fields is not supported > -- > > Key: CALCITE-3138 > URL: https://issues.apache.org/jira/browse/CALCITE-3138 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Haisheng Yuan >Assignee: Igor Guzenko >Priority: Major > Labels: pull-request-available > Attachments: ROW_repro.patch > > Time Spent: 20m > Remaining Estimate: 0h > > 1) RelStructuredTypeFlattener.restructureFields(structType) doesn't support > ROW type. However, ROW type is flattened by > RelStructuredTypeFlattener just like struct. So when user queries one column > with complex type ROW, after flattening and restructuring > top level project returns reference to first inner primitive field of the > requested column. > 2) Another finding is related to ITEM expression applied to array of structs > column. For example, let's imagine a table with column of > type ARRAY>>. When user requests is SQL > array_column[1], Calcite generates ITEM($0, 1) , where $0 is ref to array > column > from Scan and 1 is index literal. Current flattener generates two field acess > expressions ITEM($0, 1).a, ITEM($0, 1).b but dont take into account > that ITEM($0, 1).b returns struct which also should be flattened. > 3) In some cases applying of ITEM after flattenning is no longer possible. > For example, consider column with type > STRUCT>. User requests column['b'] in query and > Calcite creates ITEM($0,'b'). > After flattenning Scan is covered by LogicalProject($0.a, $0.b.x, $0.b.y) and > the old projection ITEM($0,'b') can't > be applied anymore. So now it should be converted to refer only subset of > fields ($1,$2) from flattening project. > UPDATES IN EXPECTED TEST RESULTS: > -- > TEST CASE: SqlToRelConverterTest.testNestedColumnType() > {code} > select empa.home_address.zip from sales.emp_address empa where > empa.home_address.city = 'abc' > {code} > OLD RESULT: > {code} > LogicalProject(ZIP=[$4]) > LogicalFilter(condition=[=($3, 'abc':VARCHAR(20))]) > LogicalProject(EMPNO=[$0], STREET=[$1.STREET], CITY=[$1.CITY], > ZIP=[$1.ZIP], STATE=[$1.STATE], STREET5=[$2.STREET], CITY6=[$2.CITY], > ZIP7=[$2.ZIP], STATE8=[$2.STATE]) > LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]]) > {code} > 1. Above in logical filter, condition references to field $3 which is > ZIP=[$1.ZIP] field from previous project, >however in original query filtering should be done by CITY field. > 2. Also the top level project references to $4 field, which is > STATE=[$1.STATE] field from project, but original >query requested ZIP field. > > UPDATED RESULT: > {code} > LogicalProject(ZIP=[$3]) > LogicalFilter(condition=[=($2, 'abc')]) > LogicalProject(EMPNO=[$0], STREET=[$1.STREET], CITY=[$1.CITY], > ZIP=[$1.ZIP], STATE=[$1.STATE], STREET5=[$2.STREET], CITY6=[$2.CITY], > ZIP7=[$2.ZIP], STATE8=[$2.STATE]) > LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]]) > {code} > -- > TEST CASE: SqlToRelConverterTest.testStructTypeAlias() > {code} > select t.r AS myRow from (select row(row(1)) r from dept) t > {code} > OLD RESULT: > {code} > LogicalProject(MYROW$$0$$0=[1]) > LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) > {code} > 1. Inside the subselect of row(row(1)) type of returned column is > RecordType(RecordType(INTEGER EXPR$0) EXPR$0), >but the top level project uses flattened expression and returns to user > literal 1 with type RecordType(INTEGER MYROW$$0$$0), >although the type doesn't match type returned by row(row(1)) expression. > 2. Also it's suspicious that caller expects returned column to have name > 'myRow' but gets 'MYROW$$0$$0'. > > UPDATED RESULT: > {code} > LogicalProject(MYROW=[ROW(ROW($0))]) > LogicalProject(MYROW$$0$$0=[1]) > LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) > {code} > -- > TEST CASE: SqlToRelConverterTest.testFlattenRecords() > {code} > select employees[1] from dept_nested > {code} > OLD RESULT: > {code} > LogicalProject(EXPR$0=[$0]) > LogicalProject(EXPR$0$0=[ITEM($3, 1).EMPNO], EXPR$0$1=[ITEM($3, 1).ENAME], > EXPR$0$2=[ITEM($3, 1).DETAIL]) > LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]]) > {code} > 1. Given selection of element by index from array, which by definition > contain elements of type: > {c
[jira] [Commented] (CALCITE-1581) UDTF like in hive
[ https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907079#comment-16907079 ] Chunwei Lei commented on CALCITE-1581: -- [~zabetak], Sorry for the delay. I will review it this week. > UDTF like in hive > - > > Key: CALCITE-1581 > URL: https://issues.apache.org/jira/browse/CALCITE-1581 > Project: Calcite > Issue Type: New Feature >Reporter: Xiaoyong Deng >Assignee: pengzhiwei >Priority: Major > Labels: pull-request-available, udtf > Fix For: 1.21.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > Support one row in and multi-column/multi-row out(one-to-many mapping), just > like udtf in hive. > The query would like this: > {code} > select > func(c0, c1) as (f0, f1, f2) > from table_name; > {code} > c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (CALCITE-3236) Handle issues found in static code analysis
[ https://issues.apache.org/jira/browse/CALCITE-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3236. - Resolution: Fixed Fix Version/s: 1.21.0 Fixed in [5c01713|https://github.com/apache/calcite/commit/5c017135b6b923abbf1b465ac1962bcec7886dfd], thanks for your PR, [~donnyzone] ! > Handle issues found in static code analysis > > > Key: CALCITE-3236 > URL: https://issues.apache.org/jira/browse/CALCITE-3236 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Feng Zhu >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Attachments: findbugs.png > > Time Spent: 40m > Remaining Estimate: 0h > > Recently, we are conducting static code analysis to make our project robust. > In Calcite, many potential risks can also be found with the tool FindBugs, as > shown in the figure below. > !findbugs.png! > However, it is difficult to trigger most of them. Current now, we fix two > obvious issues. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[ https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907122#comment-16907122 ] Stamatis Zampetakis commented on CALCITE-2302: -- Hey [~hyuan], it seems that you reviewed the PR; is it ready to go? Will it make it for 1.21.0? > Implicit type cast support > -- > > Key: CALCITE-2302 > URL: https://issues.apache.org/jira/browse/CALCITE-2302 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.17.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive. > Implicit type cast is an useful function for many cases, So we should support > this. > I checkout Calcite code and found that: > # Now we use a validator to validate our operands types[ through kinds of > namespaces and scopes ] > # Most of the validations will finally goes to > {code:java} > SqlOperator.validateOperands > {code} > # which will use validation logic defined in corresponding > SqlOperandTypeChecker > What i'm confused about is where should i put the implicit type cast logic > in? I figured out 2 ways: > # Supply a tool class/rules to add casts into a parsed SqlNode tree which > will then go through the validation logic later on. > # Unleash the validation logic in kinds of SqlOperandTypeChecker, then > modify the RelNode/RexNodes tree converted from a validated SqlNode tree to > add in casts through custom RelOptRules. > So guys, which of the 2 ways should i go, or if there are better way to do > this? > I need your help. > > Updated 18-05-30: > Hi guys, i have made a PR in > [CALCITE-2302|https://github.com/apache/calcite/pull/706] > This is design doc: [Calcite Implicit Type Cast > Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing]. > This is the conversion types mapping: [Conversion Types > Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing]. > I really appreciate your suggestions, thx. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (CALCITE-2302) Implicit type cast support
[ https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907122#comment-16907122 ] Stamatis Zampetakis edited comment on CALCITE-2302 at 8/14/19 10:05 AM: Hey [~hyuan], it seems that you reviewed the PR; is it ready to go (minus the conflicts that can be resolved easily)? Will it make it for 1.21.0? was (Author: zabetak): Hey [~hyuan], it seems that you reviewed the PR; is it ready to go? Will it make it for 1.21.0? > Implicit type cast support > -- > > Key: CALCITE-2302 > URL: https://issues.apache.org/jira/browse/CALCITE-2302 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.17.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive. > Implicit type cast is an useful function for many cases, So we should support > this. > I checkout Calcite code and found that: > # Now we use a validator to validate our operands types[ through kinds of > namespaces and scopes ] > # Most of the validations will finally goes to > {code:java} > SqlOperator.validateOperands > {code} > # which will use validation logic defined in corresponding > SqlOperandTypeChecker > What i'm confused about is where should i put the implicit type cast logic > in? I figured out 2 ways: > # Supply a tool class/rules to add casts into a parsed SqlNode tree which > will then go through the validation logic later on. > # Unleash the validation logic in kinds of SqlOperandTypeChecker, then > modify the RelNode/RexNodes tree converted from a validated SqlNode tree to > add in casts through custom RelOptRules. > So guys, which of the 2 ways should i go, or if there are better way to do > this? > I need your help. > > Updated 18-05-30: > Hi guys, i have made a PR in > [CALCITE-2302|https://github.com/apache/calcite/pull/706] > This is design doc: [Calcite Implicit Type Cast > Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing]. > This is the conversion types mapping: [Conversion Types > Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing]. > I really appreciate your suggestions, thx. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated CALCITE-3115: - Fix Version/s: 1.21.0 > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Closed] (CALCITE-3242) RelToSqlConverter handling "NOT IN" incorrectly
[ https://issues.apache.org/jira/browse/CALCITE-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Atharv Rajput closed CALCITE-3242. -- Resolution: Not A Bug Thanks [~danny0405] & [~julianhyde] for quick help. I'm closing this issue with "Not a bug" status. > RelToSqlConverter handling "NOT IN" incorrectly > --- > > Key: CALCITE-3242 > URL: https://issues.apache.org/jira/browse/CALCITE-3242 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Atharv Rajput >Assignee: Atharv Rajput >Priority: Minor > Fix For: 1.21.0 > > Original Estimate: 1h > Remaining Estimate: 1h > > When RelNode is having `NOT IN` operator, it's being converted incorrectly by > RelToSqlConverter. For example > {code:java} > @Test public void testNotInOperator() { > final RelBuilder builder = relBuilder().scan("EMP"); > final RexNode condition = > builder.call(SqlStdOperatorTable.NOT_IN, builder.field("DEPTNO"), > builder.literal(20), builder.literal(30)); > final RelNode root = relBuilder().scan("EMP").filter(condition).build(); > final String sql = toSql(root); > final String expectedSql = "SELECT *\n" > + "FROM \"scott\".\"EMP\"\n" > + "WHERE \"DEPTNO\" NOT IN (20, 30)"; > assertThat(sql, isLinux(expectedSql)); > } > {code} > Above test fails with error: > {code:java} > Expected: is "SELECT *\nFROM \"scott\".\"EMP\"\nWHERE \"DEPTNO\" NOT IN (20, > 30)" > but: was "SELECT *\nFROM \"scott\".\"EMP\"\nWHERE \"DEPTNO\" NOT IN 20 NOT IN > 30" > {code} > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907165#comment-16907165 ] Volodymyr Vysotskyi commented on CALCITE-3115: -- Other projects including Drill also faced with this issue. It would be good to include this fix into 1.21. [~winipanda], are you planning to complete the fix before the release? In the opposite case, I can pick up this issue. > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3244) RelDecorrelator unable to decorrelate expression with filter and aggregate on top
[ https://issues.apache.org/jira/browse/CALCITE-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907182#comment-16907182 ] Volodymyr Vysotskyi commented on CALCITE-3244: -- [~julianhyde], your assumption about DAG helped to discover one more issue in {{RelDecorrelator}}. The following test (added {{EXISTS}} clause) {code:java} @Test public void testDecorrelateExists() throws Exception { final String sql = "select \n" + " exists (select count(t2.id) \n" + "from (values(1), (2)) t2(id) where t2.id = t1.id)\n" + "from (values(3), (4)) t1(id)"; checkSubQuery(sql).withLateDecorrelation(true).check(); } {code} fails with the error: {noformat} java.lang.AssertionError: Cannot add expression of different type to set: set type is RecordType(INTEGER NOT NULL ID, BOOLEAN i) NOT NULL expression type is RecordType(INTEGER NOT NULL ID, BOOLEAN NOT NULL i) NOT NULL set is rel#44:LogicalCorrelate(left=HepRelVertex#36,right=HepRelVertex#43,correlation=$cor0,joinType=left,requiredColumns={0}) expression is LogicalProject(ID=[$0], i=[true]) LogicalCorrelate(correlation=[$cor0], joinType=[left], requiredColumns=[{0}]) LogicalValues(tuples=[[{ 3 }, { 4 }]]) LogicalAggregate(group=[{}], EXPR$0=[COUNT()]) LogicalFilter(condition=[=($0, $cor0.ID)]) LogicalValues(tuples=[[{ 1 }, { 2 }]]) at org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:384) at org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57) at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:236) at org.apache.calcite.sql2rel.RelDecorrelator$AdjustProjectForCountAggregateRule.onMatch2(RelDecorrelator.java:2538) at org.apache.calcite.sql2rel.RelDecorrelator$AdjustProjectForCountAggregateRule.onMatch(RelDecorrelator.java:2451) at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319) at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560) at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:419) at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256) at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127) at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215) at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202) at org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:249) at org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214) at org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:213) at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:338) at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:321) at org.apache.calcite.test.RelOptRulesTest.testDecorrelateExists(RelOptRulesTest.java:5565) {noformat} Regarding the original query, the plan is not changed. > RelDecorrelator unable to decorrelate expression with filter and aggregate on > top > - > > Key: CALCITE-3244 > URL: https://issues.apache.org/jira/browse/CALCITE-3244 > Project: Calcite > Issue Type: Improvement >Affects Versions: 1.20.0 >Reporter: benj >Assignee: Danny Chan >Priority: Major > > Some very useful type of requests currently failed with: > {code:java} > SYSTEM ERROR: UnsupportedOperationException: Adding Implicit RowID column is > not supported for ValuesPrel operator > {code} > Examples from DRILL-7050: > {code:sql} > select t1.id, > (select count(t2.id) > from ( > select 1 as id > union all > select 2 as id > ) t2 where t2.id = t1.id) > from ( > select 1 as id > union all > select 2 as id > ) t1 > {code} > {code:sql} > SELECT t, > (SELECT count(*) FROM > (SELECT split(r,' ') AS r FROM > (SELECT sub.t AS r)) AS x > ,LATERAL(SELECT $unnest AS u FROM unnest(x.r)) > /* WHERE ... */) t2 > FROM > (SELECT 'unnest is useful' AS t) sub > {code} > > _Please note that in 1.18 the error for these requests was:_ > {code:java} > Error: PLAN ERROR: Cannot convert RexNode to equivalent Drill expression. > RexNode Class: org.apache.calcite.rex.RexCorrelVariable, RexNode Digest: $cor0 > {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[ https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907185#comment-16907185 ] Stamatis Zampetakis commented on CALCITE-3115: -- I agree that we should try to include this in 1.21.0; it does not seem too complicated to finish it. I think we should retain Convention classes intact and add some kind of normalization to the rules name without adding random stuff. [~vvysotskyi] given that [~winipanda] didn't reply earlier to your comment in the PR I think it is safe to assume that he doesn't have time to finish it. I guess you can take over if you want but don't forget to give appropriate credit for his work. > Cannot add JdbcRules which have different JdbcConvention to same > VolcanoPlanner's RuleSet. > -- > > Key: CALCITE-3115 > URL: https://issues.apache.org/jira/browse/CALCITE-3115 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: TANG Wen-hui >Assignee: TANG Wen-hui >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > When we use Calcite via JDBC to run a sql which involves two difference jdbc > schema: > {code:java} > select * from (select "a",max("b") as max_b, sum("c") as sum_c from > "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having > "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from > "test2"."temp2" group by "a","b","c" > {code} > the sql get a plan like that: > {code:java} > EnumerableUnion(all=[false]) > JdbcToEnumerableConverter > JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4]) > JdbcSort(fetch=[10]) > JdbcFilter(condition=[<(CAST($3):BIGINT, 20)]) > JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)]) > JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0, > 100))]) > JdbcTableScan(table=[[test, temp]]) > EnumerableAggregate(group=[{0, 1, 2}]) > JdbcToEnumerableConverter > JdbcTableScan(table=[[test2, temp2]]) > {code} > And the EnumerableAggregate for table test2.temp2 cannot be converted to > JdbcAggregate. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3219) Allow finer control over which expressions get expanded in SqlToRelConverter
[ https://issues.apache.org/jira/browse/CALCITE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3219: Labels: pull-request-available (was: ) > Allow finer control over which expressions get expanded in SqlToRelConverter > > > Key: CALCITE-3219 > URL: https://issues.apache.org/jira/browse/CALCITE-3219 > Project: Calcite > Issue Type: Improvement >Reporter: Pressenna >Priority: Major > Labels: pull-request-available > > Currently one can control if sub-query expansion happens via > SqlToRelConverter.Config.isExpand(). This flag is global to the query and > causes expansion of all supported expression types. For some systems it is > required that one can dynamically narrow the set of expressions that should > be expanded. > In order to allow this, I would propose a new setter with a default > implementation that returns the above flag. > {code} > // a BiPredicate in order to pass both the original query and the expression > in question for expansion > java.util.function.BiPredicate > SqlToRelConverter.Config.getExpandPredicate() > {code} > If this is OK, I would provide a PR with the relevant changes. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3219) Allow finer control over which expressions get expanded in SqlToRelConverter
[ https://issues.apache.org/jira/browse/CALCITE-3219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907211#comment-16907211 ] Pressenna commented on CALCITE-3219: created https://github.com/apache/calcite/pull/1378 for the changes, which I would use as a discussion basis. > Allow finer control over which expressions get expanded in SqlToRelConverter > > > Key: CALCITE-3219 > URL: https://issues.apache.org/jira/browse/CALCITE-3219 > Project: Calcite > Issue Type: Improvement >Reporter: Pressenna >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently one can control if sub-query expansion happens via > SqlToRelConverter.Config.isExpand(). This flag is global to the query and > causes expansion of all supported expression types. For some systems it is > required that one can dynamically narrow the set of expressions that should > be expanded. > In order to allow this, I would propose a new setter with a default > implementation that returns the above flag. > {code} > // a BiPredicate in order to pass both the original query and the expression > in question for expansion > java.util.function.BiPredicate > SqlToRelConverter.Config.getExpandPredicate() > {code} > If this is OK, I would provide a PR with the relevant changes. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3245) CompileException in Janino when a query contains a division between a Double and a BigDecimal
[ https://issues.apache.org/jira/browse/CALCITE-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907231#comment-16907231 ] Feng Zhu commented on CALCITE-3245: --- Hi, [~zabetak]. Could you post your modification on CATCHALL? I try to reproduce it but can't find the _bigDecimal_ column in it. > CompileException in Janino when a query contains a division between a Double > and a BigDecimal > - > > Key: CALCITE-3245 > URL: https://issues.apache.org/jira/browse/CALCITE-3245 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Stamatis Zampetakis >Priority: Major > > A CompileException in Janino is raised when the SQL query contains a division > operator between a column of type Double and a column of type BigDecimal. > The following test added in {{ReflectiveSchemaTest}} can reproduce the > problem. > {code:java} > @Test public void testDivideDoubleBigDecimal() { > final CalciteAssert.AssertThat with = > CalciteAssert.that().withSchema("s", CATCHALL); > with.query("select \"wrapperDouble\" / \"bigDecimal\" as c\n" > + " from \"s\".\"everyTypes\"") > .runs(); > } > {code} > The generated code is the following: > {code:java} > public org.apache.calcite.linq4j.Enumerable bind(final > org.apache.calcite.DataContext root) { > final org.apache.calcite.linq4j.Enumerable _inputEnumerable = > org.apache.calcite.linq4j.Linq4j.asEnumerable(((org.apache.calcite.test.ReflectiveSchemaTest.CatchallSchema) > ((org.apache.calcite.adapter.java.ReflectiveSchema) > root.getRootSchema().getSubSchema("s").unwrap(org.apache.calcite.adapter.java.ReflectiveSchema.class)).getTarget()).everyTypes); > return new org.apache.calcite.linq4j.AbstractEnumerable(){ > public org.apache.calcite.linq4j.Enumerator enumerator() { > return new org.apache.calcite.linq4j.Enumerator(){ > public final org.apache.calcite.linq4j.Enumerator inputEnumerator > = _inputEnumerable.enumerator(); > public void reset() { > inputEnumerator.reset(); > } > public boolean moveNext() { > return inputEnumerator.moveNext(); > } > public void close() { > inputEnumerator.close(); > } > public Object current() { > final org.apache.calcite.test.ReflectiveSchemaTest.EveryType > current = (org.apache.calcite.test.ReflectiveSchemaTest.EveryType) > inputEnumerator.current(); > final Double inp15_ = current.wrapperDouble; > final java.math.BigDecimal inp21_ = current.bigDecimal; > return inp15_ == null || inp21_ == null ? > (java.math.BigDecimal) null : (java.math.BigDecimal) (inp15_.doubleValue() / > inp21_.doubleValue()); > } > }; > } > }; > } > public Class getElementType() { > return java.math.BigDecimal.class; > } > {code} > and the stack trace is given below: > {noformat} > Caused by: java.lang.RuntimeException: Error while compiling generated Java > code: > ... > at org.apache.calcite.avatica.Helper.wrap(Helper.java:37) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1110) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156) > ... 28 more > Caused by: org.codehaus.commons.compiler.CompileException: Line 23, Column > 111: Cannot cast "double" to "java.math.BigDecimal" > at > org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) > at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5051) > at org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:215) > at org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4418) >
[jira] [Commented] (CALCITE-3236) Handle issues found in static code analysis
[ https://issues.apache.org/jira/browse/CALCITE-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907235#comment-16907235 ] Feng Zhu commented on CALCITE-3236: --- Hi, [~krisden], thank for your comments and advice. Maybe we can have a try. > Handle issues found in static code analysis > > > Key: CALCITE-3236 > URL: https://issues.apache.org/jira/browse/CALCITE-3236 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Feng Zhu >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Attachments: findbugs.png > > Time Spent: 40m > Remaining Estimate: 0h > > Recently, we are conducting static code analysis to make our project robust. > In Calcite, many potential risks can also be found with the tool FindBugs, as > shown in the figure below. > !findbugs.png! > However, it is difficult to trigger most of them. Current now, we fix two > obvious issues. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3236) Handle issues found in static code analysis
[ https://issues.apache.org/jira/browse/CALCITE-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907269#comment-16907269 ] Danny Chan commented on CALCITE-3236: - [~krisden] Does the plugin have performance issue ? > Handle issues found in static code analysis > > > Key: CALCITE-3236 > URL: https://issues.apache.org/jira/browse/CALCITE-3236 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Feng Zhu >Assignee: Feng Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Attachments: findbugs.png > > Time Spent: 40m > Remaining Estimate: 0h > > Recently, we are conducting static code analysis to make our project robust. > In Calcite, many potential risks can also be found with the tool FindBugs, as > shown in the figure below. > !findbugs.png! > However, it is difficult to trigger most of them. Current now, we fix two > obvious issues. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3243) Incomplete validation of operands in JSON functions
[ https://issues.apache.org/jira/browse/CALCITE-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907275#comment-16907275 ] Matt Wang commented on CALCITE-3243: [~zabetak] I have the same error with the latest code, I used an old version(0.19) to run before. It's my mistake, I will pay attention to this. > Incomplete validation of operands in JSON functions > - > > Key: CALCITE-3243 > URL: https://issues.apache.org/jira/browse/CALCITE-3243 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Stamatis Zampetakis >Priority: Major > > The operands of various JSON functions are not validated correctly. > Consider for instance the {{JSON_VALUE}} function and the following calls: > {code:sql} > json_value('{\"foo\":100}', 'strict $.foo')" > json_value(1, 'strict $.foo')" > json_value(TRUE, 'strict $.foo')" > {code} > The first call is legal but the next are not; if I am not wrong the operands > should be always CHAR or VARCHAR literals which is not the case above. > Queries involving such calls fail at runtime when compiling generated code. I > guess that such kind of problems should be captured by the validator and we > should not even arrive at the code generation step. > The problems can be easily reproduced by adding such calls in the > {{SqlOperatorBaseTest}}. > The problem does not only affect {{JSON_VALUE}} but other JSON functions > (such as {{JSON_QUERY}} etc.) as well. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3245) CompileException in Janino when a query contains a division between a Double and a BigDecimal
[ https://issues.apache.org/jira/browse/CALCITE-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907282#comment-16907282 ] Stamatis Zampetakis commented on CALCITE-3245: -- Hey [~donnyzone], the complete test is already added in [https://github.com/apache/calcite/pull/1375] as I am in the middle of solving another problem. > CompileException in Janino when a query contains a division between a Double > and a BigDecimal > - > > Key: CALCITE-3245 > URL: https://issues.apache.org/jira/browse/CALCITE-3245 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Stamatis Zampetakis >Priority: Major > > A CompileException in Janino is raised when the SQL query contains a division > operator between a column of type Double and a column of type BigDecimal. > The following test added in {{ReflectiveSchemaTest}} can reproduce the > problem. > {code:java} > @Test public void testDivideDoubleBigDecimal() { > final CalciteAssert.AssertThat with = > CalciteAssert.that().withSchema("s", CATCHALL); > with.query("select \"wrapperDouble\" / \"bigDecimal\" as c\n" > + " from \"s\".\"everyTypes\"") > .runs(); > } > {code} > The generated code is the following: > {code:java} > public org.apache.calcite.linq4j.Enumerable bind(final > org.apache.calcite.DataContext root) { > final org.apache.calcite.linq4j.Enumerable _inputEnumerable = > org.apache.calcite.linq4j.Linq4j.asEnumerable(((org.apache.calcite.test.ReflectiveSchemaTest.CatchallSchema) > ((org.apache.calcite.adapter.java.ReflectiveSchema) > root.getRootSchema().getSubSchema("s").unwrap(org.apache.calcite.adapter.java.ReflectiveSchema.class)).getTarget()).everyTypes); > return new org.apache.calcite.linq4j.AbstractEnumerable(){ > public org.apache.calcite.linq4j.Enumerator enumerator() { > return new org.apache.calcite.linq4j.Enumerator(){ > public final org.apache.calcite.linq4j.Enumerator inputEnumerator > = _inputEnumerable.enumerator(); > public void reset() { > inputEnumerator.reset(); > } > public boolean moveNext() { > return inputEnumerator.moveNext(); > } > public void close() { > inputEnumerator.close(); > } > public Object current() { > final org.apache.calcite.test.ReflectiveSchemaTest.EveryType > current = (org.apache.calcite.test.ReflectiveSchemaTest.EveryType) > inputEnumerator.current(); > final Double inp15_ = current.wrapperDouble; > final java.math.BigDecimal inp21_ = current.bigDecimal; > return inp15_ == null || inp21_ == null ? > (java.math.BigDecimal) null : (java.math.BigDecimal) (inp15_.doubleValue() / > inp21_.doubleValue()); > } > }; > } > }; > } > public Class getElementType() { > return java.math.BigDecimal.class; > } > {code} > and the stack trace is given below: > {noformat} > Caused by: java.lang.RuntimeException: Error while compiling generated Java > code: > ... > at org.apache.calcite.avatica.Helper.wrap(Helper.java:37) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1110) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156) > ... 28 more > Caused by: org.codehaus.commons.compiler.CompileException: Line 23, Column > 111: Cannot cast "double" to "java.math.BigDecimal" > at > org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) > at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5051) > at org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:215) > at org.codehaus.janino.UnitCompiler$
[jira] [Created] (CALCITE-3251) BinaryExpression evaluate method support Long type.
xzh_dz created CALCITE-3251: --- Summary: BinaryExpression evaluate method support Long type. Key: CALCITE-3251 URL: https://issues.apache.org/jira/browse/CALCITE-3251 Project: Calcite Issue Type: Wish Reporter: xzh_dz -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3138) Restructuring ROW type fields is not supported
[ https://issues.apache.org/jira/browse/CALCITE-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Vysotskyi updated CALCITE-3138: - Fix Version/s: 1.21.0 > Restructuring ROW type fields is not supported > -- > > Key: CALCITE-3138 > URL: https://issues.apache.org/jira/browse/CALCITE-3138 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Haisheng Yuan >Assignee: Igor Guzenko >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Attachments: ROW_repro.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > 1) RelStructuredTypeFlattener.restructureFields(structType) doesn't support > ROW type. However, ROW type is flattened by > RelStructuredTypeFlattener just like struct. So when user queries one column > with complex type ROW, after flattening and restructuring > top level project returns reference to first inner primitive field of the > requested column. > 2) Another finding is related to ITEM expression applied to array of structs > column. For example, let's imagine a table with column of > type ARRAY>>. When user requests is SQL > array_column[1], Calcite generates ITEM($0, 1) , where $0 is ref to array > column > from Scan and 1 is index literal. Current flattener generates two field acess > expressions ITEM($0, 1).a, ITEM($0, 1).b but dont take into account > that ITEM($0, 1).b returns struct which also should be flattened. > 3) In some cases applying of ITEM after flattenning is no longer possible. > For example, consider column with type > STRUCT>. User requests column['b'] in query and > Calcite creates ITEM($0,'b'). > After flattenning Scan is covered by LogicalProject($0.a, $0.b.x, $0.b.y) and > the old projection ITEM($0,'b') can't > be applied anymore. So now it should be converted to refer only subset of > fields ($1,$2) from flattening project. > UPDATES IN EXPECTED TEST RESULTS: > -- > TEST CASE: SqlToRelConverterTest.testNestedColumnType() > {code} > select empa.home_address.zip from sales.emp_address empa where > empa.home_address.city = 'abc' > {code} > OLD RESULT: > {code} > LogicalProject(ZIP=[$4]) > LogicalFilter(condition=[=($3, 'abc':VARCHAR(20))]) > LogicalProject(EMPNO=[$0], STREET=[$1.STREET], CITY=[$1.CITY], > ZIP=[$1.ZIP], STATE=[$1.STATE], STREET5=[$2.STREET], CITY6=[$2.CITY], > ZIP7=[$2.ZIP], STATE8=[$2.STATE]) > LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]]) > {code} > 1. Above in logical filter, condition references to field $3 which is > ZIP=[$1.ZIP] field from previous project, >however in original query filtering should be done by CITY field. > 2. Also the top level project references to $4 field, which is > STATE=[$1.STATE] field from project, but original >query requested ZIP field. > > UPDATED RESULT: > {code} > LogicalProject(ZIP=[$3]) > LogicalFilter(condition=[=($2, 'abc')]) > LogicalProject(EMPNO=[$0], STREET=[$1.STREET], CITY=[$1.CITY], > ZIP=[$1.ZIP], STATE=[$1.STATE], STREET5=[$2.STREET], CITY6=[$2.CITY], > ZIP7=[$2.ZIP], STATE8=[$2.STATE]) > LogicalTableScan(table=[[CATALOG, SALES, EMP_ADDRESS]]) > {code} > -- > TEST CASE: SqlToRelConverterTest.testStructTypeAlias() > {code} > select t.r AS myRow from (select row(row(1)) r from dept) t > {code} > OLD RESULT: > {code} > LogicalProject(MYROW$$0$$0=[1]) > LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) > {code} > 1. Inside the subselect of row(row(1)) type of returned column is > RecordType(RecordType(INTEGER EXPR$0) EXPR$0), >but the top level project uses flattened expression and returns to user > literal 1 with type RecordType(INTEGER MYROW$$0$$0), >although the type doesn't match type returned by row(row(1)) expression. > 2. Also it's suspicious that caller expects returned column to have name > 'myRow' but gets 'MYROW$$0$$0'. > > UPDATED RESULT: > {code} > LogicalProject(MYROW=[ROW(ROW($0))]) > LogicalProject(MYROW$$0$$0=[1]) > LogicalTableScan(table=[[CATALOG, SALES, DEPT]]) > {code} > -- > TEST CASE: SqlToRelConverterTest.testFlattenRecords() > {code} > select employees[1] from dept_nested > {code} > OLD RESULT: > {code} > LogicalProject(EXPR$0=[$0]) > LogicalProject(EXPR$0$0=[ITEM($3, 1).EMPNO], EXPR$0$1=[ITEM($3, 1).ENAME], > EXPR$0$2=[ITEM($3, 1).DETAIL]) > LogicalTableScan(table=[[CATALOG, SALES, DEPT_NESTED]]) > {code} > 1. Given selection of element by index from array, which by definition > contain elements of type: > {code} > STRUCTENAME:VARCHA
[jira] [Assigned] (CALCITE-3243) Incomplete validation of operands in JSON functions
[ https://issues.apache.org/jira/browse/CALCITE-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Wang reassigned CALCITE-3243: -- Assignee: Matt Wang > Incomplete validation of operands in JSON functions > - > > Key: CALCITE-3243 > URL: https://issues.apache.org/jira/browse/CALCITE-3243 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Stamatis Zampetakis >Assignee: Matt Wang >Priority: Major > > The operands of various JSON functions are not validated correctly. > Consider for instance the {{JSON_VALUE}} function and the following calls: > {code:sql} > json_value('{\"foo\":100}', 'strict $.foo')" > json_value(1, 'strict $.foo')" > json_value(TRUE, 'strict $.foo')" > {code} > The first call is legal but the next are not; if I am not wrong the operands > should be always CHAR or VARCHAR literals which is not the case above. > Queries involving such calls fail at runtime when compiling generated code. I > guess that such kind of problems should be captured by the validator and we > should not even arrive at the code generation step. > The problems can be easily reproduced by adding such calls in the > {{SqlOperatorBaseTest}}. > The problem does not only affect {{JSON_VALUE}} but other JSON functions > (such as {{JSON_QUERY}} etc.) as well. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (CALCITE-3252) Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions on Redshift
Lindsey Meyer created CALCITE-3252: -- Summary: Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions on Redshift Key: CALCITE-3252 URL: https://issues.apache.org/jira/browse/CALCITE-3252 Project: Calcite Issue Type: Bug Reporter: Lindsey Meyer Example queries that error for these functions: {code:java} SELECT CONVERT_TIMEZONE('UTC', 'America/Los_Angeles', CAST('2019-01-01 01:00:00' AS TIMESTAMP) FROM mytable {code} {code:java} SELECT TO_DATE('2019-01-01', '-MM-DD') FROM mytable {code} {code:java} SELECT TO_TIMESTAMP('2019-01-01 01:00:00', '-MM-DD HH:MM:SS') FROM mytable {code} With errors like: {code:java} No match found for function signature CONVERT_TIMEZONE(, , ) {code} These are valid in Redshift and Postgres, except for `CONVERT_TIMEZONE`, which I believe is only valid on Redshift. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3252) Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions on Redshift
[ https://issues.apache.org/jira/browse/CALCITE-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lindsey Meyer updated CALCITE-3252: --- Description: Example queries that error for these functions: {code:java} SELECT CONVERT_TIMEZONE('UTC', 'America/Los_Angeles', CAST('2019-01-01 01:00:00' AS TIMESTAMP) FROM mytable {code} {code:java} SELECT TO_DATE('2019-01-01', '-MM-DD') FROM mytable {code} {code:java} SELECT TO_TIMESTAMP('2019-01-01 01:00:00', '-MM-DD HH:MM:SS') FROM mytable {code} With errors like: {code:java} No match found for function signature CONVERT_TIMEZONE(, , ) {code} These are valid in Redshift and Postgres, except for CONVERT_TIMEZONE, which I believe is only valid on Redshift. was: Example queries that error for these functions: {code:java} SELECT CONVERT_TIMEZONE('UTC', 'America/Los_Angeles', CAST('2019-01-01 01:00:00' AS TIMESTAMP) FROM mytable {code} {code:java} SELECT TO_DATE('2019-01-01', '-MM-DD') FROM mytable {code} {code:java} SELECT TO_TIMESTAMP('2019-01-01 01:00:00', '-MM-DD HH:MM:SS') FROM mytable {code} With errors like: {code:java} No match found for function signature CONVERT_TIMEZONE(, , ) {code} These are valid in Redshift and Postgres, except for `CONVERT_TIMEZONE`, which I believe is only valid on Redshift. > Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions > on Redshift > > > Key: CALCITE-3252 > URL: https://issues.apache.org/jira/browse/CALCITE-3252 > Project: Calcite > Issue Type: Bug >Reporter: Lindsey Meyer >Priority: Major > > Example queries that error for these functions: > {code:java} > SELECT CONVERT_TIMEZONE('UTC', 'America/Los_Angeles', CAST('2019-01-01 > 01:00:00' AS TIMESTAMP) FROM mytable {code} > {code:java} > SELECT TO_DATE('2019-01-01', '-MM-DD') FROM mytable {code} > {code:java} > SELECT TO_TIMESTAMP('2019-01-01 01:00:00', '-MM-DD HH:MM:SS') FROM > mytable > {code} > With errors like: > {code:java} > No match found for function signature CONVERT_TIMEZONE(, > , ) > {code} > These are valid in Redshift and Postgres, except for CONVERT_TIMEZONE, which > I believe is only valid on Redshift. > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3252) Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions on Redshift
[ https://issues.apache.org/jira/browse/CALCITE-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907516#comment-16907516 ] Lindsey Meyer commented on CALCITE-3252: I am working on a PR for this! > Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions > on Redshift > > > Key: CALCITE-3252 > URL: https://issues.apache.org/jira/browse/CALCITE-3252 > Project: Calcite > Issue Type: Bug >Reporter: Lindsey Meyer >Priority: Major > > Example queries that error for these functions: > {code:java} > SELECT CONVERT_TIMEZONE('UTC', 'America/Los_Angeles', CAST('2019-01-01 > 01:00:00' AS TIMESTAMP) FROM mytable {code} > {code:java} > SELECT TO_DATE('2019-01-01', '-MM-DD') FROM mytable {code} > {code:java} > SELECT TO_TIMESTAMP('2019-01-01 01:00:00', '-MM-DD HH:MM:SS') FROM > mytable > {code} > With errors like: > {code:java} > No match found for function signature CONVERT_TIMEZONE(, > , ) > {code} > These are valid in Redshift and Postgres, except for CONVERT_TIMEZONE, which > I believe is only valid on Redshift. > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (CALCITE-3235) Add CONCAT function for Redshift
[ https://issues.apache.org/jira/browse/CALCITE-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907521#comment-16907521 ] Ryan Fu commented on CALCITE-3235: -- PR: [https://github.com/apache/calcite/pull/1376] is ready for review > Add CONCAT function for Redshift > > > Key: CALCITE-3235 > URL: https://issues.apache.org/jira/browse/CALCITE-3235 > Project: Calcite > Issue Type: Improvement >Reporter: Ryan Fu >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Get this error: > {{No match found for function signature CONCAT(, , > ...)}} > When using CONCAT, e.g. > {{SELECT CONCAT('a', city) FROM public.aircraft}} > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3252) Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions on Redshift
[ https://issues.apache.org/jira/browse/CALCITE-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3252: Labels: pull-request-available (was: ) > Syntax error while parsing CONVERT_TIMEZONE, TO_DATE, TO_TIMESTAMP functions > on Redshift > > > Key: CALCITE-3252 > URL: https://issues.apache.org/jira/browse/CALCITE-3252 > Project: Calcite > Issue Type: Bug >Reporter: Lindsey Meyer >Priority: Major > Labels: pull-request-available > > Example queries that error for these functions: > {code:java} > SELECT CONVERT_TIMEZONE('UTC', 'America/Los_Angeles', CAST('2019-01-01 > 01:00:00' AS TIMESTAMP) FROM mytable {code} > {code:java} > SELECT TO_DATE('2019-01-01', '-MM-DD') FROM mytable {code} > {code:java} > SELECT TO_TIMESTAMP('2019-01-01 01:00:00', '-MM-DD HH:MM:SS') FROM > mytable > {code} > With errors like: > {code:java} > No match found for function signature CONVERT_TIMEZONE(, > , ) > {code} > These are valid in Redshift and Postgres, except for CONVERT_TIMEZONE, which > I believe is only valid on Redshift. > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3251: Labels: pull-request-available (was: ) > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (CALCITE-3229) UnsupportedOperationException for UPDATE with IN query
[ https://issues.apache.org/jira/browse/CALCITE-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chunwei Lei resolved CALCITE-3229. -- Resolution: Fixed Fix Version/s: 1.21.0 Fixed in [7625938f1faa37fe34540d3a7a2257ec7eae5704|7625938f1faa37fe34540d3a7a2257ec7eae5704]. > UnsupportedOperationException for UPDATE with IN query > -- > > Key: CALCITE-3229 > URL: https://issues.apache.org/jira/browse/CALCITE-3229 > Project: Calcite > Issue Type: Bug >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Query 2- "UPDATE tblspace1.table1 set n1=1000" + "WHERE k1 in (SELECT fk FROM > tblspace1.table2 WHERE k2=?)" java.lang.UnsupportedOperationException: class > org.apache.calcite.sql.SqlBasicCall: `K1` IN (SELECT `table2`.`fk` AS `FK` > FROM `tblspace1`.`table2` AS `TABLE2` WHERE `table2`.`k2` = ?) at > org.apache.calcite.util.Util.needToImplement(Util.java:967) at > org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType(SqlValidatorImpl.java:1579) > at > org.apache.calcite.sql2rel.SqlToRelConverter.findSubQueries(SqlToRelConverter.java:1802) > at > org.apache.calcite.sql2rel.SqlToRelConverter.findSubQueries(SqlToRelConverter.java:1776) > at > org.apache.calcite.sql2rel.SqlToRelConverter.replaceSubQueries(SqlToRelConverter.java:1011) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertUpdate(SqlToRelConverter.java:3570) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3172) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:563) > at org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:254) > > From the thread > [https://lists.apache.org/thread.html/67cb614ddd123a9092fdf37ace279eb563838b045a5554ad0005f030@%3Cdev.calcite.apache.org%3E] -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (CALCITE-3229) UnsupportedOperationException for UPDATE with IN query
[ https://issues.apache.org/jira/browse/CALCITE-3229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907746#comment-16907746 ] Chunwei Lei edited comment on CALCITE-3229 at 8/15/19 1:50 AM: --- Fixed in [7625938f1faa37fe34540d3a7a2257ec7eae5704|https://github.com/apache/calcite/commit/7625938f1faa37fe34540d3a7a2257ec7eae5704]. was (Author: chunwei lei): Fixed in [7625938f1faa37fe34540d3a7a2257ec7eae5704|7625938f1faa37fe34540d3a7a2257ec7eae5704]. > UnsupportedOperationException for UPDATE with IN query > -- > > Key: CALCITE-3229 > URL: https://issues.apache.org/jira/browse/CALCITE-3229 > Project: Calcite > Issue Type: Bug >Reporter: Chunwei Lei >Assignee: Chunwei Lei >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Query 2- "UPDATE tblspace1.table1 set n1=1000" + "WHERE k1 in (SELECT fk FROM > tblspace1.table2 WHERE k2=?)" java.lang.UnsupportedOperationException: class > org.apache.calcite.sql.SqlBasicCall: `K1` IN (SELECT `table2`.`fk` AS `FK` > FROM `tblspace1`.`table2` AS `TABLE2` WHERE `table2`.`k2` = ?) at > org.apache.calcite.util.Util.needToImplement(Util.java:967) at > org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType(SqlValidatorImpl.java:1579) > at > org.apache.calcite.sql2rel.SqlToRelConverter.findSubQueries(SqlToRelConverter.java:1802) > at > org.apache.calcite.sql2rel.SqlToRelConverter.findSubQueries(SqlToRelConverter.java:1776) > at > org.apache.calcite.sql2rel.SqlToRelConverter.replaceSubQueries(SqlToRelConverter.java:1011) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertUpdate(SqlToRelConverter.java:3570) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3172) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:563) > at org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:254) > > From the thread > [https://lists.apache.org/thread.html/67cb614ddd123a9092fdf37ace279eb563838b045a5554ad0005f030@%3Cdev.calcite.apache.org%3E] -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (CALCITE-3245) CompileException in Janino when a query contains a division between a Double and a BigDecimal
[ https://issues.apache.org/jira/browse/CALCITE-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Feng Zhu reassigned CALCITE-3245: - Assignee: Feng Zhu > CompileException in Janino when a query contains a division between a Double > and a BigDecimal > - > > Key: CALCITE-3245 > URL: https://issues.apache.org/jira/browse/CALCITE-3245 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Stamatis Zampetakis >Assignee: Feng Zhu >Priority: Major > > A CompileException in Janino is raised when the SQL query contains a division > operator between a column of type Double and a column of type BigDecimal. > The following test added in {{ReflectiveSchemaTest}} can reproduce the > problem. > {code:java} > @Test public void testDivideDoubleBigDecimal() { > final CalciteAssert.AssertThat with = > CalciteAssert.that().withSchema("s", CATCHALL); > with.query("select \"wrapperDouble\" / \"bigDecimal\" as c\n" > + " from \"s\".\"everyTypes\"") > .runs(); > } > {code} > The generated code is the following: > {code:java} > public org.apache.calcite.linq4j.Enumerable bind(final > org.apache.calcite.DataContext root) { > final org.apache.calcite.linq4j.Enumerable _inputEnumerable = > org.apache.calcite.linq4j.Linq4j.asEnumerable(((org.apache.calcite.test.ReflectiveSchemaTest.CatchallSchema) > ((org.apache.calcite.adapter.java.ReflectiveSchema) > root.getRootSchema().getSubSchema("s").unwrap(org.apache.calcite.adapter.java.ReflectiveSchema.class)).getTarget()).everyTypes); > return new org.apache.calcite.linq4j.AbstractEnumerable(){ > public org.apache.calcite.linq4j.Enumerator enumerator() { > return new org.apache.calcite.linq4j.Enumerator(){ > public final org.apache.calcite.linq4j.Enumerator inputEnumerator > = _inputEnumerable.enumerator(); > public void reset() { > inputEnumerator.reset(); > } > public boolean moveNext() { > return inputEnumerator.moveNext(); > } > public void close() { > inputEnumerator.close(); > } > public Object current() { > final org.apache.calcite.test.ReflectiveSchemaTest.EveryType > current = (org.apache.calcite.test.ReflectiveSchemaTest.EveryType) > inputEnumerator.current(); > final Double inp15_ = current.wrapperDouble; > final java.math.BigDecimal inp21_ = current.bigDecimal; > return inp15_ == null || inp21_ == null ? > (java.math.BigDecimal) null : (java.math.BigDecimal) (inp15_.doubleValue() / > inp21_.doubleValue()); > } > }; > } > }; > } > public Class getElementType() { > return java.math.BigDecimal.class; > } > {code} > and the stack trace is given below: > {noformat} > Caused by: java.lang.RuntimeException: Error while compiling generated Java > code: > ... > at org.apache.calcite.avatica.Helper.wrap(Helper.java:37) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1110) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156) > ... 28 more > Caused by: org.codehaus.commons.compiler.CompileException: Line 23, Column > 111: Cannot cast "double" to "java.math.BigDecimal" > at > org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) > at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5051) > at org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:215) > at org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4418) > at org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4396) > at org.codehaus.janino.Java$Cast.accept(Java.j
[jira] [Commented] (CALCITE-3245) CompileException in Janino when a query contains a division between a Double and a BigDecimal
[ https://issues.apache.org/jira/browse/CALCITE-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907769#comment-16907769 ] Feng Zhu commented on CALCITE-3245: --- Thanks [~zabetak], I'd like to solve this problem since it is also about codegen. > CompileException in Janino when a query contains a division between a Double > and a BigDecimal > - > > Key: CALCITE-3245 > URL: https://issues.apache.org/jira/browse/CALCITE-3245 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Stamatis Zampetakis >Priority: Major > > A CompileException in Janino is raised when the SQL query contains a division > operator between a column of type Double and a column of type BigDecimal. > The following test added in {{ReflectiveSchemaTest}} can reproduce the > problem. > {code:java} > @Test public void testDivideDoubleBigDecimal() { > final CalciteAssert.AssertThat with = > CalciteAssert.that().withSchema("s", CATCHALL); > with.query("select \"wrapperDouble\" / \"bigDecimal\" as c\n" > + " from \"s\".\"everyTypes\"") > .runs(); > } > {code} > The generated code is the following: > {code:java} > public org.apache.calcite.linq4j.Enumerable bind(final > org.apache.calcite.DataContext root) { > final org.apache.calcite.linq4j.Enumerable _inputEnumerable = > org.apache.calcite.linq4j.Linq4j.asEnumerable(((org.apache.calcite.test.ReflectiveSchemaTest.CatchallSchema) > ((org.apache.calcite.adapter.java.ReflectiveSchema) > root.getRootSchema().getSubSchema("s").unwrap(org.apache.calcite.adapter.java.ReflectiveSchema.class)).getTarget()).everyTypes); > return new org.apache.calcite.linq4j.AbstractEnumerable(){ > public org.apache.calcite.linq4j.Enumerator enumerator() { > return new org.apache.calcite.linq4j.Enumerator(){ > public final org.apache.calcite.linq4j.Enumerator inputEnumerator > = _inputEnumerable.enumerator(); > public void reset() { > inputEnumerator.reset(); > } > public boolean moveNext() { > return inputEnumerator.moveNext(); > } > public void close() { > inputEnumerator.close(); > } > public Object current() { > final org.apache.calcite.test.ReflectiveSchemaTest.EveryType > current = (org.apache.calcite.test.ReflectiveSchemaTest.EveryType) > inputEnumerator.current(); > final Double inp15_ = current.wrapperDouble; > final java.math.BigDecimal inp21_ = current.bigDecimal; > return inp15_ == null || inp21_ == null ? > (java.math.BigDecimal) null : (java.math.BigDecimal) (inp15_.doubleValue() / > inp21_.doubleValue()); > } > }; > } > }; > } > public Class getElementType() { > return java.math.BigDecimal.class; > } > {code} > and the stack trace is given below: > {noformat} > Caused by: java.lang.RuntimeException: Error while compiling generated Java > code: > ... > at org.apache.calcite.avatica.Helper.wrap(Helper.java:37) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1110) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:638) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:502) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:472) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156) > ... 28 more > Caused by: org.codehaus.commons.compiler.CompileException: Line 23, Column > 111: Cannot cast "double" to "java.math.BigDecimal" > at > org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) > at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5051) > at org.codehaus.janino.UnitCompiler.access$8600(UnitCompiler.java:215) > at org.codehaus.janino.UnitCompiler$16.visitCast(UnitCompiler.java:4418) > at org.codehaus.janino.UnitCompiler$16.visitCast
[jira] [Assigned] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing reassigned CALCITE-3251: - Assignee: jin xing > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Assignee: jin xing >Priority: Minor > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing reassigned CALCITE-3251: - Assignee: (was: jin xing) > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated CALCITE-3251: -- Issue Type: Bug (was: Wish) > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Bug >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.14#76016)