[jira] [Commented] (CALCITE-5881) Support to get foreign keys metadata in RelMetadataQuery

2023-08-03 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750925#comment-17750925
 ] 

Chunwei Lei commented on CALCITE-5881:
--

Sounds reasonable.

> Support to get foreign keys metadata in RelMetadataQuery
> 
>
> Key: CALCITE-5881
> URL: https://issues.apache.org/jira/browse/CALCITE-5881
> Project: Calcite
>  Issue Type: New Feature
>Reporter: JingDas
>Assignee: JingDas
>Priority: Major
>  Labels: pull-request-available
>
> We can get constraints by RelOptTable#getReferentialConstraints method, but 
> maybe can't get appropriate constraints at top relNode.
> For example:
> SQL:
> {code:java}
> SELECT DEPT.name, emp_agg.deptno, emp_agg.ename, DEPT.deptno
> FROM DEPT
> RIGHT JOIN
> (SELECT COUNT(sal), deptno, ename FROM EMP GROUP BY deptno, ename) emp_agg
> ON DEPT.deptno = emp_agg.deptno
> WHERE emp_agg.ename = 'job'{code}
> The relNode is:
> {code:java}
> LogicalProject(NAME=[$1], DEPTNO=[$3], ENAME=[$4], DEPTNO0=[$0])
>   LogicalFilter(condition=[=($4, 'job')])
>     LogicalJoin(condition=[=($0, $3)], joinType=[right])
>       LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
>       LogicalProject(EXPR$0=[$2], DEPTNO=[$0], ENAME=[$1])
>         LogicalAggregate(group=[{0, 1}], EXPR$0=[COUNT($2)])
>           LogicalProject(DEPTNO=[$7], ENAME=[$1], SAL=[$5])
>             LogicalTableScan(table=[[CATALOG, SALES, EMPNULLABLES]]){code}
> where the foreign key is the DEPTNO column of CATALOG.SALES.EMP table,
> reference the DEPTNO unique column of CATALOG.SALES.DEPT table.
> When we want to get foreign keys metadata on `LogicalJoin` or top 
> `LogicalProject`, There is no such method currently, it seems that we should 
> trace the field column origin to get the foreign key and corresponding unique 
> key.
>  
> The final result of this feature is something likely as following:
> When we want to get foreign keys metadata on `LogicalJoin`, the `LogicalJoin` 
> rowType is
> {code:java}
> RecordType(INTEGER DEPTNO, VARCHAR(10) NAME, BIGINT EXPR$0, INTEGER DEPTNO0, 
> VARCHAR(20) ENAME).{code}
> We expect the foreign keys metadata:
> {code:java}
> foreignColumns bitset is {3}
> uniqueColumns bitset is {0}{code}
> When we want to get foreign keys metadata on top `LogicalProject`, the 
> `LogicalProject` rowType is
> {code:java}
> RecordType(VARCHAR(10) NAME, INTEGER DEPTNO, VARCHAR(20) ENAME, INTEGER 
> DEPTNO0).{code}
> We expect the foreigns key metadata:
> {code:java}
> foreignColumns bitset is {1}
> uniqueColumns bitset is {3}{code}
> All the foreign or unique columns is 0 based index.
> Foreign keys metadata is very useful in many optimize scenes. Such as it can 
> be used in join eliminate when join type is inner join and some other star 
> schema query optimize.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (CALCITE-5861) Optimization rules do not constant-fold expressions in window bounds

2023-07-19 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744849#comment-17744849
 ] 

Chunwei Lei edited comment on CALCITE-5861 at 7/20/23 2:52 AM:
---

I tried the query mentioned in PostgreSQL and it works:

{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + 5 PRECEDING 
AND 1 PRECEDING) AS w_avg from emp;
 w_avg
---
(0 rows)
{code}

So I agree with your proposal.

Besides, the following sql cannot run in PostgreSQL:

{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno 
PRECEDING AND 1 PRECEDING) AS w_avg from emp;
ERROR:  argument of ROWS must not contain variables
LINE 1: ...t COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno PREC...
{code}


but it does not throw an exception in calcite. We should fix it too. WDYT?


was (Author: chunwei lei):
I tried the query mentioned in PostgreSQL and it works:

{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + 5 PRECEDING 
AND 1 PRECEDING) AS w_avg from emp;
 w_avg
---
(0 rows)
{code}

So I agree with your proposal.

Besides, the following sql cannot run in PostgreSQL:

{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno 
PRECEDING AND 1 PRECEDING) AS w_avg from emp;
ERROR:  argument of ROWS must not contain variables
LINE 1: ...t COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno PREC...
{code}


but it does not throw an exception in calcite.

> Optimization rules do not constant-fold expressions in window bounds
> 
>
> Key: CALCITE-5861
> URL: https://issues.apache.org/jira/browse/CALCITE-5861
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.34.0
>Reporter: Mihai Budiu
>Priority: Minor
>  Labels: pull-request-available
>
> This bug is a bit more complicated to describe, and there is a possibility 
> that I am doing something wrong.
> Consider the following test that can be added to RelOptRulesTest:
> {code:java}
> @Test void testExpressionPreceding() {
> HepProgramBuilder preBuilder = new HepProgramBuilder();
> preBuilder.addRuleInstance(CoreRules.WINDOW_REDUCE_EXPRESSIONS);
> 
> preBuilder.addRuleInstance(CoreRules.PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW);
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleInstance(CoreRules.PROJECT_REDUCE_EXPRESSIONS);
> HepPlanner hepPlanner = new HepPlanner(builder.build());
> final String sql =
> "select COUNT(*) over (\n"
> + "ORDER BY empno\n"
> + "ROWS BETWEEN 5 + 5 PRECEDING AND 1 PRECEDING) AS w_avg\n"
> + "  from emp\n";
> sql(sql)
> .withPre(preBuilder.build())
> .withPlanner(hepPlanner)
> .check();
>   }
> {code}
> The plan before looks like this:
> {code:java}
> LogicalProject($0=[$2])
>   LogicalWindow(window#0=[window(order by [0] rows between $1 PRECEDING and 
> $2 PRECEDING aggs [COUNT()])])
> LogicalProject(EMPNO=[$0], $1=[+(5, 5)])
>   LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The problem is that the expression "5+5", which appears in the window bounds, 
> has not been reduced to a constant by the rule WINDOW_REDUCE_EXPRESSIONS. 
> Moreover, the next optimization rule PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW 
> has pushed this expression into the LogicalProject. So it appears locally 
> that the LogicalWindow no longer has a constant bound, which is required by 
> the SQL language spec (it is constant, but that is no longer apparent in the 
> query). (At least our code generator is upset by this state of affairs.)
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-5861) Optimization rules do not constant-fold expressions in window bounds

2023-07-19 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744849#comment-17744849
 ] 

Chunwei Lei commented on CALCITE-5861:
--

I tried the query mentioned in PostgreSQL and it works:

{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + 5 PRECEDING 
AND 1 PRECEDING) AS w_avg from emp;
 w_avg
---
(0 rows)
{code}

So I agree with your proposal.

Besides, the following sql cannot run in PostgreSQL:

{code:sql}
postgres=# select COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno 
PRECEDING AND 1 PRECEDING) AS w_avg from emp;
ERROR:  argument of ROWS must not contain variables
LINE 1: ...t COUNT(*) over ( ORDER BY empno ROWS BETWEEN 5 + empno PREC...
{code}


but it does not throw an exception in calcite.

> Optimization rules do not constant-fold expressions in window bounds
> 
>
> Key: CALCITE-5861
> URL: https://issues.apache.org/jira/browse/CALCITE-5861
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.34.0
>Reporter: Mihai Budiu
>Priority: Minor
>  Labels: pull-request-available
>
> This bug is a bit more complicated to describe, and there is a possibility 
> that I am doing something wrong.
> Consider the following test that can be added to RelOptRulesTest:
> {code:java}
> @Test void testExpressionPreceding() {
> HepProgramBuilder preBuilder = new HepProgramBuilder();
> preBuilder.addRuleInstance(CoreRules.WINDOW_REDUCE_EXPRESSIONS);
> 
> preBuilder.addRuleInstance(CoreRules.PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW);
> HepProgramBuilder builder = new HepProgramBuilder();
> builder.addRuleInstance(CoreRules.PROJECT_REDUCE_EXPRESSIONS);
> HepPlanner hepPlanner = new HepPlanner(builder.build());
> final String sql =
> "select COUNT(*) over (\n"
> + "ORDER BY empno\n"
> + "ROWS BETWEEN 5 + 5 PRECEDING AND 1 PRECEDING) AS w_avg\n"
> + "  from emp\n";
> sql(sql)
> .withPre(preBuilder.build())
> .withPlanner(hepPlanner)
> .check();
>   }
> {code}
> The plan before looks like this:
> {code:java}
> LogicalProject($0=[$2])
>   LogicalWindow(window#0=[window(order by [0] rows between $1 PRECEDING and 
> $2 PRECEDING aggs [COUNT()])])
> LogicalProject(EMPNO=[$0], $1=[+(5, 5)])
>   LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> {code}
> The problem is that the expression "5+5", which appears in the window bounds, 
> has not been reduced to a constant by the rule WINDOW_REDUCE_EXPRESSIONS. 
> Moreover, the next optimization rule PROJECT_TO_LOGICAL_PROJECT_AND_WINDOW 
> has pushed this expression into the LogicalProject. So it appears locally 
> that the LogicalWindow no longer has a constant bound, which is required by 
> the SQL language spec (it is constant, but that is no longer apparent in the 
> query). (At least our code generator is upset by this state of affairs.)
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-5837) After RexUtil.pullFactors method,sometimes the condition order of RexNode is not the same as before

2023-07-12 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742626#comment-17742626
 ] 

Chunwei Lei commented on CALCITE-5837:
--

+1.

> After RexUtil.pullFactors method,sometimes the condition order of RexNode is 
> not the same as before
> ---
>
> Key: CALCITE-5837
> URL: https://issues.apache.org/jira/browse/CALCITE-5837
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.34.0
>Reporter: LakeShen
>Priority: Minor
> Fix For: 1.36.0
>
>
> In my rule, I used the RexUtil.pullFactors to simplify the Filter's condition 
> and I tested the rule with TPC-H q19.sql.
> When I print the plan tree with RelOptUtil.toString,sometimes the plan is :
> {code:java}
> LogicalAggregate(group=[{}], revenue=[SUM($0)])
>   LogicalProject($f0=[*($5, -(1, $6))])
> LogicalFilter(condition=[AND(=($16, $1), SEARCH($14, Sarg['AIR':CHAR(7), 
> 'AIR REG']:CHAR(7)), =($13, 'DELIVER IN PERSON'), OR(AND(=($19, 'Brand#12'), 
> SEARCH($22, Sarg['SM BOX':CHAR(7), 'SM CASE', 'SM PACK', 'SM 
> PKG':CHAR(7)]:CHAR(7)), >=($4, 1), <=($4, +(1, 10)), SEARCH($21, 
> Sarg[[1..5]])), AND(=($19, 'Brand#23'), SEARCH($22, Sarg['MED BAG':CHAR(8), 
> 'MED BOX':CHAR(8), 'MED PACK', 'MED PKG':CHAR(8)]:CHAR(8)), >=($4, 10), 
> <=($4, +(10, 10)), SEARCH($21, Sarg[[1..10]])), AND(=($19, 'Brand#34'), 
> SEARCH($22, Sarg['LG BOX':CHAR(7), 'LG CASE', 'LG PACK', 'LG 
> PKG':CHAR(7)]:CHAR(7)), >=($4, 20), <=($4, +(20, 10)), SEARCH($21, 
> Sarg[[1..15]]])
>   LogicalJoin(condition=[true], joinType=[inner])
> LogicalTableScan(table=[[tpch, LINEITEM]])
> LogicalTableScan(table=[[tpch, PART]])
> {code}
> and sometimes the plan is :
> {code:java}
> LogicalAggregate(group=[{}], revenue=[SUM($0)])
>   LogicalProject($f0=[*($5, -(1, $6))])
> LogicalFilter(condition=[AND(=($16, $1), =($13, 'DELIVER IN PERSON'), 
> SEARCH($14, Sarg['AIR':CHAR(7), 'AIR REG']:CHAR(7)), OR(AND(=($19, 
> 'Brand#12'), SEARCH($22, Sarg['SM BOX':CHAR(7), 'SM CASE', 'SM PACK', 'SM 
> PKG':CHAR(7)]:CHAR(7)), >=($4, 1), <=($4, +(1, 10)), SEARCH($21, 
> Sarg[[1..5]])), AND(=($19, 'Brand#23'), SEARCH($22, Sarg['MED BAG':CHAR(8), 
> 'MED BOX':CHAR(8), 'MED PACK', 'MED PKG':CHAR(8)]:CHAR(8)), >=($4, 10), 
> <=($4, +(10, 10)), SEARCH($21, Sarg[[1..10]])), AND(=($19, 'Brand#34'), 
> SEARCH($22, Sarg['LG BOX':CHAR(7), 'LG CASE', 'LG PACK', 'LG 
> PKG':CHAR(7)]:CHAR(7)), >=($4, 20), <=($4, +(20, 10)), SEARCH($21, 
> Sarg[[1..15]]])
>   LogicalJoin(condition=[true], joinType=[inner])
> LogicalTableScan(table=[[tpch, LINEITEM]])
> LogicalTableScan(table=[[tpch, PART]])
> {code}
> The above two SQL Plan semantics are the same, the only difference is that 
> the Filter conditions are in different order.
> Although this has no effect on the SQL execution results, it is difficult for 
> me to monitor my plan because of the variability of the plan.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-5842) LogicalProject deepHashCode creates same value with different RowType

2023-07-12 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17742622#comment-17742622
 ] 

Chunwei Lei commented on CALCITE-5842:
--

bq. The 2 logicalProjects from the 2 LogicalTableScans have same hashCode value 
based on the deepHashCode method in LogicalProject.

AFAIU, it means the exps and the input of 2  projects are same. We can infer 
that the RowType of 2 Projects is same too. So even we consider the RowType, it 
does not make a difference. Correct me if I am wrong.

> LogicalProject deepHashCode creates same value with different RowType 
> --
>
> Key: CALCITE-5842
> URL: https://issues.apache.org/jira/browse/CALCITE-5842
> Project: Calcite
>  Issue Type: Bug
>Affects Versions: 1.32.0
>Reporter: Yu Tian
>Priority: Major
>
> The LogicalProject class has deepEquals0 and deepHashCode0 methods, in the 
> deepEquals0 method, it consider getRowType() as one equal standard, however, 
> in the deepHashCode0, it is missing the getRowType() to generated the hash 
> value. Do we do this on purpose or it is a bug?
> [https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/rel/core/Project.java#L348,L368]
>  
>  
> The reason we ask is that we are trying 2 use cases from our side.
> The first one is two LogicalTableScan with similar configurations, which are 
> connected to 2 separate LogicalFiler, then we LogicalJoin these 2 together. 
> One issue we noticed is that, in HepPlanner, it has logics as below
>  
> package org.apache.calcite.plan.hep.HepPlanner
> {code:java}
> // try to find equivalent rel only if DAG is allowed
> if (!noDag) {
>   // Now, check if an equivalent vertex already exists in graph.
>   HepRelVertex equivVertex = mapDigestToVertex.get(rel.getRelDigest());
>   if (equivVertex != null) {
> // Use existing vertex.
> return equivVertex;
>   }
> } {code}
>  
> The 2 logicalProjects from the 2 LogicalTableScans have same hashCode value 
> based on the deepHashCode method in LogicalProject, because it didn’t 
> consider the getRowType() value, the planner is replacing LogicalTableScan2 
> with LogicalTableScan1, in fact, we should treat them as separate items to 
> process. 
>  
> Another use case we have, we have 2 diagrams, each diagram with 
> LogicalTableScan, LogicalFiler, LogicalTableModify, LogicalTableScan have 
> similar setup with different rowType information. This time, HepPlanner is 
> passing, since it has separate HepPlanner stage, so above issue is not 
> happening. However, when it reach the VolcanoPlanner, the logics
>  
> package org.apache.calcite.plan.volcano.VolcanoPlanner
> {code:java}
> // If it is equivalent to an existing expression, return the set that
> // the equivalent expression belongs to.
> RelDigest digest = rel.getRelDigest();
> RelNode equivExp = mapDigestToRel.get(digest); {code}
>  
> The map replace the LogicalTableScan1 with LogicalTableScan2 in the 
> LogicalProject stage since they have same hashCode, and the map is reusing 
> earlier processed RelNode, which caused the issues.
>  
> Here are the proposals we have,
>  
>  * Narrow Scope change: LogicalProject is the most frequently used project 
> type, we only change it.
>  ** Modify the LogicalProject method deepHashCode method to use 
> {code:java}
> @Override public int deepHashCode() {
>   return Objects.hash(traitSet, input.deepHashCode(), exps, hints, 
> getRowType());
> }{code}
> Consider the getRowType() value in the hash generation will resolve the 
> issue, since the rowType contains the field names and data types information. 
>  
>  * Whole Scope change: Change the deepHashCode method in Project class.
>  ** Similar change as above, however, the scope of this change is wide 
> compared to the first one.
>  
> Is it something we can consider to improve in the following release of Apache 
> Calcite?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-5639) RexSimplify should remove IS NOT NULL check when LIKE comparison is present

2023-04-11 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711168#comment-17711168
 ] 

Chunwei Lei commented on CALCITE-5639:
--

I can't agree more. It would be great if we can find a more general solution.

> RexSimplify should remove IS NOT NULL check when LIKE comparison is present
> ---
>
> Key: CALCITE-5639
> URL: https://issues.apache.org/jira/browse/CALCITE-5639
> Project: Calcite
>  Issue Type: Bug
>Affects Versions: 1.34.0
>Reporter: Leonid Chistov
>Assignee: Leonid Chistov
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Consider query like
> {code:java}
> select r_reason_desc from reason
> where r_reason_desc is not null and r_reason_desc like '%child%' 
> {code}
> where "r_reason_desc" is a nullable field.
> When RexSimplify::simplifyFilterPredicates is called on that conjunction of 
> expressions, expression is not simplified, meaning that redundant  "is not 
> null" check is not removed.
> In the same time, if query like 
> {code:java}
> select r_reason_desc from reason
> where r_reason_desc is not null and r_reason_desc >= 'A'{code}
> is passed to optimizer, redundant "is not null" check is eliminated.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule

2022-09-06 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690
 ] 

Chunwei Lei edited comment on CALCITE-5267 at 9/6/22 9:27 AM:
--

I checked the history. The rule is introduced from Druid[1], in which newCasts 
is used. It may be better to correct it instead of removing it.

[1] 
https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7


was (Author: chunwei lei):
I checked the history. The rule is introduced from Druid[1], in which {{ 
newCasts }} is used. It may be better to correct it instead of removing it.

[1] 
https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7

> Remove useless variable 'newCasts' in AggregateCaseToFilterRule
> ---
>
> Key: CALCITE-5267
> URL: https://issues.apache.org/jira/browse/CALCITE-5267
> Project: Calcite
>  Issue Type: Bug
>Reporter: Wenzhuang Zhu
>Assignee: Wenzhuang Zhu
>Priority: Minor
>  Labels: pull-request-available
>   Original Estimate: 24h
>  Time Spent: 10m
>  Remaining Estimate: 23h 50m
>
> AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has 
> never been used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule

2022-09-06 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690
 ] 

Chunwei Lei edited comment on CALCITE-5267 at 9/6/22 9:26 AM:
--

I checked the history. The rule is introduced from Druid[1], in which {{ 
newCasts }} is used. It may be better to correct it instead of removing it.

[1] 
https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7


was (Author: chunwei lei):
I checked the history. The rule is introduced from Druid[1], in which newCasts 
is used. It may be better to correct it instead of remove it.

[1]https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7

> Remove useless variable 'newCasts' in AggregateCaseToFilterRule
> ---
>
> Key: CALCITE-5267
> URL: https://issues.apache.org/jira/browse/CALCITE-5267
> Project: Calcite
>  Issue Type: Bug
>Reporter: Wenzhuang Zhu
>Assignee: Wenzhuang Zhu
>Priority: Minor
>  Labels: pull-request-available
>   Original Estimate: 24h
>  Time Spent: 10m
>  Remaining Estimate: 23h 50m
>
> AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has 
> never been used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule

2022-09-06 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690
 ] 

Chunwei Lei edited comment on CALCITE-5267 at 9/6/22 9:26 AM:
--

I checked the history. The rule is introduced from Druid[1], in which newCasts 
is used. It may be better to correct it instead of remove it.

[1]https://github.com/apache/druid/pull/4360/files#diff-809e3a73d1485dad83629598ea5bcbab8f5ce7724e9572975b052176c44fbdf7


was (Author: chunwei lei):
Makes sense.

> Remove useless variable 'newCasts' in AggregateCaseToFilterRule
> ---
>
> Key: CALCITE-5267
> URL: https://issues.apache.org/jira/browse/CALCITE-5267
> Project: Calcite
>  Issue Type: Bug
>Reporter: Wenzhuang Zhu
>Assignee: Wenzhuang Zhu
>Priority: Minor
>  Labels: pull-request-available
>   Original Estimate: 24h
>  Time Spent: 10m
>  Remaining Estimate: 23h 50m
>
> AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has 
> never been used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-5267) Remove useless variable 'newCasts' in AggregateCaseToFilterRule

2022-09-06 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600690#comment-17600690
 ] 

Chunwei Lei commented on CALCITE-5267:
--

Makes sense.

> Remove useless variable 'newCasts' in AggregateCaseToFilterRule
> ---
>
> Key: CALCITE-5267
> URL: https://issues.apache.org/jira/browse/CALCITE-5267
> Project: Calcite
>  Issue Type: Bug
>Reporter: Wenzhuang Zhu
>Assignee: Wenzhuang Zhu
>Priority: Minor
>  Labels: pull-request-available
>   Original Estimate: 24h
>  Time Spent: 10m
>  Remaining Estimate: 23h 50m
>
> AggregateCaseToFilterRule::onMatch contains a variable 'newCasts' that has 
> never been used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-5231) Unable to create Oracle JDBC connection in DBeaver

2022-08-09 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577257#comment-17577257
 ] 

Chunwei Lei commented on CALCITE-5231:
--

Hi, [~kartik.kud...@exate.com]. It's better to send an email to the dev mailing 
list if you need help.

> Unable to create Oracle JDBC connection in DBeaver
> --
>
> Key: CALCITE-5231
> URL: https://issues.apache.org/jira/browse/CALCITE-5231
> Project: Calcite
>  Issue Type: Bug
>  Components: core, jdbc-adapter
>Affects Versions: 1.22.0
>Reporter: kartik 
>Priority: Major
> Attachments: Calcite JDBC-DBeaver-connection-Issue.png, 
> JDBC-calcite-connection-Error-Message.png
>
>
> I am trying to connect my Oracle database using calcite in DBever. 
> Driver class used : org.apache.calcite.jdbc.Driver
> URL  - jdbc:oracle:thin:XXX:1521:
>  
> But when I use same same URL and driver  with main java program . it works 
> fine .
> Connection con= DriverManager.getConnection("jdbc:calcite:schemaType=JDBC; 
> schema.jdbcUser=USERNAME; " +
> "schema.jdbcPassword=XX; 
> schema.jdbcUrl=jdbc:oracle:thin:@X:1521:ORCL; 
> schema.conformance=ORACLE_10; schema.fun=oracle" );
>  
>  
> I have cloned calcite project and running main program on that . Please let 
> me know if I have done any mistake .
>  
> After Oracle , I need to work for mySql using calcite .



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (CALCITE-4223) Introducing column statistics to RelOptTable

2022-08-09 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-4223.
--
Fix Version/s: 1.32.0
   Resolution: Fixed

Fixed in 
https://github.com/apache/calcite/commit/7e0057e8de93930f1b2952a1cbcee8ad7a6bfb4b.
 Thank you for your view, [~asolimando]!

> Introducing column statistics to RelOptTable
> 
>
> Key: CALCITE-4223
> URL: https://issues.apache.org/jira/browse/CALCITE-4223
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.32.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Many systems depend on column statistics to compute more accurate stats, such 
> as NDV, average column size, and so on. It would be nice if Calcite can 
> provide such an interface.
> Column statistics might include NDV, average/max column length, number of 
> nulls, number of trues, number of falses and so on. 
> What do you think?
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-07-25 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571209#comment-17571209
 ] 

Chunwei Lei commented on CALCITE-3890:
--

Thank you for your attention, [~zabetak]. IMHO,  the most important point about 
how to implement this rule is how to avoid applying the rule infinitely, which 
is the big difference between the new rule I added and the one used in Hive.  
To achieve this goal, Hive uses some extra data structures to save the 
generated predicates, while the new rule I added just uses MetadataQuery and 
RexSimplify to see whether the ISNOTNULL predicate is redundant or not. 

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable

2022-07-17 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17567773#comment-17567773
 ] 

Chunwei Lei commented on CALCITE-4223:
--

I would like to merge the PR in the next 48hours if no objections appear.

> Introducing column statistics to RelOptTable
> 
>
> Key: CALCITE-4223
> URL: https://issues.apache.org/jira/browse/CALCITE-4223
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Many systems depend on column statistics to compute more accurate stats, such 
> as NDV, average column size, and so on. It would be nice if Calcite can 
> provide such an interface.
> Column statistics might include NDV, average/max column length, number of 
> nulls, number of trues, number of falses and so on. 
> What do you think?
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (CALCITE-4223) Introducing column statistics to RelOptTable

2022-07-17 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17567773#comment-17567773
 ] 

Chunwei Lei edited comment on CALCITE-4223 at 7/18/22 2:08 AM:
---

I would like to merge the PR in the next 48 hours if no objections appear.


was (Author: chunwei lei):
I would like to merge the PR in the next 48hours if no objections appear.

> Introducing column statistics to RelOptTable
> 
>
> Key: CALCITE-4223
> URL: https://issues.apache.org/jira/browse/CALCITE-4223
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Many systems depend on column statistics to compute more accurate stats, such 
> as NDV, average column size, and so on. It would be nice if Calcite can 
> provide such an interface.
> Column statistics might include NDV, average/max column length, number of 
> nulls, number of trues, number of falses and so on. 
> What do you think?
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable

2022-07-05 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562492#comment-17562492
 ] 

Chunwei Lei commented on CALCITE-4223:
--

Could you please review the PR, [~julianhyde]?

> Introducing column statistics to RelOptTable
> 
>
> Key: CALCITE-4223
> URL: https://issues.apache.org/jira/browse/CALCITE-4223
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Many systems depend on column statistics to compute more accurate stats, such 
> as NDV, average column size, and so on. It would be nice if Calcite can 
> provide such an interface.
> Column statistics might include NDV, average/max column length, number of 
> nulls, number of trues, number of falses and so on. 
> What do you think?
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable

2022-07-03 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562018#comment-17562018
 ] 

Chunwei Lei commented on CALCITE-4223:
--

Thank you for your reply, [~asolimando]. I would like to continue this work. 
Hope it can help.

> Introducing column statistics to RelOptTable
> 
>
> Key: CALCITE-4223
> URL: https://issues.apache.org/jira/browse/CALCITE-4223
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Many systems depend on column statistics to compute more accurate stats, such 
> as NDV, average column size, and so on. It would be nice if Calcite can 
> provide such an interface.
> Column statistics might include NDV, average/max column length, number of 
> nulls, number of trues, number of falses and so on. 
> What do you think?
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (CALCITE-4223) Introducing column statistics to RelOptTable

2022-06-28 Thread Chunwei Lei (Jira)
Title: Message Title


 
 
 
 

 
 
 

 
   
 Chunwei Lei commented on  CALCITE-4223  
 

  
 
 
 
 

 
 
  
 
 
 
 

 
  Re: Introducing column statistics to RelOptTable   
 

  
 
 
 
 

 
 It has been a long time since the last discussion. Recently I have time to move it forward. After I reviewed all discussions above, I opened a new PR as Julian suggested: https://github.com/apache/calcite/pull/2845. I am not sure whether it is the best way to introduce the column stats. So I would like to see what others think about it, especially those involved in the downstream projects.  
 

  
 
 
 
 

 
 
 

 
 
 Add Comment  
 

  
 

  
 
 
 
  
 

  
 
 
 
 

 
 This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)  
 
 

 
   
 

  
 

  
 

   



[jira] [Resolved] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-26 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5195.
--
Resolution: Fixed

Fixed in 
https://github.com/apache/calcite/commit/300f5200fe50870da418366613df1cd09921b1b6.

> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-26 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558951#comment-17558951
 ] 

Chunwei Lei commented on CALCITE-5195:
--

Go ahead, [~libenchao]! You can open another JIRA for it.

> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-26 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558283#comment-17558283
 ] 

Chunwei Lei edited comment on CALCITE-5195 at 6/27/22 2:25 AM:
---

Thanks for your reply,[~libenchao]. I tried the following sql:

{code:sql}
select * 
from emp
where exists(select * from dept where emp.deptno = dept.deptno and emp.empno = 
dept.deptno)
{code}
. But the produced plan isn't what I want:

{code:java}
LogicalProject(DEPTNO=[$7])
  LogicalFilter(condition=[IS NOT NULL($11)])
LogicalJoin(condition=[AND(=($0, $9), =($7, $10))], joinType=[left])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0, 1}], agg#0=[MIN($2)])
LogicalProject(DEPTNO=[$0], DEPTNO2=[$0], $f0=[true])
  LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{code}

It seems difficult to use a sql to produce a semi join with join conditions 
like {{A.x = B.y and A.x = B.z}}.




was (Author: chunwei lei):
Thanks for your reply,[~libenchao]. I tried the following sql:

{code:sql}
select * 
from emp
where exists(select * from dept where emp.deptno = dept.deptno and emp.empno = 
dept.deptno)
{code}
. But the produced plan isn't what I want:

{code:java}
LogicalProject(DEPTNO=[$7])
  LogicalFilter(condition=[IS NOT NULL($11)])
LogicalJoin(condition=[AND(=($0, $9), =($7, $10))], joinType=[left])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0, 1}], agg#0=[MIN($2)])
LogicalProject(DEPTNO=[$0], DEPTNO2=[$0], $f0=[true])
  LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{code}

It seems difficult to use a sql to produce a semi join with join conditions 
like {{A.x = B.y and A.x = B.z}.



> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-23 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558283#comment-17558283
 ] 

Chunwei Lei commented on CALCITE-5195:
--

Thanks for your reply,[~libenchao]. I tried the following sql:

{code:sql}
select * 
from emp
where exists(select * from dept where emp.deptno = dept.deptno and emp.empno = 
dept.deptno)
{code}
. But the produced plan isn't what I want:

{code:java}
LogicalProject(DEPTNO=[$7])
  LogicalFilter(condition=[IS NOT NULL($11)])
LogicalJoin(condition=[AND(=($0, $9), =($7, $10))], joinType=[left])
  LogicalTableScan(table=[[CATALOG, SALES, EMP]])
  LogicalAggregate(group=[{0, 1}], agg#0=[MIN($2)])
LogicalProject(DEPTNO=[$0], DEPTNO2=[$0], $f0=[true])
  LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
{code}

It seems difficult to use a sql to produce a semi join with join conditions 
like {{A.x = B.y and A.x = B.z}.



> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-22 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557766#comment-17557766
 ] 

Chunwei Lei edited comment on CALCITE-5195 at 6/23/22 2:08 AM:
---

The following sql also can reproduce the bug.
{code:sql}
select *
from emp e semi join dept d
on e.deptno = d.deptno and e.empno = d.deptno
{code}

But it threw an exception due to the keyword {{semi}} can't be parsed 
correctly. Therefore, I had to use RelBuilder to create such sql to reproduce 
the bug.  Is there a way to support such sql?


was (Author: chunwei lei):
The following sql also can reproduce the bug.
{code:sql}
select *
from emp e semi join dept d
on e.deptno = d.deptno and e.empno = d.deptno
{code}

But it threw an exception due to the keyword {{semi}} can't be parsed 
correctly. Therefore, I had to use RelBuilder to create such sql to reproduce 
the bug. 

> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-22 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557766#comment-17557766
 ] 

Chunwei Lei commented on CALCITE-5195:
--

The following sql also can reproduce the bug.
{code:sql}
select *
from emp e semi join dept d
on e.deptno = d.deptno and e.empno = d.deptno
{code}

But it threw an exception due to the keyword {{semi}} can't be parsed 
correctly. Therefore, I had to use RelBuilder to create such sql to reproduce 
the bug. 

> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-21 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557222#comment-17557222
 ] 

Chunwei Lei commented on CALCITE-5195:
--

PR: https://github.com/apache/calcite/pull/2837. Could you please review it, 
[~libenchao]?

> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Fix For: 1.31.0
>
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-21 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557218#comment-17557218
 ] 

Chunwei Lei commented on CALCITE-5195:
--

I plan to open a PR today so that it can be included in 1.31.

> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Fix For: 1.31.0
>
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-21 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5195:
-
Fix Version/s: 1.31.0

> ArrayIndexOutOfBoundsException when inferring more equal conditions from join 
> condition for semi join
> -
>
> Key: CALCITE-5195
> URL: https://issues.apache.org/jira/browse/CALCITE-5195
> Project: Calcite
>  Issue Type: Bug
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
> Fix For: 1.31.0
>
>
> The following test can reproduce the exception.
> {code:java}
>   @Test void testJoinConditionPushdown4() {
> final Function relFn = b -> {
>   RelNode left = b.scan("EMP")
>   .project(
>   b.field("DEPTNO"),
>   b.field("ENAME"))
>   .build();
>   RelNode right = b.scan("DEPT")
>   .project(
>   b.field("DEPTNO"),
>   b.field("DNAME"))
>   .build();
>   b.push(left).push(right);
>   RexInputRef ref1 = b.field(2, 0, "DEPTNO");
>   RexInputRef ref2 = b.field(2, 1, "DEPTNO");
>   RexInputRef ref3 = b.field(2, 1, "DNAME");
>   RexCall cond1 = (RexCall) b.equals(ref1, ref2);
>   RexCall cond2 = (RexCall) b.equals(ref1, ref3);
>   RexNode cond = b.and(cond1, cond2);
>   return b.semiJoin(cond)
>   .project(b.field(0))
>   .build();
> };
> relFn(relFn)
> .withRule(
> CoreRules.JOIN_PUSH_EXPRESSIONS,
> CoreRules.JOIN_CONDITION_PUSH,
> CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
> CoreRules.JOIN_REDUCE_EXPRESSIONS,
> CoreRules.FILTER_REDUCE_EXPRESSIONS)
> .check();
>   }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Created] (CALCITE-5195) ArrayIndexOutOfBoundsException when inferring more equal conditions from join condition for semi join

2022-06-21 Thread Chunwei Lei (Jira)
Chunwei Lei created CALCITE-5195:


 Summary: ArrayIndexOutOfBoundsException when inferring more equal 
conditions from join condition for semi join
 Key: CALCITE-5195
 URL: https://issues.apache.org/jira/browse/CALCITE-5195
 Project: Calcite
  Issue Type: Bug
Reporter: Chunwei Lei
Assignee: Chunwei Lei


The following test can reproduce the exception.

{code:java}
  @Test void testJoinConditionPushdown4() {
final Function relFn = b -> {
  RelNode left = b.scan("EMP")
  .project(
  b.field("DEPTNO"),
  b.field("ENAME"))
  .build();
  RelNode right = b.scan("DEPT")
  .project(
  b.field("DEPTNO"),
  b.field("DNAME"))
  .build();

  b.push(left).push(right);

  RexInputRef ref1 = b.field(2, 0, "DEPTNO");
  RexInputRef ref2 = b.field(2, 1, "DEPTNO");
  RexInputRef ref3 = b.field(2, 1, "DNAME");

  RexCall cond1 = (RexCall) b.equals(ref1, ref2);
  RexCall cond2 = (RexCall) b.equals(ref1, ref3);

  RexNode cond = b.and(cond1, cond2);
  return b.semiJoin(cond)
  .project(b.field(0))
  .build();
};

relFn(relFn)
.withRule(
CoreRules.JOIN_PUSH_EXPRESSIONS,
CoreRules.JOIN_CONDITION_PUSH,
CoreRules.SEMI_JOIN_PROJECT_TRANSPOSE,
CoreRules.JOIN_REDUCE_EXPRESSIONS,
CoreRules.FILTER_REDUCE_EXPRESSIONS)
.check();
  }
{code}




--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5191) Allow sort by alias in BigQuery

2022-06-19 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556160#comment-17556160
 ] 

Chunwei Lei edited comment on CALCITE-5191 at 6/20/22 2:23 AM:
---

Fixed in 
https://github.com/apache/calcite/commit/fa65a2ec460abcd574e7f28672478a0510da65a1.
Thank you for your PR, [~wenruimeng]!


was (Author: chunwei lei):
Fixed in 
https://github.com/apache/calcite/commit/bbdef5389908121ee67ba3a7381266ab586f36bb.
Thank you for your PR, [~wenruimeng]!

> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation 
> flow due to allow alias not enabled for the BigQuery conformance. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5191) Allow sort by alias in BigQuery

2022-06-19 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5191.
--
Resolution: Fixed

Fixed in 
https://github.com/apache/calcite/commit/bbdef5389908121ee67ba3a7381266ab586f36bb.
Thank you for your PR, [~wenruimeng]!

> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation 
> flow due to allow alias not enabled for the BigQuery conformance. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join

2022-06-15 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554917#comment-17554917
 ] 

Chunwei Lei edited comment on CALCITE-5193 at 6/16/22 6:53 AM:
---

[~libenchao] yes, you're right. I changed the description.


was (Author: chunwei lei):
[~libenchao]yes, you're right. I changed the description.

> Push filter whose conditions include join keys and are composed by OR into 
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:sql}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
> {code}
> can be transformed to 
> {code:sql}
> select * from 
> (select * from a where id=1 or id=2) a 
> full join 
> (select * from b where id=1 or id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key, 
> we can do partition pruning with this transformation, which is a big 
> improvement.
> This improvement is inspired by the query 
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join

2022-06-15 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5193:
-
Description: 
For example,

{code:sql}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
{code}

can be transformed to 

{code:sql}
select * from 
(select * from a where id=1 or id=2) a 
full join 
(select * from b where id=1 or id=2) b
on a.id=b.id;
{code}

If {{a}} and {{b}} are both partitioned tables and id is the partition key, we 
can do partition pruning with this transformation, which is a big improvement.

This improvement is inspired by the query 

{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
b.pt)='20220601';
{code}

which costs a lot due to it scans all partitions in table {{a}} and {{b}}.




  was:
For example,

{code:sql}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2;

{code}

can be transformed to 

{code:java}
select * from 
(select * from a where id=1 or id=2) a 
full join 
(select * from b where id=1 or id=2) b
on a.id=b.id;
{code}

If {{a}} and {{b}} are both partitioned tables and id is the partition key, we 
can do partition pruning with this transformation, which is a big improvement.

This improvement is inspired by the query 

{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
b.pt)='20220601';
{code}

which costs a lot due to it scans all partitions in table {{a}} and {{b}}.





> Push filter whose conditions include join keys and are composed by OR into 
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:sql}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
> {code}
> can be transformed to 
> {code:sql}
> select * from 
> (select * from a where id=1 or id=2) a 
> full join 
> (select * from b where id=1 or id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key, 
> we can do partition pruning with this transformation, which is a big 
> improvement.
> This improvement is inspired by the query 
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join

2022-06-15 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554917#comment-17554917
 ] 

Chunwei Lei commented on CALCITE-5193:
--

[~libenchao]yes, you're right. I changed the description.

> Push filter whose conditions include join keys and are composed by OR into 
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:java}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2
> {code}
> can be transformed to 
> {code:java}
> select * from 
> (select * from a where id=1) a 
> full join 
> (select * from b where id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key, 
> we can do partition pruning with this transformation, which is a big 
> improvement.
> This improvement is inspired by the query 
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join

2022-06-15 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5193:
-
Description: 
For example,

{code:sql}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2;

{code}

can be transformed to 

{code:java}
select * from 
(select * from a where id=1 or id=2) a 
full join 
(select * from b where id=1 or id=2) b
on a.id=b.id;
{code}

If {{a}} and {{b}} are both partitioned tables and id is the partition key, we 
can do partition pruning with this transformation, which is a big improvement.

This improvement is inspired by the query 

{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
b.pt)='20220601';
{code}

which costs a lot due to it scans all partitions in table {{a}} and {{b}}.




  was:
For example,

{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2

{code}

can be transformed to 

{code:java}
select * from 
(select * from a where id=1) a 
full join 
(select * from b where id=2) b
on a.id=b.id;
{code}

If {{a}} and {{b}} are both partitioned tables and id is the partition key, we 
can do partition pruning with this transformation, which is a big improvement.

This improvement is inspired by the query 

{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
b.pt)='20220601';
{code}

which costs a lot due to it scans all partitions in table {{a}} and {{b}}.





> Push filter whose conditions include join keys and are composed by OR into 
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:sql}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2;
> {code}
> can be transformed to 
> {code:java}
> select * from 
> (select * from a where id=1 or id=2) a 
> full join 
> (select * from b where id=1 or id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key, 
> we can do partition pruning with this transformation, which is a big 
> improvement.
> This improvement is inspired by the query 
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join

2022-06-15 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5193:
-
Description: 
For example,

{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2

{code}

can be transformed to 

{code:java}
select * from 
(select * from a where id=1) a 
full join 
(select * from b where id=2) b
on a.id=b.id;
{code}

If {{a}} and {{b}} are both partitioned tables and id is the partition key, we 
can do partition pruning with this transformation, which is a big improvement.

This improvement is inspired by the query 

{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
b.pt)='20220601';
{code}

which costs a lot due to it scans all partitions in table {{a}} and {{b}}.




  was:
For example,

{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2

{code}

can be transformed to 

{code:java}
select * from 
(select * from a where id=1) a 
full join 
(select * from b where id=2) b
on a.id=b.id;
{code}

If {{a}} and {{b}} are both partitioned tables and id is the partition key, we 
can do partition pruning with this transformation, which is a big improvement.

This improvement is inspired by query 

{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
b.pt)='20220601';
{code}

which costs a lot due to it scans all partitions in table a and b.





> Push filter whose conditions include join keys and are composed by OR into 
> inputs of full join
> --
>
> Key: CALCITE-5193
> URL: https://issues.apache.org/jira/browse/CALCITE-5193
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Priority: Major
>
> For example,
> {code:java}
> select * from a full join b on a.id=b.id where a.id=1 or b.id=2
> {code}
> can be transformed to 
> {code:java}
> select * from 
> (select * from a where id=1) a 
> full join 
> (select * from b where id=2) b
> on a.id=b.id;
> {code}
> If {{a}} and {{b}} are both partitioned tables and id is the partition key, 
> we can do partition pruning with this transformation, which is a big 
> improvement.
> This improvement is inspired by the query 
> {code:java}
> select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
> b.pt)='20220601';
> {code}
> which costs a lot due to it scans all partitions in table {{a}} and {{b}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Created] (CALCITE-5193) Push filter whose conditions include join keys and are composed by OR into inputs of full join

2022-06-15 Thread Chunwei Lei (Jira)
Chunwei Lei created CALCITE-5193:


 Summary: Push filter whose conditions include join keys and are 
composed by OR into inputs of full join
 Key: CALCITE-5193
 URL: https://issues.apache.org/jira/browse/CALCITE-5193
 Project: Calcite
  Issue Type: Improvement
Reporter: Chunwei Lei


For example,

{code:java}
select * from a full join b on a.id=b.id where a.id=1 or b.id=2

{code}

can be transformed to 

{code:java}
select * from 
(select * from a where id=1) a 
full join 
(select * from b where id=2) b
on a.id=b.id;
{code}

If {{a}} and {{b}} are both partitioned tables and id is the partition key, we 
can do partition pruning with this transformation, which is a big improvement.

This improvement is inspired by query 

{code:java}
select * from a full join b on a.id=b.id and a.pt=b.pt where COALESCE(a.pt, 
b.pt)='20220601';
{code}

which costs a lot due to it scans all partitions in table a and b.






--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5191) Allow sort by alias in BigQuery

2022-06-14 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554341#comment-17554341
 ] 

Chunwei Lei commented on CALCITE-5191:
--

The doc[1] has said that BigQuery supports alias in the ORDER BY clause.

 

[1]https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#order_by_clause

> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Fix For: 1.31.0
>
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation 
> flow due to allow alias not enabled for the BigQuery conformance. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5191) Allow sort by alias in BigQuery

2022-06-14 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554341#comment-17554341
 ] 

Chunwei Lei edited comment on CALCITE-5191 at 6/15/22 1:51 AM:
---

The doc[1] has said that BigQuery supports alias in the ORDER BY clause.

[1] 
https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#order_by_clause


was (Author: chunwei lei):
The doc[1] has said that BigQuery supports alias in the ORDER BY clause.

 

[1]https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#order_by_clause

> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Fix For: 1.31.0
>
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation 
> flow due to allow alias not enabled for the BigQuery conformance. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-14 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
 ] 

Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:44 AM:
---

Fixed in 
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b].
 Thank you for your PR, [~libenchao]!


was (Author: chunwei lei):
Fixed in 
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b]

. Thank you for your PR, [~libenchao]!

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}.
> See discussion in ML: 
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be 
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or 
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}}, 
> the type inference result is different for these two cases. We suspect that 
> the type inference result leads to the difference of simplification. 
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to 
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '') 
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and 
> ename <> '3'}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-14 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
 ] 

Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:44 AM:
---

Fixed in 
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b]

. Thank you for your PR, [~libenchao]!


was (Author: chunwei lei):
Fixed in 
https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.Thank
 you for your PR, [~libenchao]!

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}.
> See discussion in ML: 
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be 
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or 
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}}, 
> the type inference result is different for these two cases. We suspect that 
> the type inference result leads to the difference of simplification. 
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to 
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '') 
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and 
> ename <> '3'}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-14 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
 ] 

Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:43 AM:
---

Fixed in 
https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.Thank
 you for your PR, [~libenchao]!


was (Author: chunwei lei):
Fixed in 
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
 Thank you for your PR, [~libenchao]!

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}.
> See discussion in ML: 
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be 
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or 
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}}, 
> the type inference result is different for these two cases. We suspect that 
> the type inference result leads to the difference of simplification. 
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to 
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '') 
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and 
> ename <> '3'}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-14 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5169.
--
Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
 Thank you for your PR, [~libenchao]!

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}.
> See discussion in ML: 
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be 
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or 
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}}, 
> the type inference result is different for these two cases. We suspect that 
> the type inference result leads to the difference of simplification. 
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to 
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '') 
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and 
> ename <> '3'}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-14 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554339#comment-17554339
 ] 

Chunwei Lei edited comment on CALCITE-5169 at 6/15/22 1:42 AM:
---

Fixed in 
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
 Thank you for your PR, [~libenchao]!


was (Author: chunwei lei):
Fixed in 
[https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b|https://github.com/apache/calcite/commit/e64a21ee27600537e0498cd1f72475ad1a1a0d2b.].
 Thank you for your PR, [~libenchao]!

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}.
> See discussion in ML: 
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be 
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or 
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}}, 
> the type inference result is different for these two cases. We suspect that 
> the type inference result leads to the difference of simplification. 
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to 
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '') 
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and 
> ename <> '3'}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5191) Allow sort by alias in BigQuery

2022-06-14 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554337#comment-17554337
 ] 

Chunwei Lei commented on CALCITE-5191:
--

Good catch. The PR looks good to me.

> Allow sort by alias in BigQuery
> ---
>
> Key: CALCITE-5191
> URL: https://issues.apache.org/jira/browse/CALCITE-5191
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Wenrui Meng
>Assignee: Wenrui Meng
>Priority: Major
> Fix For: 1.31.0
>
>
> {code:java}
> select 1 as a order by a {code}
> the above statement is valid in BigQuery, but failed in current validation 
> flow due to allow alias not enabled for the BigQuery conformance. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-12 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553366#comment-17553366
 ] 

Chunwei Lei commented on CALCITE-5169:
--

The PR looks good. I will merge the PR shortly.

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}.
> See discussion in ML: 
> https://lists.apache.org/thread/hsdgx0jb3214v6k9p7slbkk3xsbl0b7f
> The original problem is that {{ename <> '' and ename <> '3'}} could be 
> simplified to {{Sarg}} while {{(ename < '' or ename > '') and (ename < '3' or 
> ename > '3')}} cannot.
> Since {{<}} and {{<>}} have different {{SqlOperandTypeChecker.Consistency}}, 
> the type inference result is different for these two cases. We suspect that 
> the type inference result leads to the difference of simplification. 
> However, the real reason is that {{xx < 1 OR xx > 1}} cannot be simplified to 
> {{xx <> 1}} in simplification, which prevents {{(ename < '' or ename > '') 
> and (ename < '3' or ename > '3')}} being simplified to {{ename <> '' and 
> ename <> '3'}}.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-07 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551352#comment-17551352
 ] 

Chunwei Lei edited comment on CALCITE-5169 at 6/8/22 2:18 AM:
--

[~julianhyde], could you please double confirm the PR is the right way to fix 
the bug?


was (Author: chunwei lei):
Could you please double confirm the PR is the right way to fix the bug, 
[~julianhyde]? 

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5169) 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'

2022-06-07 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551352#comment-17551352
 ] 

Chunwei Lei commented on CALCITE-5169:
--

Could you please double confirm the PR is the right way to fix the bug, 
[~julianhyde]? 

> 'xx < 1 OR xx > 1' cannot be simplified to 'xx <> 1'
> 
>
> Key: CALCITE-5169
> URL: https://issues.apache.org/jira/browse/CALCITE-5169
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> As described in the title, {{xx < 1 OR xx > 1}} should be simplified to {{xx 
> <> 1}}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5179) In RelToSqlConverter, AssertionError for values with more than two items when SqlDialect#supportsAliasedValues is false

2022-06-07 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17551350#comment-17551350
 ] 

Chunwei Lei commented on CALCITE-5179:
--

I have put the label "LGTM-will-merge-soon". I will merge it shortly if no 
objections appear.

>  In RelToSqlConverter, AssertionError for values with more than two items 
> when SqlDialect#supportsAliasedValues is false
> 
>
> Key: CALCITE-5179
> URL: https://issues.apache.org/jira/browse/CALCITE-5179
> Project: Calcite
>  Issue Type: Bug
>Reporter: Steven Talbot
>Assignee: Jiajun Xie
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The code at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803]
>  creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL 
> operator is a 
> SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109]
>  which is a "SqlBinaryOperator" (Binary syntax). So that generates an 
> assertion error when it unparses more than two values at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426]
>  
> I don't _think_ I've done anything nonstandard, other than use a dialect with 
> that flag set to false. To reproduce, create a values RelNode with more than 
> two items, convert it with RelToSqlConverter with a dialect using 
> supportsAliasedValues() = false (like Redshift), and attempt to unparse the 
> resulting SqlNode.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5150) Parser should parse subquery with order by inside array constructor

2022-06-05 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5150.
--
Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/4a4114419b0341fa4887b4838195ef7855c00e22|https://github.com/apache/calcite/commit/4a4114419b0341fa4887b4838195ef7855c00e22.].
 Thank you for your PR, [~dmsysolyatin]!

> Parser should parse subquery with order by inside array constructor
> ---
>
> Key: CALCITE-5150
> URL: https://issues.apache.org/jira/browse/CALCITE-5150
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Parser can't parse query that looks like:
> {code}
> SELECT array(select x from (VALUES(1)) x ORDER BY x)
> {code}
> {code}
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered 
> "ORDER" at line 1, column 42.
> Was expecting one of:
>     ")" ...
>     "NATURAL" ...
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false

2022-06-05 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261
 ] 

Chunwei Lei edited comment on CALCITE-5179 at 6/6/22 2:11 AM:
--

IMOP, it would be great if the title can show that the error only appears when 
converting rel node to sql.


was (Author: chunwei lei):
IMOP, it would be great if the title can specify the error that only appears 
when converting rel node to sql.

> AssertionError in values with more than two items when 
> SqlDialect#supportsAliasedValues is false
> 
>
> Key: CALCITE-5179
> URL: https://issues.apache.org/jira/browse/CALCITE-5179
> Project: Calcite
>  Issue Type: Bug
>Reporter: Steven Talbot
>Assignee: Jiajun Xie
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The code at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803]
>  creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL 
> operator is a 
> SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109]
>  which is a "SqlBinaryOperator" (Binary syntax). So that generates an 
> assertion error when it unparses more than two values at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426]
>  
> I don't _think_ I've done anything nonstandard, other than use a dialect with 
> that flag set to false. To reproduce, create a values RelNode with more than 
> two items, convert it with RelToSqlConverter with a dialect using 
> supportsAliasedValues() = false (like Redshift), and attempt to unparse the 
> resulting SqlNode.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false

2022-06-05 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261
 ] 

Chunwei Lei edited comment on CALCITE-5179 at 6/6/22 2:09 AM:
--

IMOP, it would be great if the title can specify the error that only appears 
when converting rel node to sql.


was (Author: chunwei lei):
IMOP, it would be great if the title can specify the error appears when 
converting rel node to sql.

> AssertionError in values with more than two items when 
> SqlDialect#supportsAliasedValues is false
> 
>
> Key: CALCITE-5179
> URL: https://issues.apache.org/jira/browse/CALCITE-5179
> Project: Calcite
>  Issue Type: Bug
>Reporter: Steven Talbot
>Assignee: Jiajun Xie
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The code at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803]
>  creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL 
> operator is a 
> SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109]
>  which is a "SqlBinaryOperator" (Binary syntax). So that generates an 
> assertion error when it unparses more than two values at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426]
>  
> I don't _think_ I've done anything nonstandard, other than use a dialect with 
> that flag set to false. To reproduce, create a values RelNode with more than 
> two items, convert it with RelToSqlConverter with a dialect using 
> supportsAliasedValues() = false (like Redshift), and attempt to unparse the 
> resulting SqlNode.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false

2022-06-05 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261
 ] 

Chunwei Lei edited comment on CALCITE-5179 at 6/6/22 2:07 AM:
--

IMOP, it would be great if the title can specify the error appears when 
converting rel node to sql.


was (Author: chunwei lei):
IMOP, it would be great if the error appears when converting rel node to sql.

> AssertionError in values with more than two items when 
> SqlDialect#supportsAliasedValues is false
> 
>
> Key: CALCITE-5179
> URL: https://issues.apache.org/jira/browse/CALCITE-5179
> Project: Calcite
>  Issue Type: Bug
>Reporter: Steven Talbot
>Assignee: Jiajun Xie
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The code at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803]
>  creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL 
> operator is a 
> SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109]
>  which is a "SqlBinaryOperator" (Binary syntax). So that generates an 
> assertion error when it unparses more than two values at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426]
>  
> I don't _think_ I've done anything nonstandard, other than use a dialect with 
> that flag set to false. To reproduce, create a values RelNode with more than 
> two items, convert it with RelToSqlConverter with a dialect using 
> supportsAliasedValues() = false (like Redshift), and attempt to unparse the 
> resulting SqlNode.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5179) AssertionError in values with more than two items when SqlDialect#supportsAliasedValues is false

2022-06-05 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550261#comment-17550261
 ] 

Chunwei Lei commented on CALCITE-5179:
--

IMOP, it would be great if the error appears when converting rel node to sql.

> AssertionError in values with more than two items when 
> SqlDialect#supportsAliasedValues is false
> 
>
> Key: CALCITE-5179
> URL: https://issues.apache.org/jira/browse/CALCITE-5179
> Project: Calcite
>  Issue Type: Bug
>Reporter: Steven Talbot
>Assignee: Jiajun Xie
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The code at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/rel/rel2sql/RelToSqlConverter.java#L802-L803]
>  creates a UNION ALL with an arbitrary number of operands, but [the UNION_ALL 
> operator is a 
> SqlSetOperator|https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/fun/SqlStdOperatorTable.java#L108-L109]
>  which is a "SqlBinaryOperator" (Binary syntax). So that generates an 
> assertion error when it unparses more than two values at 
> [https://github.com/apache/calcite/blob/4b349032c17b95735b12593a65f7027d54d8b8b6/core/src/main/java/org/apache/calcite/sql/SqlUtil.java#L426]
>  
> I don't _think_ I've done anything nonstandard, other than use a dialect with 
> that flag set to false. To reproduce, create a values RelNode with more than 
> two items, convert it with RelToSqlConverter with a dialect using 
> supportsAliasedValues() = false (like Redshift), and attempt to unparse the 
> resulting SqlNode.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5170) Assertion error on range distribution creation

2022-05-31 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5170.
--
Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/a5af80642787e3f1d24671d3a6339568d0c6b201|https://github.com/apache/calcite/commit/a5af80642787e3f1d24671d3a6339568d0c6b201.].
 Thank you for your PR, [~rkondakov]!

> Assertion error on range distribution creation
> --
>
> Key: CALCITE-5170
> URL: https://issues.apache.org/jira/browse/CALCITE-5170
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> An assertion error occurs on range distribution creation. This test fails:
> {code:java}
> @Test void testRangeRelDistributionKeys() {
> RelDistributions.range(Arrays.asList(0, 1));
> }
> {code}
> This happens because there is an incorrect assertion in 
> {{RelDistributionImpl}} constructor:
> {code:java}
>   assert type == Type.HASH_DISTRIBUTED
>   || type == Type.RANDOM_DISTRIBUTED
>   || keys.isEmpty();
> {code}
> It should be  {{RANGE_DISTRIBUTED}} instead of {{RANDOM_DISTRIBUTED}}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator

2022-05-30 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544091#comment-17544091
 ] 

Chunwei Lei commented on CALCITE-5157:
--

I am not sure whether the proposed fix is the best way. Let's see others' 
points.

> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When a query contains nested field access and is using parentheses to 
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method 
> throws a {{ClassCastException}}, assuming that the input of the DOT operator 
> is a {{SqlCall}}. I think this assumption is wrong, the DOT operator 
> typically has {{SqlIdentifier}} as an input, probably also other classes.
> Here's the stack trace:
> {noformat}
> java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier 
> cannot be cast to class org.apache.calcite.sql.SqlCall 
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are 
> in unnamed module of loader 'app')
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)
>     at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
>     at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)
>     at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)
>     at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)
>     at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)
>     at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)
>     at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
>     at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)
>     at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)
>     at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)
>     at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)
>     at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)
>     at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)
>     at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
>     ... 67 more
> {noformat}
> The problem can be reproduced by modifying the 
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}} 
> test and putting {{au."birthPlace"}} into parentheses:
> {code:sql}
> select (au."birthPlace")."city" as city from ... 
> {code}
> Putting identifiers into parentheses is common to disambiguate field access 
> from identifier qualification. For example, if the {{au}} prefix is removed 
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}} 
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put 
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a 
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the 
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the 
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5150) Parser should parse subquery with order by inside array constructor

2022-05-30 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17544092#comment-17544092
 ] 

Chunwei Lei commented on CALCITE-5150:
--

Get it! The PR looks good to me.

> Parser should parse subquery with order by inside array constructor
> ---
>
> Key: CALCITE-5150
> URL: https://issues.apache.org/jira/browse/CALCITE-5150
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Parser can't parse query that looks like:
> {code}
> SELECT array(select x from (VALUES(1)) x ORDER BY x)
> {code}
> {code}
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered 
> "ORDER" at line 1, column 42.
> Was expecting one of:
>     ")" ...
>     "NATURAL" ...
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5170) Assertion error on range distribution creation

2022-05-29 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17543741#comment-17543741
 ] 

Chunwei Lei commented on CALCITE-5170:
--

Good catch! The PR looks good.

> Assertion error on range distribution creation
> --
>
> Key: CALCITE-5170
> URL: https://issues.apache.org/jira/browse/CALCITE-5170
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>
> An assertion error occurs on range distribution creation. This test fails:
> {code:java}
> @Test void testRangeRelDistributionKeys() {
> RelDistributions.range(Arrays.asList(0, 1));
> }
> {code}
> This happens because there is an incorrect assertion in 
> {{RelDistributionImpl}} constructor:
> {code:java}
>   assert type == Type.HASH_DISTRIBUTED
>   || type == Type.RANDOM_DISTRIBUTED
>   || keys.isEmpty();
> {code}
> It should be  {{RANGE_DISTRIBUTED}} instead of {{RANDOM_DISTRIBUTED}}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5162) RelMdUniqueKeys can return more precise unique keys for Aggregate

2022-05-29 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5162.
--
Fix Version/s: 1.31.0
   Resolution: Fixed

Fixed in 
https://github.com/apache/calcite/commit/f278efb0411ba29c7bd167f5d02e566bf542acb3.

> RelMdUniqueKeys can return more precise unique keys for Aggregate
> -
>
> Key: CALCITE-5162
> URL: https://issues.apache.org/jira/browse/CALCITE-5162
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently, RelMdUniqueKeys always returns group by keys as the unique key. 
> However, it can return more precise unique keys by looking through group by 
> keys. For instance:
> {code:java}
> select deptno, deptname, count(*) from dept group by deptno, deptname;{code}
> RelMdUniqueKeys can return {{deptno}} as the unique key instead of (deptno, 
> deptname) if {{deptno}} is unique.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'

2022-05-26 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5073.
--
Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/4969b9690efe999c522daf1151c4a00a33be0eb0|https://github.com/apache/calcite/commit/4969b9690efe999c522daf1151c4a00a33be0eb0.].
 Thank you for your PR, [~libenchao]!

> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id' 
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above 
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can 
> infer more conditions from join condition (or above filter condition) to the 
> Join's operands.
> It's reported in ML: 
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5163) MysqlSqlDialect unparse LISTAGG aggregate function error

2022-05-26 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5163.
--
Fix Version/s: 1.31.0
   Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/52c9b7836bbed90038f28a933c0cb56d2d69cd28|https://github.com/apache/calcite/commit/52c9b7836bbed90038f28a933c0cb56d2d69cd28.].
 Thank you for your PR, [~wojustme]!

> MysqlSqlDialect unparse LISTAGG aggregate function error
> 
>
> Key: CALCITE-5163
> URL: https://issues.apache.org/jira/browse/CALCITE-5163
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Xurenhe
>Assignee: Xurenhe
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> CALCITE-4349 supports GROUP_CONCAT aggregate function. [The 
> code|https://github.com/apache/calcite/blob/7c1e2746b94a50f0bb08571287acb5327fdc1b16/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5746]
>  translated *GROUP_CONCAT* to *LISTAGG* during sql_to_rel.
> But, *MysqlSqlDialect* keep *LISTAGG* after uparsing query's relnode, the 
> result sql couldn't be executed in MySQL engine.
>  
> I expect the following test case to be successful, but it's failed.
> {code:java}
> // test in org.apache.calcite.rel.rel2sql.RelToSqlConverterTest
> @Test void testMySqlGroupConcat() {
>   final String query = "select\n"
>   + "listagg(distinct \"product_name\", ',') within group(order by 
> \"cases_per_pallet\")"
>   + "from \"product\"\n"
>   + "group by \"product_id\"\n";
>   final String expected = "SELECT GROUP_CONCAT(DISTINCT `product_name` ORDER 
> BY "
>   + "`cases_per_pallet` IS NULL, `cases_per_pallet` SEPARATOR ',')\n"
>   + "FROM `foodmart`.`product`\n"
>   + "GROUP BY `product_id`";
>   sql(query).withMysql().ok(expected);
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5166) Method accept(RelShuttle) is not overridden in LogicalCalc and LogicalTableModify

2022-05-25 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5166.
--
Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/9bdfd9a178f493b235d8785afd94fd0c998e8cce|https://github.com/apache/calcite/commit/9bdfd9a178f493b235d8785afd94fd0c998e8cce.].
 Thank you for your PR, [~rkondakov]!

> Method accept(RelShuttle) is not overridden in LogicalCalc and 
> LogicalTableModify
> -
>
> Key: CALCITE-5166
> URL: https://issues.apache.org/jira/browse/CALCITE-5166
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
>  Labels: pull-request-available, pull_request_available
> Fix For: 1.31.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Method {{RelNode#accept(RelShuttle)}} is not overridden for {{LogicalCalc}} 
> and {{{}LogicalTableModify{}}}. This leads to the bug when logic implemented 
> in {{RelShuttle#visit(LogicalCalc)}} is never applied because {{visit()}} 
> method is never called due to incorrect dispatching.
> This test will fail without {{accept(RelShuttle)}} method is overridden in 
> Calc:
> {code:java}
> @Test void testRelShuttleForLogicalCalc() {
>   final String sql = "select ename from emp";
>   final RelNode rel = sql(sql).toRel();
>   final HepProgramBuilder programBuilder = HepProgram.builder();
>   programBuilder.addRuleInstance(CoreRules.PROJECT_TO_CALC);
>   final HepPlanner planner = new HepPlanner(programBuilder.build());
>   planner.setRoot(rel);
>   final RelNode calc = planner.findBestExp();
>   final List rels = new ArrayList<>();
>   final RelShuttleImpl visitor = new RelShuttleImpl() {
> @Override public RelNode visit(LogicalCalc calc) {
>   RelNode visitedRel = super.visit(calc);
>   rels.add(visitedRel);
>   return visitedRel;
> }
>   };
>   calc.accept(visitor);
>   assertThat(rels.size(), is(1));
>   assertThat(rels.get(0), isA(LogicalCalc.class));
> } {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5163) MysqlSqlDialect unparse LISTAGG aggregate function error

2022-05-23 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541238#comment-17541238
 ] 

Chunwei Lei commented on CALCITE-5163:
--

Looks basically good. I left some minor comments.

> MysqlSqlDialect unparse LISTAGG aggregate function error
> 
>
> Key: CALCITE-5163
> URL: https://issues.apache.org/jira/browse/CALCITE-5163
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Xurenhe
>Assignee: Xurenhe
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> CALCITE-4349 supports GROUP_CONCAT aggregate function. [The 
> code|https://github.com/apache/calcite/blob/7c1e2746b94a50f0bb08571287acb5327fdc1b16/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L5746]
>  translated *GROUP_CONCAT* to *LISTAGG* during sql_to_rel.
> But, *MysqlSqlDialect* keep *LISTAGG* after uparsing query's relnode, the 
> result sql couldn't be executed in MySQL engine.
>  
> I expect the following test case to be successful, but it's failed.
> {code:java}
> // test in org.apache.calcite.rel.rel2sql.RelToSqlConverterTest
> @Test void testMySqlGroupConcat() {
>   final String query = "select\n"
>   + "listagg(distinct \"product_name\", ',') within group(order by 
> \"cases_per_pallet\")"
>   + "from \"product\"\n"
>   + "group by \"product_id\"\n";
>   final String expected = "SELECT GROUP_CONCAT(DISTINCT `product_name` ORDER 
> BY "
>   + "`cases_per_pallet` IS NULL, `cases_per_pallet` SEPARATOR ',')\n"
>   + "FROM `foodmart`.`product`\n"
>   + "GROUP BY `product_id`";
>   sql(query).withMysql().ok(expected);
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5166) Method accept(RelShuttle) is not overridden in LogicalCalc and LogicalTableModify

2022-05-22 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540725#comment-17540725
 ] 

Chunwei Lei commented on CALCITE-5166:
--

Makes sense. There're some operators that have the same issue, such as 
{{{}TableSpool{}}}, {{{}TableFunctionScan{}}}. Could you please check whether 
they need to override the method too? 

> Method accept(RelShuttle) is not overridden in LogicalCalc and 
> LogicalTableModify
> -
>
> Key: CALCITE-5166
> URL: https://issues.apache.org/jira/browse/CALCITE-5166
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Roman Kondakov
>Assignee: Roman Kondakov
>Priority: Major
>  Labels: pull_request_available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Method {{RelNode#accept(RelShuttle)}} is not overridden for {{LogicalCalc}} 
> and {{{}LogicalTableModify{}}}. This leads to the bug when logic implemented 
> in {{RelShuttle#visit(LogicalCalc)}} is never applied because {{visit()}} 
> method is never called due to incorrect dispatching.
> This test will fail without {{accept(RelShuttle)}} method is overridden in 
> Calc:
> {code:java}
> @Test void testRelShuttleForLogicalCalc() {
>   final String sql = "select ename from emp";
>   final RelNode rel = sql(sql).toRel();
>   final HepProgramBuilder programBuilder = HepProgram.builder();
>   programBuilder.addRuleInstance(CoreRules.PROJECT_TO_CALC);
>   final HepPlanner planner = new HepPlanner(programBuilder.build());
>   planner.setRoot(rel);
>   final RelNode calc = planner.findBestExp();
>   final List rels = new ArrayList<>();
>   final RelShuttleImpl visitor = new RelShuttleImpl() {
> @Override public RelNode visit(LogicalCalc calc) {
>   RelNode visitedRel = super.visit(calc);
>   rels.add(visitedRel);
>   return visitedRel;
> }
>   };
>   calc.accept(visitor);
>   assertThat(rels.size(), is(1));
>   assertThat(rels.get(0), isA(LogicalCalc.class));
> } {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-20 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-3890.
--
Resolution: Fixed

Fixed in 
https://github.com/apache/calcite/commit/acf82f7784823c30fb7a64e905c3acacd0ed4f2b.

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Created] (CALCITE-5162) RelMdUniqueKeys can return more precise unique keys for Aggregate

2022-05-19 Thread Chunwei Lei (Jira)
Chunwei Lei created CALCITE-5162:


 Summary: RelMdUniqueKeys can return more precise unique keys for 
Aggregate
 Key: CALCITE-5162
 URL: https://issues.apache.org/jira/browse/CALCITE-5162
 Project: Calcite
  Issue Type: Improvement
Reporter: Chunwei Lei
Assignee: Chunwei Lei


Currently, RelMdUniqueKeys always returns group by keys as the unique key. 
However, it can return more precise unique keys by looking through group by 
keys. For instance:
{code:java}
select deptno, deptname, count(*) from dept group by deptno, deptname;{code}
RelMdUniqueKeys can return {{deptno}} as the unique key instead of (deptno, 
deptname) if {{deptno}} is unique.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'

2022-05-18 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539190#comment-17539190
 ] 

Chunwei Lei commented on CALCITE-5073:
--

I am +1 for this positive change. I will find some time to review the PR this 
week.

> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id' 
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above 
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can 
> infer more conditions from join condition (or above filter condition) to the 
> Join's operands.
> It's reported in ML: 
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator

2022-05-18 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
 ] 

Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:51 AM:
---

I did some investigation on this. It seems `authors`.`birthPlace`.`city' is 
recognized as a {{SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads to 
the error. Therefore, I wonder if the right way is to fix the wrong 
recognization.


was (Author: chunwei lei):
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city' 
}}{{is recognized as a SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads 
to the error. Therefore, I wonder if the right way is to fix the wrong 
recognization.

> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to 
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method 
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT 
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT 
> operator typically has {{SqlIdentifier}} as an input, probably also other 
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier 
> cannot be cast to class org.apache.calcite.sql.SqlCall 
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are 
> in unnamed module of loader 'app')}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{    at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{    at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{    at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{    at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{    ... 67 more}}
> The problem can be reproduced by modifying the 
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}} 
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access 
> from identifier qualification. For exmaple, if the {{au}} prefix is removed 
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}} 
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put 
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a 
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the 
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the 
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.



--
This message was sent by Atlassian Jira
(v8

[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator

2022-05-18 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
 ] 

Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:49 AM:
---

I did some investigation on this. It seems {{`authors`.`birthPlace`.`city' 
}}{{is recognized as a SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads 
to the error. Therefore, I wonder if the right way is to fix the wrong 
recognization.


was (Author: chunwei lei):
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is 
recognized as a {{SqlCall rather than {{{}SqlIdentifier{}}}, which leads to 
the error. Therefore, I wonder if the right way is to fix the wrong 
recognization.

> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to 
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method 
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT 
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT 
> operator typically has {{SqlIdentifier}} as an input, probably also other 
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier 
> cannot be cast to class org.apache.calcite.sql.SqlCall 
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are 
> in unnamed module of loader 'app')}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{    at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{    at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{    at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{    at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{    ... 67 more}}
> The problem can be reproduced by modifying the 
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}} 
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access 
> from identifier qualification. For exmaple, if the {{au}} prefix is removed 
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}} 
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put 
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a 
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the 
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the 
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.



--
This message was sent by Atlassian Ji

[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator

2022-05-18 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
 ] 

Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:49 AM:
---

I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is 
recognized as a {{SqlCall rather than {{{}SqlIdentifier{}}}, which leads to 
the error. Therefore, I wonder if the right way is to fix the wrong 
recognization.


was (Author: chunwei lei):
I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is 
recognized as a {{SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads to 
the error. Therefore, I wonder if the right way is to fix the wrong 
recognization.

> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to 
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method 
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT 
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT 
> operator typically has {{SqlIdentifier}} as an input, probably also other 
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier 
> cannot be cast to class org.apache.calcite.sql.SqlCall 
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are 
> in unnamed module of loader 'app')}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{    at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{    at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{    at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{    at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{    ... 67 more}}
> The problem can be reproduced by modifying the 
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}} 
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access 
> from identifier qualification. For exmaple, if the {{au}} prefix is removed 
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}} 
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put 
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a 
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the 
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the 
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.



--
This message was sent by Atlassian Ji

[jira] [Commented] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator

2022-05-18 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539174#comment-17539174
 ] 

Chunwei Lei commented on CALCITE-5157:
--

I did some investigation on this. It seems {{`authors`.`birthPlace`.`city` }}is 
recognized as a {{SqlCall}} rather than {{{}SqlIdentifier{}}}, which leads to 
the error. Therefore, I wonder if the right way is to fix the wrong 
recognization.

> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to 
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method 
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT 
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT 
> operator typically has {{SqlIdentifier}} as an input, probably also other 
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier 
> cannot be cast to class org.apache.calcite.sql.SqlCall 
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are 
> in unnamed module of loader 'app')}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{    at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{    at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{    at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{    at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{    ... 67 more}}
> The problem can be reproduced by modifying the 
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}} 
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access 
> from identifier qualification. For exmaple, if the {{au}} prefix is removed 
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}} 
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put 
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a 
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the 
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the 
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator

2022-05-18 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539168#comment-17539168
 ] 

Chunwei Lei edited comment on CALCITE-5157 at 5/19/22 2:34 AM:
---

FYI: I tried the following SQL in PostgreSQL and it works well.
{code:java}
postgres=# SELECT ("item")."name" FROM "on_hand" ;
    name

 fuzzy dice
(1 row){code}
So I think it's a bug.


was (Author: chunwei lei):
FYI: I tried the following SQL in PostgreSQL and it works well.
{code:java}
postgres=# SELECT ("item")."name" FROM "on_hand" ;
    name

 fuzzy dice
(1 row){code}

> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to 
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method 
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT 
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT 
> operator typically has {{SqlIdentifier}} as an input, probably also other 
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier 
> cannot be cast to class org.apache.calcite.sql.SqlCall 
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are 
> in unnamed module of loader 'app')}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{    at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{    at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{    at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{    at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{    ... 67 more}}
> The problem can be reproduced by modifying the 
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}} 
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access 
> from identifier qualification. For exmaple, if the {{au}} prefix is removed 
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}} 
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put 
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a 
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the 
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the 
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5157) ClassCastException in checkRollUp with DOT operator

2022-05-18 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539168#comment-17539168
 ] 

Chunwei Lei commented on CALCITE-5157:
--

FYI: I tried the following SQL in PostgreSQL and it works well.
{code:java}
postgres=# SELECT ("item")."name" FROM "on_hand" ;
    name

 fuzzy dice
(1 row){code}

> ClassCastException in checkRollUp with DOT operator
> ---
>
> Key: CALCITE-5157
> URL: https://issues.apache.org/jira/browse/CALCITE-5157
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Viliam Durina
>Priority: Major
>
> When a query contains nested field access and is using parentheses to 
> disambiguate the identifier, the {{SqlValidatorImpl.checkRollup()}} method 
> throws a {{{}ClassCastException{}}}, assuming that the input of the DOT 
> operator is a {{{}SqlCall{}}}. I think this assumption is wrong, the DOT 
> operator typically has {{SqlIdentifier}} as an input, probably also other 
> classes.
> Here's the stack trace:
> {{java.lang.ClassCastException: class org.apache.calcite.sql.SqlIdentifier 
> cannot be cast to class org.apache.calcite.sql.SqlCall 
> (org.apache.calcite.sql.SqlIdentifier and org.apache.calcite.sql.SqlCall are 
> in unnamed module of loader 'app')}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3730)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUp(SqlValidatorImpl.java:3749)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.checkRollUpInSelectList(SqlValidatorImpl.java:3673)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3661)}}
> {{    at 
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)}}
> {{    at 
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)}}
> {{    at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)}}
> {{    at 
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)}}
> {{    at 
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:587)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)}}
> {{    at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)}}
> {{    at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)}}
> {{    at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)}}
> {{    at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)}}
> {{    ... 67 more}}
> The problem can be reproduced by modifying the 
> {{ReflectiveSchemaTest.testSelectWithFieldAccessOnFirstLevelRecordType()}} 
> test and putting {{au."birthPlace"}} into parentheses:
> {{select (au."birthPlace")."city" as city from ... }}
> Putting identifiers into parentheses is common to disambiguate field access 
> from identifier qualification. For exmaple, if the {{au}} prefix is removed 
> from the query in the {{testSelectWithFieldAccessOnFirstLevelRecordType}} 
> test, it will fail with {{{}Table 'birthPlace' not found{}}}. But if we put 
> {{\"birthPlace\"}} into parentheses, then it is correctly recognized as a 
> column of the {{authors}} table.
> I'm not sure about the correct fix to this issue which would not break the 
> roll-up functionality, perhaps its author [~zhumayun] can help reviewing the 
> PR or suggesting another fix. I'll soon create a PR with a proposed fix.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-18 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-3890:
-
Fix Version/s: 1.31.0

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys between target keys and group keys

2022-05-17 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5149.
--
Fix Version/s: 1.31.0
   Resolution: Fixed

Fixed in 
https://github.com/apache/calcite/commit/c5d45cb094407658f5f933240bc4492b6239e45e.

> Refine RelMdColumnUniqueness for Aggregate by considering intersect keys 
> between target keys and group keys 
> 
>
> Key: CALCITE-5149
> URL: https://issues.apache.org/jira/browse/CALCITE-5149
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> For instance,
> columns: [0, 1]
> group keys: [0, 2]
> If [0] is unique, then [0, 1] is unique too.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Comment Edited] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'

2022-05-16 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537896#comment-17537896
 ] 

Chunwei Lei edited comment on CALCITE-5073 at 5/17/22 3:37 AM:
---

FYI: Flink has a rule named {{JoinConditionEqualityTransferRule}} [1] to do the 
same thing.

[1] 
https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/JoinConditionEqualityTransferRule.scala


was (Author: chunwei lei):
FYI: Flink has a rule named {{JoinConditionEqualityTransferRule}} [1] to do the 
same thing.

[1]https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/JoinConditionEqualityTransferRule.scala

> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id' 
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above 
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can 
> infer more conditions from join condition (or above filter condition) to the 
> Join's operands.
> It's reported in ML: 
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5073) JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 AND LHS.C2 = RHS.C1'

2022-05-16 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537896#comment-17537896
 ] 

Chunwei Lei commented on CALCITE-5073:
--

FYI: Flink has a rule named {{JoinConditionEqualityTransferRule}} [1] to do the 
same thing.

[1]https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/JoinConditionEqualityTransferRule.scala

> JoinConditionPushRule cannot infer 'LHS.C1 = LHS.C2' from 'LHS.C1 = RHS.C1 
> AND LHS.C2 = RHS.C1'
> ---
>
> Key: CALCITE-5073
> URL: https://issues.apache.org/jira/browse/CALCITE-5073
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> For the below SQL:
> {code:java}
> SELECT * FROM T1, T2, T3
> WHERE T1.id = T3.id AND T2.id = T3.id{code}
> Currently, JoinConditionPushRule cannot infer the condition 'T1.id = T2.id' 
> for the first join relation.
> There is another rule {{JoinPushThroughJoinRule}}, it can handle the above 
> case in a different way (by reordering joins).
> This can be a more general optimization, which like the title says, we can 
> infer more conditions from join condition (or above filter condition) to the 
> Join's operands.
> It's reported in ML: 
> https://lists.apache.org/thread/6fs09cgyz5rzrty90632ywoo477q0gdk



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)

2022-05-16 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5137.
--
Fix Version/s: 1.31.0
   Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/664e24921c521b9b8df03e4744d878a69dfeb10e.]
 Thank you for your PR, [~dmsysolyatin]!

> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT 
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5142) Upgrade Calcite to Avatica 1.21.0

2022-05-16 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5142.
--
Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/4ce91575dbe553c9dd09396f9c76b9fda8bba1bb.]
 Thank you for the PR, [~libenchao] !

> Upgrade Calcite to Avatica 1.21.0
> -
>
> Key: CALCITE-5142
> URL: https://issues.apache.org/jira/browse/CALCITE-5142
> Project: Calcite
>  Issue Type: Improvement
>Affects Versions: 1.30.0
>Reporter: Benchao Li
>Assignee: Benchao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Avatica 1.21.0 was just 
> [released|https://lists.apache.org/thread/nspxj2jq0owt982woyxnty6bdzstbyxd], 
> Calcite should use the latest Avatica version.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5137) EnumerableUncollect throws NPE if input has ((List) null)

2022-05-15 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537305#comment-17537305
 ] 

Chunwei Lei commented on CALCITE-5137:
--

The PR looks good to me. I will merge the PR if no other comments in the next 
24 hours.

> EnumerableUncollect throws NPE if input has ((List) null)
> -
>
> Key: CALCITE-5137
> URL: https://issues.apache.org/jira/browse/CALCITE-5137
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>  Labels: pull-request-available
>
> EnumerableUncollect throws NPE when an input has element = ((List) null)
> Example:
> {code}
> SELECT * FROM UNNEST(CAST(null AS INTEGER ARRAY))
> {code}
> In a real situation, it can appear if to use left join. For example:
> {code}
> SELECT ARRAY(SELECT * FROM UNNEST(t.x)) FROM (VALUES(1)) LEFT JOIN (SELECT 
> ARRAY[1] as x, 2 as y) t ON t.y = 1
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5150) Parser should parse subquery with order by inside array constructor

2022-05-15 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537303#comment-17537303
 ] 

Chunwei Lei commented on CALCITE-5150:
--

Thank you for reporting this, [~dmsysolyatin] . Honestly, I have not seen this 
syntax before. Are there other systems that support this syntax?

> Parser should parse subquery with order by inside array constructor
> ---
>
> Key: CALCITE-5150
> URL: https://issues.apache.org/jira/browse/CALCITE-5150
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Parser can't parse query that looks like:
> {code}
> SELECT array(select x from (VALUES(1)) x ORDER BY x)
> {code}
> {code}
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered 
> "ORDER" at line 1, column 42.
> Was expecting one of:
>     ")" ...
>     "NATURAL" ...
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys between target keys and group keys

2022-05-15 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5149:
-
Summary: Refine RelMdColumnUniqueness for Aggregate by considering 
intersect keys between target keys and group keys   (was: Refine 
RelMdColumnUniqueness for Aggregate by considering intersect keys)

> Refine RelMdColumnUniqueness for Aggregate by considering intersect keys 
> between target keys and group keys 
> 
>
> Key: CALCITE-5149
> URL: https://issues.apache.org/jira/browse/CALCITE-5149
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> For instance,
> columns: [0, 1]
> group keys: [0, 2]
> If [0] is unique, then [0, 1] is unique too.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5081) Group keys of Aggregate are wrongly changed during decorrelation

2022-05-15 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5081.
--
Fix Version/s: 1.31.0
   Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/1bce280a2957326dc5c249cfd079edfd2c54adf4.]
 Thank you for your PR, [~libenchao] !

> Group keys of Aggregate are wrongly changed during decorrelation
> 
>
> Key: CALCITE-5081
> URL: https://issues.apache.org/jira/browse/CALCITE-5081
> Project: Calcite
>  Issue Type: Bug
>  Components: core
>Reporter: Shiven Dvrk
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Bug in RelDecorrelator.decorrelate(Aggregate, boolean) is changing the group 
> keys. We have following tables
>  table1:
> {noformat}
>     "ID", SqlTypeName.VARCHAR,
>     "K509", SqlTypeName.BIGINT,
>     "K505", SqlTypeName.BIGINT
> {noformat}
>  table2:
> {noformat}
>       "K14507", SqlTypeName.VARCHAR,
>       "K14506", SqlTypeName.BIGINT,
>       "K14509", SqlTypeName.BIGINT
> {noformat}
> A source relnode representing the following SQL:
> {code}    SELECT
>    `K505`,
>    SUM(`test_total`) AS ` $ f1`
> FROM
>    (
>       SELECT
>          ` $ cor0`.`K509`,
>          ` $ cor0`.`K505`,
>          ` $ cor0`.` $ f0` AS `test_total`
>       FROM
>          `table1` AS ` $ cor0`,
>          LATERAL (
>          SELECT
>             SUM(`K14506`) AS ` $ f0`
>          FROM
>             `table2`
>          WHERE
>             `K14507` = ` $ cor0`.`ID`) AS `t1`
>    )
>    AS `t2`
> WHERE
>    `t2`.`K505` = 1000
> GROUP BY
>    `K505`
> {code}
>  
>    After decorelation, the decorrelated relNode represents the following 
> incorrect SQL:
> {code}   SELECT
>    `t`.`K509`,
>    SUM(`t1`.` $ f1`) AS ` $ f1`
> FROM
>    (
>       SELECT
>          *
>       FROM
>          `table1`
>       WHERE
>          `K505` = 1000
>    )
>    AS `t`
>    LEFT JOIN
>       (
>          SELECT
>             `K14507`,
>             SUM(`K14506`) AS ` $ f1`
>          FROM
>             `table2`
>          GROUP BY
>             `K14507`
>       )
>       AS `t1`
>       ON `t`.`ID` = `t1`.`K14507`
> GROUP BY
>    `t`.`K509`
> {code}
>    it changed the group key.
>    
>    it looks like the logic in RelDecorrelator.decorrelate(Aggregate, boolean) 
> is always picking the 0th index group key.
>    We built the relNode using the following logic(used sqls above to explain 
> the problem)
> {code} 
> val builder = RelBuilder.create(createConfig())
> val v = Holder.of[RexCorrelVariable](null)
> val relNode = builder
>   .scan("table1")
>   .variable(v)
>   .scan("table2")
>   .filter(builder.equals(builder.field(0), builder.field(v.get(), "ID")))
>   .project(builder.field("K14506"))
>   .aggregate(builder.groupKey(), builder.sum(builder.field(0)))
>   .project(builder.alias(builder.field(0), "test_total"))
>   .correlate(JoinRelType.LEFT, v.get().id, builder.field(2, 0, "ID"))
>   .project(builder.field(1), builder.field(2), builder.field(3))
>   .filter(builder.equals(builder.field(1), builder.literal(1000)))
>   .aggregate(builder.groupKey("K505"), builder.sum(builder.field(2)))
>   .build()
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys

2022-05-12 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5149:
-
Description: 
For instance,

columns: [0, 1]
group keys: [0, 2]

If [0] is unique, then [0, 1] is unique too.

  was:
For instance,

columns: [0, 1]
group by [0, 2]



If [0] is unique, then [0, 1] is unique too.


> Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
> 
>
> Key: CALCITE-5149
> URL: https://issues.apache.org/jira/browse/CALCITE-5149
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> For instance,
> columns: [0, 1]
> group keys: [0, 2]
> If [0] is unique, then [0, 1] is unique too.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys

2022-05-12 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5149:
-
Description: 
For instance,

columns: [0, 1]
group by [0, 2]



If [0] is unique, then [0, 1] is unique too.

  was:
Currently, RelMdColumnUniqueness always returns null when meeting the Window 
operator. We can improve it by considering its partition keys. For instance,
{code:java}
select empno, rank () over (partition by empno order by empno) as rn from 
emp{code}
If {{empno}} is the primary key of the table, {{empno}} is unique obviously.

 


> Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
> 
>
> Key: CALCITE-5149
> URL: https://issues.apache.org/jira/browse/CALCITE-5149
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> For instance,
> columns: [0, 1]
> group by [0, 2]
> If [0] is unique, then [0, 1] is unique too.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-5149) Refine RelMdColumnUniqueness for Aggregate by considering intersect keys

2022-05-12 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-5149:
-
Summary: Refine RelMdColumnUniqueness for Aggregate by considering 
intersect keys  (was: Refine RelMdColumnUniqueness for Window by considering 
partition keys)

> Refine RelMdColumnUniqueness for Aggregate by considering intersect keys
> 
>
> Key: CALCITE-5149
> URL: https://issues.apache.org/jira/browse/CALCITE-5149
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> Currently, RelMdColumnUniqueness always returns null when meeting the Window 
> operator. We can improve it by considering its partition keys. For instance,
> {code:java}
> select empno, rank () over (partition by empno order by empno) as rn from 
> emp{code}
> If {{empno}} is the primary key of the table, {{empno}} is unique obviously.
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Created] (CALCITE-5149) Refine RelMdColumnUniqueness for Window by considering partition keys

2022-05-12 Thread Chunwei Lei (Jira)
Chunwei Lei created CALCITE-5149:


 Summary: Refine RelMdColumnUniqueness for Window by considering 
partition keys
 Key: CALCITE-5149
 URL: https://issues.apache.org/jira/browse/CALCITE-5149
 Project: Calcite
  Issue Type: Improvement
Reporter: Chunwei Lei
Assignee: Chunwei Lei


Currently, RelMdColumnUniqueness always returns null when meeting the Window 
operator. We can improve it by considering its partition keys. For instance,
{code:java}
select empno, rank () over (partition by empno order by empno) as rn from 
emp{code}
If {{empno}} is the primary key of the table, {{empno}} is unique obviously.

 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5125) Extend "||" operator to work with arrays

2022-05-11 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5125.
--
Fix Version/s: 1.31.0
   Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/b7dbee54df60720ece301ba39cd928844e1a42ee.]
 Thank you for your PR, [~dmsysolyatin] !

> Extend "||" operator to work with arrays
> 
>
> Key: CALCITE-5125
> URL: https://issues.apache.org/jira/browse/CALCITE-5125
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> "||" operator can be used only for string concatenation but it would be good 
> to use it also for array concatenation as PostgreSQL



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-11 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17535815#comment-17535815
 ] 

Chunwei Lei commented on CALCITE-3890:
--

Exactly. I think {{Strong}} can handle it well.

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-10 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534652#comment-17534652
 ] 

Chunwei Lei commented on CALCITE-3890:
--

I am trying to use {{Strong}} to analyze the join condition, as Julian said.

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5125) Extend "||" operator to work with arrays

2022-05-10 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534640#comment-17534640
 ] 

Chunwei Lei commented on CALCITE-5125:
--

The PR looks good to me. I will merge the PR if there're no more comments in 
the next 24 hours.

> Extend "||" operator to work with arrays
> 
>
> Key: CALCITE-5125
> URL: https://issues.apache.org/jira/browse/CALCITE-5125
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Dmitry Sysolyatin
>Assignee: Dmitry Sysolyatin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> "||" operator can be used only for string concatenation but it would be good 
> to use it also for array concatenation as PostgreSQL



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Resolved] (CALCITE-5131) Remove redundant type cast

2022-05-10 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei resolved CALCITE-5131.
--
Fix Version/s: 1.31.0
   Resolution: Fixed

Fixed in 
[https://github.com/apache/calcite/commit/f58a6aa8d04accbca3d1237eddfd82dbd3553cf8.]

Thank you for your review, [~asolimando] !

> Remove redundant type cast
> --
>
> Key: CALCITE-5131
> URL: https://issues.apache.org/jira/browse/CALCITE-5131
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.31.0
>
> Attachments: image-2022-05-05-11-03-26-735.png, index.html, 
> script.js, styles.css
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> There're a number of redundant type casts. I suggest removing them.
> !image-2022-05-05-11-03-26-735.png!
>  
> What do you think? BTW, it seems the CheckStyle plugin doesn't provide such a 
> check. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-10 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534637#comment-17534637
 ] 

Chunwei Lei commented on CALCITE-3890:
--

{quote}The description does not say that the filter is a relational expression, 
nor does it say that the filter is before the join rather than after.
{quote}
The description has been updated.

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-10 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-3890:
-
Description: 
We can infer IS NOT NULL predicate from join which implies some columns may not 
be null. For instance, 

 
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null/b.id is not null and push down them into the 
child node of the join. Then it becomes
{code:java}
select * from (select* from a where id is null) t1 join (select * from b where 
id is not null) on t1.id = t2.id;
{code}
 

 

  was:
We can infer IS NOT NULL predicate from join which implies some columns may not 
be null. For instance, 

 
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null and b.id is not null and pushdown them into the 
child node of the join. Then it becomes
{code:java}
select * from (select* from a where id is null) t1 join (select * from b where 
id is not null) on t1.id = t2.id;
{code}
 

 


> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null/b.id is not null and push down them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-10 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-3890:
-
Description: 
We can infer IS NOT NULL predicate from join which implies some columns may not 
be null. For instance, 

 
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null and b.id is not null and pushdown them into the 
child node of the join. Then it becomes
{code:java}
select * from (select* from a where id is null) t1 join (select * from b where 
id is not null) on t1.id = t2.id;
{code}
 

 

  was:
We can infer IS NOT NULL predicate from join which implies some columns may not 
be null. For instance, 

 
{code:java}
select * from a join b on a.id = b.id;
{code}
we can infer a.id is not null and b.id is not null.


> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null and pushdown them into the 
> child node of the join. Then it becomes
> {code:java}
> select * from (select* from a where id is null) t1 join (select * from b 
> where id is not null) on t1.id = t2.id;
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-10 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534632#comment-17534632
 ] 

Chunwei Lei commented on CALCITE-3890:
--

{quote}Emp e LEFT JOIN Dept d USING (deptno)

becomes

(SELECT * FROM Emp WHERE deptno IS NOT NULL) LEFT JOIN Dept d USING (deptno)
{quote}
AFAIK, this transformation is wrong. For left/right join, the 
non-NULL-generating sides may have null values and they can not be filtered in 
advance.

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter for the inputs of inner join

2022-05-10 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-3890:
-
Summary: Derive IS NOT NULL filter for the inputs of inner join   (was: 
Derive IS NOT NULL filter from inner join)

> Derive IS NOT NULL filter for the inputs of inner join 
> ---
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Updated] (CALCITE-3890) Derive IS NOT NULL filter from inner join

2022-05-09 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei updated CALCITE-3890:
-
Summary: Derive IS NOT NULL filter from inner join  (was: Infer IS NOT NULL 
predicate from join)

> Derive IS NOT NULL filter from inner join
> -
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-3890) Infer IS NOT NULL predicate from join

2022-05-09 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534097#comment-17534097
 ] 

Chunwei Lei commented on CALCITE-3890:
--

I opened a PR for this feature: [https://github.com/apache/calcite/pull/2800.] 
It would be great if someone can review it.

> Infer IS NOT NULL predicate from join
> -
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Vineet Garg
>Priority: Major
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Assigned] (CALCITE-3890) Infer IS NOT NULL predicate from join

2022-05-09 Thread Chunwei Lei (Jira)


 [ 
https://issues.apache.org/jira/browse/CALCITE-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chunwei Lei reassigned CALCITE-3890:


Assignee: Chunwei Lei  (was: Vineet Garg)

> Infer IS NOT NULL predicate from join
> -
>
> Key: CALCITE-3890
> URL: https://issues.apache.org/jira/browse/CALCITE-3890
> Project: Calcite
>  Issue Type: Improvement
>  Components: core
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>
> We can infer IS NOT NULL predicate from join which implies some columns may 
> not be null. For instance, 
>  
> {code:java}
> select * from a join b on a.id = b.id;
> {code}
> we can infer a.id is not null and b.id is not null.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


[jira] [Commented] (CALCITE-5131) Remove redundant type cast

2022-05-09 Thread Chunwei Lei (Jira)


[ 
https://issues.apache.org/jira/browse/CALCITE-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534078#comment-17534078
 ] 

Chunwei Lei commented on CALCITE-5131:
--

Thank you for your trust. I will merge the PR if no other comments in the next 
24 hours.

> Remove redundant type cast
> --
>
> Key: CALCITE-5131
> URL: https://issues.apache.org/jira/browse/CALCITE-5131
> Project: Calcite
>  Issue Type: Improvement
>Reporter: Chunwei Lei
>Assignee: Chunwei Lei
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2022-05-05-11-03-26-735.png, index.html, 
> script.js, styles.css
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There're a number of redundant type casts. I suggest removing them.
> !image-2022-05-05-11-03-26-735.png!
>  
> What do you think? BTW, it seems the CheckStyle plugin doesn't provide such a 
> check. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)


  1   2   3   4   5   6   7   8   9   >