[jira] [Updated] (CALCITE-3201) SqlValidator throws exception for sql insert target table with virtual columns
[
https://issues.apache.org/jira/browse/CALCITE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3201:
Summary: SqlValidator throws exception for sql insert target table with
virtual columns (was: Skip validation for sql insert columns with column
strategy that can not insert into)
> SqlValidator throws exception for sql insert target table with virtual columns
> --
>
> Key: CALCITE-3201
> URL: https://issues.apache.org/jira/browse/CALCITE-3201
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> Now if we have a target table with schema:
> {code:sql}
> t1
> : - a int not null
> - b bigint STORED
> - c varchar VIRTUAL
> {code}
> and validate a query like:
> {code:sql}
> insert into t1 select a from t1;
> {code}
> It would throw exception because the target table fields count is 3 but the
> query is 1,
> actually the table t1 columns b and c have strategy that we can not insert
> into, so we better
> ignore these columns when do validation.
> The following query in SQLSERVER-2017 passed:
> {code:sql}
> create table t(
> a int,
> b int,
> c as a+2
> );
> -- insert into t values(1, 2, 3) fails with non-equal schema fields count.
> insert into t values(1, 2);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (CALCITE-3201) Skip validation for sql insert columns with column strategy that can not insert into
[
https://issues.apache.org/jira/browse/CALCITE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3201:
Description:
Now if we have a target table with schema:
{code:sql}
t1
: - a int not null
- b bigint STORED
- c varchar VIRTUAL
{code}
and validate a query like:
{code:sql}
insert into t1 select a from t1;
{code}
It would throw exception because the target table fields count is 3 but the
query is 1,
actually the table t1 columns b and c have strategy that we can not insert
into, so we better
ignore these columns when do validation.
The following query in SQLSERVER-2017 passed:
{code:sql}
create table t(
a int,
b int,
c as a+2
);
-- insert into t values(1, 2, 3) fails with non-equal schema fields count.
insert into t values(1, 2);
{code}
was:
Now if we have a target table with schema:
{code:sql}
t1
: - a int not null
- b bigint STORED
- c varchar VIRTUAL
{code}
and validate a query like:
{code:sql}
insert into t1 select a from t1;
{code}
It would throw exception because the target table fields count is 3 but the
query is 1,
actually the table t1 columns b and c have strategy that we can not insert
into, so we better
ignore these columns when do validation
The following query in SQLSERVER-2017 passed:
{code:sql}
create table t(
a int,
b int,
c as a+2
);
insert into t values(1, 2); // insert into t values(1, 2, 3) fails with
non-equal schema fields count.
{code}
> Skip validation for sql insert columns with column strategy that can not
> insert into
>
>
> Key: CALCITE-3201
> URL: https://issues.apache.org/jira/browse/CALCITE-3201
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> Now if we have a target table with schema:
> {code:sql}
> t1
> : - a int not null
> - b bigint STORED
> - c varchar VIRTUAL
> {code}
> and validate a query like:
> {code:sql}
> insert into t1 select a from t1;
> {code}
> It would throw exception because the target table fields count is 3 but the
> query is 1,
> actually the table t1 columns b and c have strategy that we can not insert
> into, so we better
> ignore these columns when do validation.
> The following query in SQLSERVER-2017 passed:
> {code:sql}
> create table t(
> a int,
> b int,
> c as a+2
> );
> -- insert into t values(1, 2, 3) fails with non-equal schema fields count.
> insert into t values(1, 2);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (CALCITE-3201) Skip validation for sql insert columns with column strategy that can not insert into
[
https://issues.apache.org/jira/browse/CALCITE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3201:
Description:
Now if we have a target table with schema:
{code:sql}
t1
: - a int not null
- b bigint STORED
- c varchar VIRTUAL
{code}
and validate a query like:
{code:sql}
insert into t1 select a from t1;
{code}
It would throw exception because the target table fields count is 3 but the
query is 1,
actually the table t1 columns b and c have strategy that we can not insert
into, so we better
ignore these columns when do validation
The following query in SQLSERVER-2017 passed:
{code:sql}
create table t(
a int,
b int,
c as a+2
);
insert into t values(1, 2); // insert into t values(1, 2, 3) fails with
non-equal schema fields count.
{code}
was:
Now if we have a target table with schema:
{code:sql}
t1
: - a int not null
- b bigint STORED
- c varchar VIRTUAL
{code}
and validate a query like:
{code:sql}
insert into t1 select a from t1;
{code}
It would throw exception because the target table fields count is 3 but the
query is 1,
actually the table t1 columns b and c have strategy that we can not insert
into, so we better
ignore these columns when do validation
> Skip validation for sql insert columns with column strategy that can not
> insert into
>
>
> Key: CALCITE-3201
> URL: https://issues.apache.org/jira/browse/CALCITE-3201
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> Now if we have a target table with schema:
> {code:sql}
> t1
> : - a int not null
> - b bigint STORED
> - c varchar VIRTUAL
> {code}
> and validate a query like:
> {code:sql}
> insert into t1 select a from t1;
> {code}
> It would throw exception because the target table fields count is 3 but the
> query is 1,
> actually the table t1 columns b and c have strategy that we can not insert
> into, so we better
> ignore these columns when do validation
> The following query in SQLSERVER-2017 passed:
> {code:sql}
> create table t(
> a int,
> b int,
> c as a+2
> );
> insert into t values(1, 2); // insert into t values(1, 2, 3) fails with
> non-equal schema fields count.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (CALCITE-3201) Skip validation for sql insert columns with column strategy that can not insert into
Danny Chan created CALCITE-3201:
---
Summary: Skip validation for sql insert columns with column
strategy that can not insert into
Key: CALCITE-3201
URL: https://issues.apache.org/jira/browse/CALCITE-3201
Project: Calcite
Issue Type: Improvement
Components: core
Affects Versions: 1.20.0
Reporter: Danny Chan
Assignee: Danny Chan
Fix For: 1.21.0
Now if we have a target table with schema:
{code:sql}
t1
: - a int not null
- b bigint STORED
- c varchar VIRTUAL
{code}
and validate a query like:
{code:sql}
insert into t1 select a from t1;
{code}
It would throw exception because the target table fields count is 3 but the
query is 1,
actually the table t1 columns b and c have strategy that we can not insert
into, so we better
ignore these columns when do validation
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (CALCITE-3113) Equivalent MutableAggregates with different row types fail with AssertionError
[
https://issues.apache.org/jira/browse/CALCITE-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886908#comment-16886908
]
Danny Chan commented on CALCITE-3113:
-
Fixed in
[0a87daf|https://github.com/apache/calcite/commit/0a87daf30d635277b8f1c94355e04ef907fc4002],
thanks for your PR, [~jinxing6...@126.com] !
> Equivalent MutableAggregates with different row types fail with AssertionError
> --
>
> Key: CALCITE-3113
> URL: https://issues.apache.org/jira/browse/CALCITE-3113
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Add test case in MaterializationTest:
> {code:java}
> @Test public void testAggregateAlias() {
> checkMaterialize(
> "select count(*) as c from \"emps\" group by \"empid\"",
> "select count(*) + 1 as c from \"emps\" group by \"empid\"");
> }
> {code}
> It fails due to different rowtype.
> {code:java}
> java.lang.AssertionError
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:504)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:465)
> at
> org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56)
> at
> org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200)
> at
> org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:347)
> {code}
> However, according to MutableAggregate's hashCode implementation, this
> materialization can be reused, i.e., queryDescedant=targetDescendant.
> {code:java}
> queryDescendant: RecordType(JavaType(int) empid, BIGINT $f1)
> =
> Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()])
> Project(projects: [$0])
> Scan(table: [hr, emps])
> targetDescendant: RecordType(JavaType(int) empid, BIGINT C)
> =
> Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()])
> Project(projects: [$0])
> Scan(table: [hr, emps])
> {code}
> So, how can we align them?
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (CALCITE-3113) Equivalent MutableAggregates with different row types fail with AssertionError
[
https://issues.apache.org/jira/browse/CALCITE-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3113.
-
Resolution: Fixed
Assignee: Danny Chan
Fix Version/s: 1.21.0
> Equivalent MutableAggregates with different row types fail with AssertionError
> --
>
> Key: CALCITE-3113
> URL: https://issues.apache.org/jira/browse/CALCITE-3113
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: Feng Zhu
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Add test case in MaterializationTest:
> {code:java}
> @Test public void testAggregateAlias() {
> checkMaterialize(
> "select count(*) as c from \"emps\" group by \"empid\"",
> "select count(*) + 1 as c from \"emps\" group by \"empid\"");
> }
> {code}
> It fails due to different rowtype.
> {code:java}
> java.lang.AssertionError
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:504)
> at
> org.apache.calcite.plan.SubstitutionVisitor.go(SubstitutionVisitor.java:465)
> at
> org.apache.calcite.plan.MaterializedViewSubstitutionVisitor.go(MaterializedViewSubstitutionVisitor.java:56)
> at
> org.apache.calcite.plan.RelOptMaterializations.substitute(RelOptMaterializations.java:200)
> at
> org.apache.calcite.plan.RelOptMaterializations.useMaterializedViews(RelOptMaterializations.java:72)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.registerMaterializations(VolcanoPlanner.java:347)
> {code}
> However, according to MutableAggregate's hashCode implementation, this
> materialization can be reused, i.e., queryDescedant=targetDescendant.
> {code:java}
> queryDescendant: RecordType(JavaType(int) empid, BIGINT $f1)
> =
> Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()])
> Project(projects: [$0])
> Scan(table: [hr, emps])
> targetDescendant: RecordType(JavaType(int) empid, BIGINT C)
> =
> Aggregate(groupSet: {0}, groupSets: [{0}], calls: [COUNT()])
> Project(projects: [$0])
> Scan(table: [hr, emps])
> {code}
> So, how can we align them?
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (CALCITE-3186) IN expressions in UPDATE statements throws Exceptions
[
https://issues.apache.org/jira/browse/CALCITE-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3186:
Summary: IN expressions in UPDATE statements throws Exceptions (was: IN
expressions in UPDATE statements throwing Exceptions)
> IN expressions in UPDATE statements throws Exceptions
> -
>
> Key: CALCITE-3186
> URL: https://issues.apache.org/jira/browse/CALCITE-3186
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.20.0
>Reporter: Pressenna
>Priority: Major
>
> The patch to get correlated sub-queries working in UPDATE statements had this
> side-effect.
>
> {code:java}
> CREATE TABLE t1 (id1 INT, val1 TEXT);
> CREATE TABLE t2 (id2 INT, val2 TEXT);
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (1, 2, 3);
> -- or
> UPDATE t1 SET val1 = 't2' WHERE id1 IN (SELECT id2 FROM t2);{code}
>
> These seem to raise exceptions now.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (CALCITE-3196) In Frameworks, add BasePrepareAction (a functional interface) and deprecate PrepareAction
[
https://issues.apache.org/jira/browse/CALCITE-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3196:
---
Assignee: Danny Chan
> In Frameworks, add BasePrepareAction (a functional interface) and deprecate
> PrepareAction
> -
>
> Key: CALCITE-3196
> URL: https://issues.apache.org/jira/browse/CALCITE-3196
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Danny Chan
>Priority: Major
>
> In {{Frameworks}}, add {{interface BasePrepareAction}} (a functional
> interface) and deprecate {{abstract class PrepareAction}}. Because
> {{PrepareAction}} has a field ({{FrameworkConfig config}}), it cannot be
> implemented using a lambda. It is simpler and clearer to pass {{config}} as
> an argument to all methods where it is needed.
> {{PrepareAction}} was introduced in CALCITE-247.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (CALCITE-3155) AssertionError in RelToSqlConverter when visiting empty Values node
[
https://issues.apache.org/jira/browse/CALCITE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883461#comment-16883461
]
Danny Chan commented on CALCITE-3155:
-
[~julianhyde] If the whole `values` statement is special enough to have
limitation for some dialects, maybe we should add another control flag in
SqlDialect like supportsValues ? Instead of using values, maybe we could use
subquery instead:
{code:sql}
select * from (
select null as c0, null as c1
) t1 where 1=0
{code}
> AssertionError in RelToSqlConverter when visiting empty Values node
> ---
>
> Key: CALCITE-3155
> URL: https://issues.apache.org/jira/browse/CALCITE-3155
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Musbah EL FIL
>Assignee: Danny Chan
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Given the query:
> *select A."department_id" from "foodmart"."employee" A where
> A."department_id" = ( select min( A."department_id") from
> "foodmart"."department" B where 1=2 )*
> The conversion from SqlNode to RelNode is correct, however when going from
> RelNode to SqlNode, an exception was being thrown.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (CALCITE-3188) IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT COUNT(*)
[
https://issues.apache.org/jira/browse/CALCITE-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3188.
-
Resolution: Fixed
Fixed in
[954f27c|https://github.com/apache/calcite/commit/954f27c2a09e019dd22d689ddd770ffb42c1c10c]!
> IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT
> COUNT(*)
> --
>
> Key: CALCITE-3188
> URL: https://issues.apache.org/jira/browse/CALCITE-3188
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The exception is raised by many tests in {{DruidAdapterIT}}.
> What they all have in common is a query of the following form:
> {code:java}
> SELECT COUNT(*) FROM ...
> {code}
> where the query projects only a single column.
> The stacktrace for {{DruidAdapterIT#testExpressionsFilterWithExtract}} is the
> following.
> {noformat}
> Caused by: java.lang.RuntimeException: Error while applying rule
> DruidProjectFilterTransposeRule, args
> [rel#226:LogicalProject.NONE.[](input=RelSubset#50),
> rel#83:BindableFilter.BINDABLE.[](input=RelSubset#18,condition==(CAST(+(/(-(EXTRACT(FLAG(MONTH),
> $0), 1), 3), 1)):INTEGER NOT NULL, 1)),
> rel#1:DruidQuery.BINDABLE.[](table=[foodmart,
> foodmart],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z])]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:283)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:343)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:320)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:637)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:501)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:471)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 28 more
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:102)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 41 more
> {noformat}
> It is regression that must be solved before the release.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Issue Comment Deleted] (CALCITE-3188) IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT COUNT(*)
[
https://issues.apache.org/jira/browse/CALCITE-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3188:
Comment: was deleted
(was: [~zabetak], thanks for reporting this, i tried but don't know how to make
a rel of struct row type but with empty field list, cause this bug has been
fixed, i would merge the PR soon if the tests are all passed.)
> IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT
> COUNT(*)
> --
>
> Key: CALCITE-3188
> URL: https://issues.apache.org/jira/browse/CALCITE-3188
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The exception is raised by many tests in {{DruidAdapterIT}}.
> What they all have in common is a query of the following form:
> {code:java}
> SELECT COUNT(*) FROM ...
> {code}
> where the query projects only a single column.
> The stacktrace for {{DruidAdapterIT#testExpressionsFilterWithExtract}} is the
> following.
> {noformat}
> Caused by: java.lang.RuntimeException: Error while applying rule
> DruidProjectFilterTransposeRule, args
> [rel#226:LogicalProject.NONE.[](input=RelSubset#50),
> rel#83:BindableFilter.BINDABLE.[](input=RelSubset#18,condition==(CAST(+(/(-(EXTRACT(FLAG(MONTH),
> $0), 1), 3), 1)):INTEGER NOT NULL, 1)),
> rel#1:DruidQuery.BINDABLE.[](table=[foodmart,
> foodmart],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z])]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:283)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:343)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:320)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:637)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:501)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:471)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 28 more
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:102)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 41 more
> {noformat}
> It is regression that must be solved before the release.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3188) IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT COUNT(*)
[
https://issues.apache.org/jira/browse/CALCITE-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882638#comment-16882638
]
Danny Chan commented on CALCITE-3188:
-
[~zabetak], thanks for reporting this, i tried but don't know how to make a rel
of struct row type but with empty field list, cause this bug has been fixed, i
would merge the PR soon if the tests are all passed.
> IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT
> COUNT(*)
> --
>
> Key: CALCITE-3188
> URL: https://issues.apache.org/jira/browse/CALCITE-3188
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The exception is raised by many tests in {{DruidAdapterIT}}.
> What they all have in common is a query of the following form:
> {code:java}
> SELECT COUNT(*) FROM ...
> {code}
> where the query projects only a single column.
> The stacktrace for {{DruidAdapterIT#testExpressionsFilterWithExtract}} is the
> following.
> {noformat}
> Caused by: java.lang.RuntimeException: Error while applying rule
> DruidProjectFilterTransposeRule, args
> [rel#226:LogicalProject.NONE.[](input=RelSubset#50),
> rel#83:BindableFilter.BINDABLE.[](input=RelSubset#18,condition==(CAST(+(/(-(EXTRACT(FLAG(MONTH),
> $0), 1), 3), 1)):INTEGER NOT NULL, 1)),
> rel#1:DruidQuery.BINDABLE.[](table=[foodmart,
> foodmart],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z])]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:283)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:343)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:320)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:637)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:501)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:471)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 28 more
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:102)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 41 more
> {noformat}
> It is regression that must be solved before the release.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3188) IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT COUNT(*)
[
https://issues.apache.org/jira/browse/CALCITE-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882042#comment-16882042
]
Danny Chan commented on CALCITE-3188:
-
[~zabetak] It seems that the row type is a struct type but with empty field
list [1], i'm a little curious about how can this happen ? I'm trying to
reproduce this in Calcite
[1]
https://github.com/apache/calcite/blob/72c52f46eab7fbe57ae5236ecc3831113684ff71/core/src/main/java/org/apache/calcite/rel/type/RelDataTypeImpl.java#L146
> IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT
> COUNT(*)
> --
>
> Key: CALCITE-3188
> URL: https://issues.apache.org/jira/browse/CALCITE-3188
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Assignee: Danny Chan
>Priority: Blocker
> Fix For: 1.21.0
>
>
> The exception is raised by many tests in {{DruidAdapterIT}}.
> What they all have in common is a query of the following form:
> {code:java}
> SELECT COUNT(*) FROM ...
> {code}
> where the query projects only a single column.
> The stacktrace for {{DruidAdapterIT#testExpressionsFilterWithExtract}} is the
> following.
> {noformat}
> Caused by: java.lang.RuntimeException: Error while applying rule
> DruidProjectFilterTransposeRule, args
> [rel#226:LogicalProject.NONE.[](input=RelSubset#50),
> rel#83:BindableFilter.BINDABLE.[](input=RelSubset#18,condition==(CAST(+(/(-(EXTRACT(FLAG(MONTH),
> $0), 1), 3), 1)):INTEGER NOT NULL, 1)),
> rel#1:DruidQuery.BINDABLE.[](table=[foodmart,
> foodmart],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z])]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:283)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:343)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:320)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:637)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:501)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:471)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 28 more
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:102)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 41 more
> {noformat}
> It is regression that must be solved before the release.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-3188) IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT COUNT(*)
[
https://issues.apache.org/jira/browse/CALCITE-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3188:
---
Assignee: Danny Chan
> IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT
> COUNT(*)
> --
>
> Key: CALCITE-3188
> URL: https://issues.apache.org/jira/browse/CALCITE-3188
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Assignee: Danny Chan
>Priority: Blocker
> Fix For: 1.21.0
>
>
> The exception is raised by many tests in {{DruidAdapterIT}}.
> What they all have in common is a query of the following form:
> {code:java}
> SELECT COUNT(*) FROM ...
> {code}
> where the query projects only a single column.
> The stacktrace for {{DruidAdapterIT#testExpressionsFilterWithExtract}} is the
> following.
> {noformat}
> Caused by: java.lang.RuntimeException: Error while applying rule
> DruidProjectFilterTransposeRule, args
> [rel#226:LogicalProject.NONE.[](input=RelSubset#50),
> rel#83:BindableFilter.BINDABLE.[](input=RelSubset#18,condition==(CAST(+(/(-(EXTRACT(FLAG(MONTH),
> $0), 1), 3), 1)):INTEGER NOT NULL, 1)),
> rel#1:DruidQuery.BINDABLE.[](table=[foodmart,
> foodmart],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z])]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:283)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:343)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:320)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:637)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:501)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:471)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 28 more
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:102)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 41 more
> {noformat}
> It is regression that must be solved before the release.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3188) IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT COUNT(*)
[
https://issues.apache.org/jira/browse/CALCITE-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882015#comment-16882015
]
Danny Chan commented on CALCITE-3188:
-
Thanks [~zabetak], i will take a look !
> IndexOutOfBoundsException in ProjectFilterTransposeRule when executing SELECT
> COUNT(*)
> --
>
> Key: CALCITE-3188
> URL: https://issues.apache.org/jira/browse/CALCITE-3188
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Priority: Blocker
> Fix For: 1.21.0
>
>
> The exception is raised by many tests in {{DruidAdapterIT}}.
> What they all have in common is a query of the following form:
> {code:java}
> SELECT COUNT(*) FROM ...
> {code}
> where the query projects only a single column.
> The stacktrace for {{DruidAdapterIT#testExpressionsFilterWithExtract}} is the
> following.
> {noformat}
> Caused by: java.lang.RuntimeException: Error while applying rule
> DruidProjectFilterTransposeRule, args
> [rel#226:LogicalProject.NONE.[](input=RelSubset#50),
> rel#83:BindableFilter.BINDABLE.[](input=RelSubset#18,condition==(CAST(+(/(-(EXTRACT(FLAG(MONTH),
> $0), 1), 3), 1)):INTEGER NOT NULL, 1)),
> rel#1:DruidQuery.BINDABLE.[](table=[foodmart,
> foodmart],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z])]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:235)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:631)
> at
> org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:283)
> at
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:343)
> at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:189)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:320)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:637)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:501)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:471)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:550)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> ... 28 more
> Caused by: java.lang.IndexOutOfBoundsException: index (0) must be less than
> size (0)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310)
> at
> com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293)
> at
> com.google.common.collect.RegularImmutableList.get(RegularImmutableList.java:67)
> at
> org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:102)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:208)
> ... 41 more
> {noformat}
> It is regression that must be solved before the release.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3187) Derive all decimal return type through type factory
[ https://issues.apache.org/jira/browse/CALCITE-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882006#comment-16882006 ] Danny Chan commented on CALCITE-3187: - You can use the operators table of your own(not the StdOperatorTable), and customize the return type inference of the operator you want. > Derive all decimal return type through type factory > --- > > Key: CALCITE-3187 > URL: https://issues.apache.org/jira/browse/CALCITE-3187 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Praveen Kumar Desabandu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently decimal product and quotient return types are derived through type > factory, this allows clients to override the return type if they so desire. > But decimal sum is embedded in return types, also decimal mod does not have a > return type inference defined. > This task is to derive all of the return types through type factory, so that > clients can override if they wish to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3187) Derive all decimal return type through type factory
[ https://issues.apache.org/jira/browse/CALCITE-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881752#comment-16881752 ] Danny Chan commented on CALCITE-3187: - The `SqlReturnTypeInference DECIMAL_SUM` itself is designed to be pluggable, and it is used to infer return types of many operators, so why we need another way (the type factory) for overriding the inferring logic ? Shouldn't we define another `SqlReturnTypeInference` and use it where we want ? > Derive all decimal return type through type factory > --- > > Key: CALCITE-3187 > URL: https://issues.apache.org/jira/browse/CALCITE-3187 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Praveen Kumar Desabandu >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Currently decimal product and quotient return types are derived through type > factory, this allows clients to override the return type if they so desire. > But decimal sum is embedded in return types, also decimal mod does not have a > return type inference defined. > This task is to derive all of the return types through type factory, so that > clients can override if they wish to. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (CALCITE-3083) Add demos(how to) doc for all supported DDLs and DMLs
[
https://issues.apache.org/jira/browse/CALCITE-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3083:
---
Assignee: (was: Danny Chan)
> Add demos(how to) doc for all supported DDLs and DMLs
> -
>
> Key: CALCITE-3083
> URL: https://issues.apache.org/jira/browse/CALCITE-3083
> Project: Calcite
> Issue Type: Bug
> Components: site
>Affects Versions: 1.19.0
>Reporter: Danny Chan
>Priority: Major
>
> Copied from DEV mail:
> Dear Dev Team's Apache Calcite,
> My name is Duc. I'm working at Toshiba Group in Vietnam.
> I'm implementing how to parser DDL file using library Apache Calcite.
> I'm trying with below solution, but has exception is thrown and i don't know
> that why.
>
> {code:java}
> InputStream ddlFile = new FileInputStream("ddlTest.sql");
> SqlDdlParserImpl ddlParser = new SqlDdlParserImpl(ddlFile,"UTF8");
> SqlNode node = ddlParser.SqlStmt(); {code}
> Could you please send me a example of DDL file or give me a suggestion about
> this proplem.
>
> We indeed need detail demos about our grammars, that will let users to some
> fast start up and our site more user friendly.
>
> Some artifacts from [~michaelmior] [1]
> [1] [https://github.com/michaelmior/calcite-notebooks]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3184) Add the new logo to the website
[ https://issues.apache.org/jira/browse/CALCITE-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880845#comment-16880845 ] Danny Chan commented on CALCITE-3184: - In out mailing list, the font of the logo are color dark grey, but on the website, it is white color ? > Add the new logo to the website > --- > > Key: CALCITE-3184 > URL: https://issues.apache.org/jira/browse/CALCITE-3184 > Project: Calcite > Issue Type: Task > Components: site >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Finalize and add the new logo to the website as discussed in the dev list > [[1|https://lists.apache.org/thread.html/582b4f14dcce117594abba7f36e46725afaa811194def510c246c909@%3Cdev.calcite.apache.org%3E], > > [2|https://lists.apache.org/thread.html/d6138f938fad7402376718ade286aae50d12d04b63d8ead78685830b@%3Cdev.calcite.apache.org%3E]]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3170) ANTI join on conditions push down generates wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3170.
-
Resolution: Fixed
Fix Version/s: 1.21.0
Fixed in
[cb364ce|https://github.com/apache/calcite/commit/cb364ce8109f92c80c17bb37d1cb1a03e6222bbe]
!
> ANTI join on conditions push down generates wrong plan
> --
>
> Key: CALCITE-3170
> URL: https://issues.apache.org/jira/browse/CALCITE-3170
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> create table A (a int, b int);
> create table B (c int, d int);
> insert into A values (1, 1);
> insert into A values (2, 2);
> sql:
> {code:sql}
> select * from A where A.a not in (select B.c from B where A.b > 1);
> {code}
> the equivalent logical tree:
> {code}
> LogicalProject(a=[$0], b=[$1])
> +- LogicalProject(a=[$0], b=[$1])
>+- LogicalJoin(condition=[AND(OR(=($0, $3), IS NULL($0), IS NULL($3)),
> $2)], joinType=[anti])
> :- LogicalProject(a=[$0], b=[$1], $f2=[>($1, 1)])
> : +- LogicalTableScan(table=[[A]])
> +- LogicalProject(c=[$0])
> +- LogicalFilter(condition=[true])
> +- LogicalTableScan(table=[[B]])
> {code}
> the correct result is: (1, 1), (2, 2)
> while if the predicate ($2 in join condition) is pushed into left side, the
> result is (2, 2) which is incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3169) RelDecorrelator should return for SEMI/ANTI join decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3169:
Summary: RelDecorrelator should return for SEMI/ANTI join decorrelate
(was: RelDecorrelator should return for SEMI/ANTI join decorrelation)
> RelDecorrelator should return for SEMI/ANTI join decorrelate
>
>
> Key: CALCITE-3169
> URL: https://issues.apache.org/jira/browse/CALCITE-3169
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> /**
>* Rewrite LogicalJoin.
>*
>* @param rel Join
>*/
> public Frame decorrelateRel(LogicalJoin rel) {
> // For SEMI/ANTI join decorrelate it's input directly,
> // because the correlate variables can only be propagated from
> // the left side, which is not supported yet.
> if (!rel.getJoinType().projectsRight()) {
> decorrelateRel((RelNode) rel);
> }
> ...
> {code}
> {{return}} is missing before {{decorrelateRel((RelNode) rel);}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3169) RelDecorrelator should return for SEMI/ANTI join decorrelation
[
https://issues.apache.org/jira/browse/CALCITE-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3169:
Summary: RelDecorrelator should return for SEMI/ANTI join decorrelation
(was: decorrelateRel method should return when meeting SEMI/ANTI join in
RelDecorrelator)
> RelDecorrelator should return for SEMI/ANTI join decorrelation
> --
>
> Key: CALCITE-3169
> URL: https://issues.apache.org/jira/browse/CALCITE-3169
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> /**
>* Rewrite LogicalJoin.
>*
>* @param rel Join
>*/
> public Frame decorrelateRel(LogicalJoin rel) {
> // For SEMI/ANTI join decorrelate it's input directly,
> // because the correlate variables can only be propagated from
> // the left side, which is not supported yet.
> if (!rel.getJoinType().projectsRight()) {
> decorrelateRel((RelNode) rel);
> }
> ...
> {code}
> {{return}} is missing before {{decorrelateRel((RelNode) rel);}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-3169) decorrelateRel method should return when meeting SEMI/ANTI join in RelDecorrelator
[
https://issues.apache.org/jira/browse/CALCITE-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3169.
-
Resolution: Fixed
Fixed in
[e723ee9|https://github.com/apache/calcite/commit/e723ee943f575823db5c8c155236368fc6bb00e2],
thanks for your PR, [~godfreyhe] !
> decorrelateRel method should return when meeting SEMI/ANTI join in
> RelDecorrelator
> --
>
> Key: CALCITE-3169
> URL: https://issues.apache.org/jira/browse/CALCITE-3169
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> /**
>* Rewrite LogicalJoin.
>*
>* @param rel Join
>*/
> public Frame decorrelateRel(LogicalJoin rel) {
> // For SEMI/ANTI join decorrelate it's input directly,
> // because the correlate variables can only be propagated from
> // the left side, which is not supported yet.
> if (!rel.getJoinType().projectsRight()) {
> decorrelateRel((RelNode) rel);
> }
> ...
> {code}
> {{return}} is missing before {{decorrelateRel((RelNode) rel);}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-3169) decorrelateRel method should return when meeting SEMI/ANTI join in RelDecorrelator
[
https://issues.apache.org/jira/browse/CALCITE-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3169:
---
Assignee: Danny Chan
> decorrelateRel method should return when meeting SEMI/ANTI join in
> RelDecorrelator
> --
>
> Key: CALCITE-3169
> URL: https://issues.apache.org/jira/browse/CALCITE-3169
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> /**
>* Rewrite LogicalJoin.
>*
>* @param rel Join
>*/
> public Frame decorrelateRel(LogicalJoin rel) {
> // For SEMI/ANTI join decorrelate it's input directly,
> // because the correlate variables can only be propagated from
> // the left side, which is not supported yet.
> if (!rel.getJoinType().projectsRight()) {
> decorrelateRel((RelNode) rel);
> }
> ...
> {code}
> {{return}} is missing before {{decorrelateRel((RelNode) rel);}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-3171) SemiJoin on conditions push down throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CALCITE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3171.
-
Resolution: Fixed
Fix Version/s: 1.21.0
Fixed in
[e1875e3|https://github.com/apache/calcite/commit/e1875e3f78d91711ba174834cdead500361b19e6],
thanks for reporting this, [~godfreyhe] !
> SemiJoin on conditions push down throws IndexOutOfBoundsException
> -
>
> Key: CALCITE-3171
> URL: https://issues.apache.org/jira/browse/CALCITE-3171
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> the following code in RelOptUtil#pushDownJoinConditions only does not
> consider SEMI/ANTI join
> {code:java}
> if (!extraLeftExprs.isEmpty() || !extraRightExprs.isEmpty()) {
> Mappings.TargetMapping mapping =
> Mappings.createShiftMapping(
> leftCount + extraLeftExprs.size()
> + rightCount + extraRightExprs.size(),
> 0, 0, leftCount,
> leftCount, leftCount + extraLeftExprs.size(), rightCount);
> relBuilder.project(relBuilder.fields(mapping.inverse()));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2843) Babel parser should parse PostgreSQL-style '::' cast operator
[
https://issues.apache.org/jira/browse/CALCITE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880063#comment-16880063
]
Danny Chan edited comment on CALCITE-2843 at 7/8/19 6:50 AM:
-
Thanks, [~mgelbana], lets try {{SqlStdOperatorTable.CAST}} first in the parser
then ping me again for the review ~
BTW, i only saw the tests for casting as int, correct me if i missed something.
was (Author: danny0405):
Thanks, [~mgelbana], lets try {{SqlStdOperatorTable.CAST}} first in the parser
then ping me again for the review ~
> Babel parser should parse PostgreSQL-style '::' cast operator
> -
>
> Key: CALCITE-2843
> URL: https://issues.apache.org/jira/browse/CALCITE-2843
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Affects Versions: 1.18.0
>Reporter: Muhammad Gelbana
>Assignee: Muhammad Gelbana
>Priority: Major
> Labels: postgresql, pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *Code to reproduce the problem*
> {code:java}
> public static void main(String[] args) throws Exception {
> Config parserConfig =
> configBuilder().setConformance(SqlConformanceEnum.BABEL).setParserFactory(SqlBabelParserImpl.FACTORY).build();
> FrameworkConfig frameworkConfig =
> Frameworks.newConfigBuilder().parserConfig(parserConfig).build();
> Planner planner = Frameworks.getPlanner(frameworkConfig);
> String pg = "SELECT 'array_in'::regproc, typtype FROM pg_catalog.pg_type";
> planner.parse(pg);
> }{code}
>
> *Thrown exception*
> {noformat}
> Exception in thread "main" org.apache.calcite.sql.parser.SqlParseException:
> Encountered ":" at line 1, column 18.
> Was expecting one of:
>
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "FROM" ...
> "," ...
> ...
> ...
> ...
> ...
> ...
> ...
> "." ...
> "IN" ...
> "<" ...
> "<=" ...
> ">" ...
> ">=" ...
> "=" ...
> "<>" ...
> "!=" ...
> "+" ...
> "-" ...
> "*" ...
> "/" ...
> "%" ...
> "||" ...
> "[" ...
> "UNION" ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
>
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.convertException(SqlBabelParserImpl.java:354)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.normalizeException(SqlBabelParserImpl.java:142)
> at
> org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:156)
> at org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:181)
> at org.apache.calcite.prepare.PlannerImpl.parse(PlannerImpl.java:174)
> at org.apache.calcite.tools.Planner.parse(Planner.java:50)
> at com.lab.calcite.App2.main(App2.java:23)
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered
> ":" at line 1, column 18.
> Was expecting one of:
>
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "FROM" ...
> "," ...
> ...
> ...
> ...
> ...
> ...
> ...
> "." ...
> "IN" ...
> "<" ...
> "<=" ...
> ">" ...
> ">=" ...
> "=" ...
> "<>" ...
> "!=" ...
> "+" ...
> "-" ...
> "*" ...
> "/" ...
> "%" ...
> "||" ...
> "[" ...
> "UNION" ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
>
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.generateParseException(SqlBabelParserImpl.java:31191)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.jj_consume_token(SqlBabelParserImpl.java:31008)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.SqlStmtEof(SqlBabelParserImpl.java:877)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.parseSqlStmtEof(SqlBabelParserImpl.java:198)
> at
> org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:148)
> ... 4 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2843) Babel parser should parse PostgreSQL-style '::' cast operator
[
https://issues.apache.org/jira/browse/CALCITE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880063#comment-16880063
]
Danny Chan commented on CALCITE-2843:
-
Thanks, [~mgelbana], lets try {{SqlStdOperatorTable.CAST}} first in the parser
then ping me again for the review ~
> Babel parser should parse PostgreSQL-style '::' cast operator
> -
>
> Key: CALCITE-2843
> URL: https://issues.apache.org/jira/browse/CALCITE-2843
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Affects Versions: 1.18.0
>Reporter: Muhammad Gelbana
>Assignee: Muhammad Gelbana
>Priority: Major
> Labels: postgresql, pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *Code to reproduce the problem*
> {code:java}
> public static void main(String[] args) throws Exception {
> Config parserConfig =
> configBuilder().setConformance(SqlConformanceEnum.BABEL).setParserFactory(SqlBabelParserImpl.FACTORY).build();
> FrameworkConfig frameworkConfig =
> Frameworks.newConfigBuilder().parserConfig(parserConfig).build();
> Planner planner = Frameworks.getPlanner(frameworkConfig);
> String pg = "SELECT 'array_in'::regproc, typtype FROM pg_catalog.pg_type";
> planner.parse(pg);
> }{code}
>
> *Thrown exception*
> {noformat}
> Exception in thread "main" org.apache.calcite.sql.parser.SqlParseException:
> Encountered ":" at line 1, column 18.
> Was expecting one of:
>
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "FROM" ...
> "," ...
> ...
> ...
> ...
> ...
> ...
> ...
> "." ...
> "IN" ...
> "<" ...
> "<=" ...
> ">" ...
> ">=" ...
> "=" ...
> "<>" ...
> "!=" ...
> "+" ...
> "-" ...
> "*" ...
> "/" ...
> "%" ...
> "||" ...
> "[" ...
> "UNION" ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
>
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.convertException(SqlBabelParserImpl.java:354)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.normalizeException(SqlBabelParserImpl.java:142)
> at
> org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:156)
> at org.apache.calcite.sql.parser.SqlParser.parseStmt(SqlParser.java:181)
> at org.apache.calcite.prepare.PlannerImpl.parse(PlannerImpl.java:174)
> at org.apache.calcite.tools.Planner.parse(Planner.java:50)
> at com.lab.calcite.App2.main(App2.java:23)
> Caused by: org.apache.calcite.sql.parser.babel.ParseException: Encountered
> ":" at line 1, column 18.
> Was expecting one of:
>
> "ORDER" ...
> "LIMIT" ...
> "OFFSET" ...
> "FETCH" ...
> "FROM" ...
> "," ...
> ...
> ...
> ...
> ...
> ...
> ...
> "." ...
> "IN" ...
> "<" ...
> "<=" ...
> ">" ...
> ">=" ...
> "=" ...
> "<>" ...
> "!=" ...
> "+" ...
> "-" ...
> "*" ...
> "/" ...
> "%" ...
> "||" ...
> "[" ...
> "UNION" ...
> "INTERSECT" ...
> "EXCEPT" ...
> "MINUS" ...
>
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.generateParseException(SqlBabelParserImpl.java:31191)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.jj_consume_token(SqlBabelParserImpl.java:31008)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.SqlStmtEof(SqlBabelParserImpl.java:877)
> at
> org.apache.calcite.sql.parser.babel.SqlBabelParserImpl.parseSqlStmtEof(SqlBabelParserImpl.java:198)
> at
> org.apache.calcite.sql.parser.SqlParser.parseQuery(SqlParser.java:148)
> ... 4 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3171) SemiJoin on conditions push down throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CALCITE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3171:
Summary: SemiJoin on conditions push down throws IndexOutOfBoundsException
(was: RelOptUtil#pushDownJoinConditions should handle SEMI/ANTI join specially
about project shift mapping)
> SemiJoin on conditions push down throws IndexOutOfBoundsException
> -
>
> Key: CALCITE-3171
> URL: https://issues.apache.org/jira/browse/CALCITE-3171
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: godfrey he
>Priority: Major
>
> the following code in RelOptUtil#pushDownJoinConditions only does not
> consider SEMI/ANTI join
> {code:java}
> if (!extraLeftExprs.isEmpty() || !extraRightExprs.isEmpty()) {
> Mappings.TargetMapping mapping =
> Mappings.createShiftMapping(
> leftCount + extraLeftExprs.size()
> + rightCount + extraRightExprs.size(),
> 0, 0, leftCount,
> leftCount, leftCount + extraLeftExprs.size(), rightCount);
> relBuilder.project(relBuilder.fields(mapping.inverse()));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-3171) SemiJoin on conditions push down throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CALCITE-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3171:
---
Assignee: Danny Chan
> SemiJoin on conditions push down throws IndexOutOfBoundsException
> -
>
> Key: CALCITE-3171
> URL: https://issues.apache.org/jira/browse/CALCITE-3171
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
>
> the following code in RelOptUtil#pushDownJoinConditions only does not
> consider SEMI/ANTI join
> {code:java}
> if (!extraLeftExprs.isEmpty() || !extraRightExprs.isEmpty()) {
> Mappings.TargetMapping mapping =
> Mappings.createShiftMapping(
> leftCount + extraLeftExprs.size()
> + rightCount + extraRightExprs.size(),
> 0, 0, leftCount,
> leftCount, leftCount + extraLeftExprs.size(), rightCount);
> relBuilder.project(relBuilder.fields(mapping.inverse()));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3170) ANTI join on conditions push down generates wrong plan
[
https://issues.apache.org/jira/browse/CALCITE-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3170:
Summary: ANTI join on conditions push down generates wrong plan (was: join
condition of ANTI join can not be pushed down)
> ANTI join on conditions push down generates wrong plan
> --
>
> Key: CALCITE-3170
> URL: https://issues.apache.org/jira/browse/CALCITE-3170
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
>
> create table A (a int, b int);
> create table B (c int, d int);
> insert into A values (1, 1);
> insert into A values (2, 2);
> sql:
> {code:sql}
> select * from A where A.a not in (select B.c from B where A.b > 1);
> {code}
> the equivalent logical tree:
> {code}
> LogicalProject(a=[$0], b=[$1])
> +- LogicalProject(a=[$0], b=[$1])
>+- LogicalJoin(condition=[AND(OR(=($0, $3), IS NULL($0), IS NULL($3)),
> $2)], joinType=[anti])
> :- LogicalProject(a=[$0], b=[$1], $f2=[>($1, 1)])
> : +- LogicalTableScan(table=[[A]])
> +- LogicalProject(c=[$0])
> +- LogicalFilter(condition=[true])
> +- LogicalTableScan(table=[[B]])
> {code}
> the correct result is: (1, 1), (2, 2)
> while if the predicate ($2 in join condition) is pushed into left side, the
> result is (2, 2) which is incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-3170) join condition of ANTI join can not be pushed down
[
https://issues.apache.org/jira/browse/CALCITE-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3170:
---
Assignee: Danny Chan
> join condition of ANTI join can not be pushed down
> --
>
> Key: CALCITE-3170
> URL: https://issues.apache.org/jira/browse/CALCITE-3170
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: godfrey he
>Assignee: Danny Chan
>Priority: Major
>
> create table A (a int, b int);
> create table B (c int, d int);
> insert into A values (1, 1);
> insert into A values (2, 2);
> sql:
> {code:sql}
> select * from A where A.a not in (select B.c from B where A.b > 1);
> {code}
> the equivalent logical tree:
> {code}
> LogicalProject(a=[$0], b=[$1])
> +- LogicalProject(a=[$0], b=[$1])
>+- LogicalJoin(condition=[AND(OR(=($0, $3), IS NULL($0), IS NULL($3)),
> $2)], joinType=[anti])
> :- LogicalProject(a=[$0], b=[$1], $f2=[>($1, 1)])
> : +- LogicalTableScan(table=[[A]])
> +- LogicalProject(c=[$0])
> +- LogicalFilter(condition=[true])
> +- LogicalTableScan(table=[[B]])
> {code}
> the correct result is: (1, 1), (2, 2)
> while if the predicate ($2 in join condition) is pushed into left side, the
> result is (2, 2) which is incorrect.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3100) cast(? as DATE) won't work with PreparedStatement
[
https://issues.apache.org/jira/browse/CALCITE-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879659#comment-16879659
]
Danny Chan commented on CALCITE-3100:
-
Thanks for you contribution [~yuqi], reviewing now ~~
You can send a mail to dev mailing list to ask for the JIRA contribution
permission :)
> cast(? as DATE) won't work with PreparedStatement
> -
>
> Key: CALCITE-3100
> URL: https://issues.apache.org/jira/browse/CALCITE-3100
> Project: Calcite
> Issue Type: Bug
>Reporter: Li Xian
>Assignee: Danny Chan
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When I do something like
>
> {code:java}
> Connection connection =
> DriverManager.getConnection("jdbc:calcite:", info);
> PreparedStatement ps = connection.prepareStatement("select * from EMPS where
> JOINEDAT=cast(? as DATE)");
> ps.setString(1, "1996-08-03");
> ps.execute();{code}
>
> It will produce error like below
>
> {code:java}
> java.sql.SQLException: exception while executing query: java.lang.String
> cannot be cast to java.lang.Integer
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:577)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.execute(AvaticaPreparedStatement.java:239)
> at org.apache.calcite.test.CsvTest.testDate(CsvTest.java:185)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Integer
> at Baz$1$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:675)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:95)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:184)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:573)
> ... 24 more
> {code}
>
> It looks like that the 'cast' clause doesn't make effect here with the '?'
> placeholder parameter.
> I am able to reproduce this error by adding the below testcase to
> org.apache.calcite.test.CsvTest and run it
> {code:java}
> @Test
> public void testDate() throws Exception {
> Properties info = new Properties();
> info.put("model", jsonPath("model"));
> Connection connection =
> DriverManager.getConnection("jdbc:calcite:", info);
> PreparedStatement ps = connection.prepareStatement("select * from EMPS
> where JOINEDAT=cast(? as DATE)");
> ps.setString(1, "1996-08-03");
> ps.execute();
> }
> {code}
--
This
[jira] [Resolved] (CALCITE-3121) VolcanoPlanner hangs due to subquery with dynamic star
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3121.
-
Resolution: Fixed
Fixed in
[0ed54bd|https://github.com/apache/calcite/commit/0ed54bd64d0beb0849a499e65e872c43f00efce9]
!
> VolcanoPlanner hangs due to subquery with dynamic star
> --
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3152) Unify throws in sql parser
[
https://issues.apache.org/jira/browse/CALCITE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3152:
Issue Type: Improvement (was: Bug)
> Unify throws in sql parser
> --
>
> Key: CALCITE-3152
> URL: https://issues.apache.org/jira/browse/CALCITE-3152
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Copied from DEV mail,
> Now our parser has 3 kinds of throws behavior:
>
> [1] Use JavaCC generateParseException
> [2] Use SqlUtil.newContextException
> [3] Use JavaCC ParseExecption directly
>
> For [1] and [2] there is a position info in the exception message, a throw
> may like:
> {code:java}
> From line 1, column 15 to line 1, column 26:
> {code}
> But for 3, we only have the error message without position info, which is not
> that user friendly when the sql text is huge (there are 10 occurance for 3 in
> our parser). So shall we unify them ? E.G. Use only 1 and 2 is enough for all
> the cases, the 2 can totally replace 3.
>
> [1]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
> [2]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
> [3]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-3168) Add test for invalid literal of sql parser
[ https://issues.apache.org/jira/browse/CALCITE-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3168. - Resolution: Fixed Fix Version/s: 1.21.0 Fixed in [021fabe|https://github.com/apache/calcite/commit/021fabe2a77264b388a3b681b78bc7e51fb2d978] ! > Add test for invalid literal of sql parser > -- > > Key: CALCITE-3168 > URL: https://issues.apache.org/jira/browse/CALCITE-3168 > Project: Calcite > Issue Type: Sub-task >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3152) Unify throws in sql parser
[
https://issues.apache.org/jira/browse/CALCITE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877435#comment-16877435
]
Danny Chan commented on CALCITE-3152:
-
Thanks, i have fired another issue CALCITE-3168, and will merge it when the
tests passed.
> Unify throws in sql parser
> --
>
> Key: CALCITE-3152
> URL: https://issues.apache.org/jira/browse/CALCITE-3152
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Copied from DEV mail,
> Now our parser has 3 kinds of throws behavior:
>
> [1] Use JavaCC generateParseException
> [2] Use SqlUtil.newContextException
> [3] Use JavaCC ParseExecption directly
>
> For [1] and [2] there is a position info in the exception message, a throw
> may like:
> {code:java}
> From line 1, column 15 to line 1, column 26:
> {code}
> But for 3, we only have the error message without position info, which is not
> that user friendly when the sql text is huge (there are 10 occurance for 3 in
> our parser). So shall we unify them ? E.G. Use only 1 and 2 is enough for all
> the cases, the 2 can totally replace 3.
>
> [1]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
> [2]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
> [3]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3168) Add test for invalid literal of sql parser
Danny Chan created CALCITE-3168: --- Summary: Add test for invalid literal of sql parser Key: CALCITE-3168 URL: https://issues.apache.org/jira/browse/CALCITE-3168 Project: Calcite Issue Type: Sub-task Reporter: Danny Chan Assignee: Danny Chan -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3152) Unify throws in sql parser
[
https://issues.apache.org/jira/browse/CALCITE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3152.
-
Resolution: Fixed
Fixed in
[65d973b|https://github.com/apache/calcite/commit/65d973b5ef0d5236021e0e265e85eccff10209bb]
!
> Unify throws in sql parser
> --
>
> Key: CALCITE-3152
> URL: https://issues.apache.org/jira/browse/CALCITE-3152
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Copied from DEV mail,
> Now our parser has 3 kinds of throws behavior:
>
> [1] Use JavaCC generateParseException
> [2] Use SqlUtil.newContextException
> [3] Use JavaCC ParseExecption directly
>
> For [1] and [2] there is a position info in the exception message, a throw
> may like:
> {code:java}
> From line 1, column 15 to line 1, column 26:
> {code}
> But for 3, we only have the error message without position info, which is not
> that user friendly when the sql text is huge (there are 10 occurance for 3 in
> our parser). So shall we unify them ? E.G. Use only 1 and 2 is enough for all
> the cases, the 2 can totally replace 3.
>
> [1]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
> [2]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
> [3]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3121) VolcanoPlanner hangs due to subquery with dynamic star
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876697#comment-16876697
]
Danny Chan edited comment on CALCITE-3121 at 7/2/19 8:32 AM:
-
Well, finally i got the reason, now the ProjectFilterTransposeRule will change
plan:
{code:xml}
LogicalProject(**=[$0])
LogicalFilter(subset=[rel#40:Subset#6.NONE], condition=[>(ITEM($0,
'N_NATIONKEY'), 1)])
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]])
{code}
to the form like:
{code:xml}
LogicalProject(**=[$0]) (1)
LogicalFilter(condition=[>($1, 1)])(2)
LogicalProject(**=[$0], ITEM=[ITEM($0, 'N_NATIONKEY')]) (3)
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]])
{code}
and the plan:
{code:xml}
LogicalProject(**=[$0], ITEM=[ITEM($0, 'N_NATIONKEY')])
LogicalFilter(subset=[rel#15:Subset#2.NONE], condition=[>(ITEM($0,
'N_NATIONKEY'), 1)])
LogicalProject(subset=[rel#13:Subset#1.NONE], **=[$0])
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]])
{code}
to the form like:
{code:xml}
LogicalFilter(condition=[>($1, 1)]) (4)
LogicalProject(**=[$0], ITEM=[ITEM($0, 'N_NATIONKEY')])
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]]){code}
See that (4) has same digest as (2), while these 4 plans are actually in the
same RelSet, so this forms a circle.
The plan-2 has more cost that the old one, because it always add a redundant
project (3), if we also have FilterProjectTransposeRule in the rule set, it
will also trigger infinite match of the ProjectMergeRule for project (1) and
new project (3).
Actually this ProjectFilterTransposeRule transformation is meaningless, because
it just add a useless project in the plan compared to the old one, so i just
skip conversion for such pattern.
was (Author: danny0405):
Well, finally i got the reason, now the ProjectFilterTransposeRule will change
plan:
{code:xml}
LogicalProject(**=[$0])
LogicalFilter(subset=[rel#40:Subset#6.NONE], condition=[>(ITEM($0,
'N_NATIONKEY'), 1)])
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]])
{code}
to the form like:
{code:xml}
LogicalProject(**=[$0]) (1)
LogicalFilter(condition=[>($1, 1)])(2)
LogicalProject(**=[$0], ITEM=[ITEM($0, 'N_NATIONKEY')]) (3)
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]])
{code}
This new plan has more cost that the old one, because it always add a redundant
project (3), if we also have FilterProjectTransposeRule in the rule set, it
will also trigger infinite match of the ProjectMergeRule for project (1) and
new project (3).
Actually this ProjectFilterTransposeRule transformation is meaningless, because
it just add a useless project in the plan compared to the old one, so i just
skip conversion for such pattern.
> VolcanoPlanner hangs due to subquery with dynamic star
> --
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to subquery with dynamic star
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876697#comment-16876697
]
Danny Chan commented on CALCITE-3121:
-
Well, finally i got the reason, now the ProjectFilterTransposeRule will change
plan:
{code:xml}
LogicalProject(**=[$0])
LogicalFilter(subset=[rel#40:Subset#6.NONE], condition=[>(ITEM($0,
'N_NATIONKEY'), 1)])
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]])
{code}
to the form like:
{code:xml}
LogicalProject(**=[$0]) (1)
LogicalFilter(condition=[>($1, 1)])(2)
LogicalProject(**=[$0], ITEM=[ITEM($0, 'N_NATIONKEY')]) (3)
LogicalTableScan(subset=[rel#11:Subset#0.NONE], table=[[CATALOG, SALES,
CUSTOMER]])
{code}
This new plan has more cost that the old one, because it always add a redundant
project (3), if we also have FilterProjectTransposeRule in the rule set, it
will also trigger infinite match of the ProjectMergeRule for project (1) and
new project (3).
Actually this ProjectFilterTransposeRule transformation is meaningless, because
it just add a useless project in the plan compared to the old one, so i just
skip conversion for such pattern.
> VolcanoPlanner hangs due to subquery with dynamic star
> --
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3121) VolcanoPlanner hangs due to subquery with dynamic star
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3121:
Summary: VolcanoPlanner hangs due to subquery with dynamic star (was:
VolcanoPlanner hangs due to removing ORDER BY from sub-query)
> VolcanoPlanner hangs due to subquery with dynamic star
> --
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3166) Make RelBuilder configurable
[
https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876677#comment-16876677
]
Danny Chan commented on CALCITE-3166:
-
Thanks [~julianhyde], i'm ok with this Context unwrap, cause it seems like a
code refactoring.
> Make RelBuilder configurable
>
>
> Key: CALCITE-3166
> URL: https://issues.apache.org/jira/browse/CALCITE-3166
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
>
> Make {{RelBuilder}} configurable, so that particular optimizations can easily
> be turned off.
> I propose to add a class {{RelBuilder.Config}}, which is immutable and has a
> public final field for each configuration property; also a class
> {{RelBuilder.ConfigBuilder}} to create a config.
> {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by
> calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3152) Unify throws in sql parser
[
https://issues.apache.org/jira/browse/CALCITE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876604#comment-16876604
]
Danny Chan commented on CALCITE-3152:
-
I'm gonna merge this PR if there are no more comments in 24 hours.
> Unify throws in sql parser
> --
>
> Key: CALCITE-3152
> URL: https://issues.apache.org/jira/browse/CALCITE-3152
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Copied from DEV mail,
> Now our parser has 3 kinds of throws behavior:
>
> [1] Use JavaCC generateParseException
> [2] Use SqlUtil.newContextException
> [3] Use JavaCC ParseExecption directly
>
> For [1] and [2] there is a position info in the exception message, a throw
> may like:
> {code:java}
> From line 1, column 15 to line 1, column 26:
> {code}
> But for 3, we only have the error message without position info, which is not
> that user friendly when the sql text is huge (there are 10 occurance for 3 in
> our parser). So shall we unify them ? E.G. Use only 1 and 2 is enough for all
> the cases, the 2 can totally replace 3.
>
> [1]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
> [2]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
> [3]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3166) Make RelBuilder configurable
[
https://issues.apache.org/jira/browse/CALCITE-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876603#comment-16876603
]
Danny Chan commented on CALCITE-3166:
-
Reviewed and left some comments.
> Make RelBuilder configurable
>
>
> Key: CALCITE-3166
> URL: https://issues.apache.org/jira/browse/CALCITE-3166
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
>
> Make {{RelBuilder}} configurable, so that particular optimizations can easily
> be turned off.
> I propose to add a class {{RelBuilder.Config}}, which is immutable and has a
> public final field for each configuration property; also a class
> {{RelBuilder.ConfigBuilder}} to create a config.
> {{RelBuilder.create(FrameworkConfig frameworkConfig)}} will get a config by
> calling {{frameworkConfig.getContext().unwrap(RelBuilder.Config.class)}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876064#comment-16876064 ] Danny Chan commented on CALCITE-3161: - [~rubenql] Oops, i'm so sorry that i missed your JIRA issue, go ahead with your work, and maybe you can just mark this issue duplicated. > Remove deprecated SemiJoinType usage completely > --- > > Key: CALCITE-3161 > URL: https://issues.apache.org/jira/browse/CALCITE-3161 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Fix For: 1.21.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3161) Remove deprecated SemiJoinType usage completely
Danny Chan created CALCITE-3161: --- Summary: Remove deprecated SemiJoinType usage completely Key: CALCITE-3161 URL: https://issues.apache.org/jira/browse/CALCITE-3161 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.20.0 Reporter: Danny Chan Assignee: Danny Chan Fix For: 1.21.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3126) Remove deprecated SemiJoin usage completely
[ https://issues.apache.org/jira/browse/CALCITE-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3126. - Resolution: Fixed Fixed in [79a4ec6|https://github.com/apache/calcite/commit/79a4ec63adfa83c6920b9ece5042f65a322ff80f] ! > Remove deprecated SemiJoin usage completely > --- > > Key: CALCITE-3126 > URL: https://issues.apache.org/jira/browse/CALCITE-3126 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As a following of CALCITE-2969, Julian Hyde has keep some deprecated usage of > SemiJoin class while supress the warnings[1], they are expected to be removed > from version 1.21 > [1] > [https://github.com/apache/calcite/commit/ef0c4c6ddb7553e2e858259007d8ca1cb3692101] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CALCITE-3155) AssertionError in RelToSqlConverter when visiting empty Values node
[ https://issues.apache.org/jira/browse/CALCITE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3155. - Resolution: Fixed Assignee: Danny Chan Fixed in [8dd4fed|https://github.com/apache/calcite/commit/8dd4fed46ff31df8737788847fdd2963944295f9], thanks for your PR, [~MusbahELFIL] ! > AssertionError in RelToSqlConverter when visiting empty Values node > --- > > Key: CALCITE-3155 > URL: https://issues.apache.org/jira/browse/CALCITE-3155 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Musbah EL FIL >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Given the query: > *select A."department_id" from "foodmart"."employee" A where > A."department_id" = ( select min( A."department_id") from > "foodmart"."department" B where 1=2 )* > The conversion from SqlNode to RelNode is correct, however when going from > RelNode to SqlNode, an exception was being thrown. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3155) AssertionError in RelToSqlConverter when visiting empty Values node
[ https://issues.apache.org/jira/browse/CALCITE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3155: Fix Version/s: 1.21.0 > AssertionError in RelToSqlConverter when visiting empty Values node > --- > > Key: CALCITE-3155 > URL: https://issues.apache.org/jira/browse/CALCITE-3155 > Project: Calcite > Issue Type: Bug > Components: core >Reporter: Musbah EL FIL >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Given the query: > *select A."department_id" from "foodmart"."employee" A where > A."department_id" = ( select min( A."department_id") from > "foodmart"."department" B where 1=2 )* > The conversion from SqlNode to RelNode is correct, however when going from > RelNode to SqlNode, an exception was being thrown. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3152) Unify throws in sql parser
[
https://issues.apache.org/jira/browse/CALCITE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3152:
Description:
Copied from DEV mail,
Now our parser has 3 kinds of throws behavior:
[1] Use JavaCC generateParseException
[2] Use SqlUtil.newContextException
[3] Use JavaCC ParseExecption directly
For [1] and [2] there is a position info in the exception message, a throw may
like:
{code:java}
>From line 1, column 15 to line 1, column 26:
{code}
But for 3, we only have the error message without position info, which is not
that user friendly when the sql text is huge (there are 10 occurance for 3 in
our parser). So shall we unify them ? E.G. Use only 1 and 2 is enough for all
the cases, the 2 can totally replace 3.
[1]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
[2]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
[3]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]
> Unify throws in sql parser
> --
>
> Key: CALCITE-3152
> URL: https://issues.apache.org/jira/browse/CALCITE-3152
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> Copied from DEV mail,
> Now our parser has 3 kinds of throws behavior:
>
> [1] Use JavaCC generateParseException
> [2] Use SqlUtil.newContextException
> [3] Use JavaCC ParseExecption directly
>
> For [1] and [2] there is a position info in the exception message, a throw
> may like:
> {code:java}
> From line 1, column 15 to line 1, column 26:
> {code}
> But for 3, we only have the error message without position info, which is not
> that user friendly when the sql text is huge (there are 10 occurance for 3 in
> our parser). So shall we unify them ? E.G. Use only 1 and 2 is enough for all
> the cases, the 2 can totally replace 3.
>
> [1]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
> [2]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
> [3]
> [https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3152) Unify throws in sql parser
Danny Chan created CALCITE-3152:
---
Summary: Unify throws in sql parser
Key: CALCITE-3152
URL: https://issues.apache.org/jira/browse/CALCITE-3152
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.20.0
Environment: {color:#33}Copied from DEV mail,{color}
{color:#33}Now our parser has 3 kinds of throws behavior:
[1] Use JavaCC generateParseException
[2] Use SqlUtil.newContextException
[3] Use JavaCC ParseExecption directly
For [1] and [2] there is a position info in the exception message, a throw may
like:{color}
{code:java}
>From line 1, column 15 to line 1, column 26:
{code}
{color:#33}{color:#33}But for 3, we only have the error message without
position info, which is not that user friendly when the sql text is huge (there
are 10 occurance for 3 in our parser). So shall we unify them ? E.G. Use only 1
and 2 is enough for all the cases, the 2 can totally replace 3.{color}{color}
{color:#33}{color:#33}{color:#33}[1]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
[2]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
[3]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]{color}{color}{color}
Reporter: Danny Chan
Assignee: Danny Chan
Fix For: 1.21.0
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (CALCITE-3152) Unify throws in sql parser
[
https://issues.apache.org/jira/browse/CALCITE-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3152:
Environment: (was: {color:#33}Copied from DEV mail,{color}
{color:#33}Now our parser has 3 kinds of throws behavior:
[1] Use JavaCC generateParseException
[2] Use SqlUtil.newContextException
[3] Use JavaCC ParseExecption directly
For [1] and [2] there is a position info in the exception message, a throw may
like:{color}
{code:java}
>From line 1, column 15 to line 1, column 26:
{code}
{color:#33}{color:#33}But for 3, we only have the error message without
position info, which is not that user friendly when the sql text is huge (there
are 10 occurance for 3 in our parser). So shall we unify them ? E.G. Use only 1
and 2 is enough for all the cases, the 2 can totally replace 3.{color}{color}
{color:#33}{color:#33}{color:#33}[1]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L4494]
[2]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L386]
[3]
[https://github.com/apache/calcite/blob/69c8053cd98ec65c55fa1c3b282b076536ab758f/core/src/main/codegen/templates/Parser.jj#L601]{color}{color}{color})
> Unify throws in sql parser
> --
>
> Key: CALCITE-3152
> URL: https://issues.apache.org/jira/browse/CALCITE-3152
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3128) Joining two tables producing only NULLs will return 0 rows
[
https://issues.apache.org/jira/browse/CALCITE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874592#comment-16874592
]
Danny Chan commented on CALCITE-3128:
-
[~julianhyde] thanks,
{quote}if the condition is '(t1.c is null) = (t2.c is null)', I think the
problem would still occur.
{quote}
I checked this query:
{code:sql}
select dept.deptno, emp.ename
from dept join emp
on (dept.dname is null) = (emp.sal is null)
order by emp.ename limit 10;
{code}
and finally plan is
{code:xml}
EnumerableLimit(fetch=[10])
EnumerableSort(sort0=[$1], dir0=[ASC])
EnumerableCalc(expr#0..4=[{inputs}], DEPTNO=[$t0], ENAME=[$t3])
EnumerableHashJoin(condition=[=($1, $4)], joinType=[inner])
EnumerableCalc(expr#0..2=[{inputs}], expr#3=[IS NULL($t1)],
DEPTNO=[$t0], $f3=[$t3])
EnumerableTableScan(table=[[scott, DEPT]])
EnumerableCalc(expr#0..7=[{inputs}], expr#8=[IS NULL($t5)],
proj#0..1=[{exprs}], $f8=[$t8])
EnumerableTableScan(table=[[scott, EMP]])
{code}
The condition '(t1.c is null) = (t2.c is null)' for hash join are all boolean
keys, which means they will never be null, so i don't think there is any
problem for this case.
{quote}The key factor, I think, is that one side has just one column, and a
null in that column.
{quote}
This is not true, like the case:
{code:sql}
SELECT *
FROM (VALUES (NULLIF(5, 5)), (NULLIF(5, 5))) a, (VALUES (NULLIF(5, 5)),
(NULLIF(5, 5))) b
{code}
> Joining two tables producing only NULLs will return 0 rows
> --
>
> Key: CALCITE-3128
> URL: https://issues.apache.org/jira/browse/CALCITE-3128
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Muhammad Gelbana
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> The following queries will return 0 rows while they're expected to ruturn
> rows with NULLs in them.
> {code:sql}
> SELECT *
> FROM (SELECT NULLIF(5, 5)) a, (SELECT NULLIF(5, 5)) b
> {code}
> {code:sql}
> SELECT *
> FROM (VALUES (NULLIF(5, 5)), (NULLIF(5, 5))) a, (VALUES (NULLIF(5, 5)),
> (NULLIF(5, 5))) b
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (CALCITE-3148) Validator throws IndexOutOfBoundsException for SqlInsert when source and sink have non-equal fields number
[ https://issues.apache.org/jira/browse/CALCITE-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3148. - Resolution: Fixed Fixed in [6375e24|https://github.com/apache/calcite/commit/6375e24d4a8848e11e776ed6968f3a6f499424ae] ! > Validator throws IndexOutOfBoundsException for SqlInsert when source and sink > have non-equal fields number > -- > > Key: CALCITE-3148 > URL: https://issues.apache.org/jira/browse/CALCITE-3148 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CALCITE-3148) Validator throws IndexOutOfBoundsException for SqlInsert when source and sink have non-equal fields number
Danny Chan created CALCITE-3148: --- Summary: Validator throws IndexOutOfBoundsException for SqlInsert when source and sink have non-equal fields number Key: CALCITE-3148 URL: https://issues.apache.org/jira/browse/CALCITE-3148 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.20.0 Reporter: Danny Chan Assignee: Danny Chan Fix For: 1.21.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (CALCITE-3142) An NPE when rounding a nullable numeric
[
https://issues.apache.org/jira/browse/CALCITE-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-3142:
Summary: An NPE when rounding a nullable numeric (was: A NPE when rounding
a nullable numeric)
> An NPE when rounding a nullable numeric
> ---
>
> Key: CALCITE-3142
> URL: https://issues.apache.org/jira/browse/CALCITE-3142
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Muhammad Gelbana
>Priority: Major
>
> The following query throws a NPE in the generated code because it assumes the
> divided value to be an initialized Java object (Not null), which is fine for
> the first row, but not for the second.
> {code:sql}
> SELECT ROUND(CAST((X/Y) AS NUMERIC), 2) FROM (VALUES (1, 2), (NULLIF(5, 5),
> NULLIF(5, 5))) A(X, Y){code}
> If I modify the query a little bit, it runs ok:
> – No casting
> {code:sql}
> SELECT ROUND((X/Y), 2) FROM (VALUES (1, 2), (NULLIF(5, 5), NULLIF(5, 5)))
> A(X, Y){code}
> – No rounding
> {code:sql}
> SELECT (X/Y)::NUMERIC FROM (VALUES (1, 2), (NULLIF(5, 5), NULLIF(5, 5))) A(X,
> Y){code}
> +This is the optimized generated code+
> {code:java}
> final Object[] current = (Object[]) inputEnumerator.current();
> final Integer inp0_ = (Integer) current[0];
> final Integer inp1_ = (Integer) current[1];
> final java.math.BigDecimal v1 = new java.math.BigDecimal(
> inp0_.intValue() / inp1_.intValue()); // <<< NPE
> return inp0_ == null || inp1_ == null ? (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.sround(v1, 2);{code}
> +This is the non-optimized one+
> {code:java}
> final Object[] current = (Object[]) inputEnumerator.current();
> final Integer inp0_ = (Integer) current[0];
> final boolean inp0__unboxed = inp0_ == null;
> final Integer inp1_ = (Integer) current[1];
> final boolean inp1__unboxed = inp1_ == null;
> final boolean v = inp0__unboxed || inp1__unboxed;
> final int inp0__unboxed0 = inp0_.intValue(); // <<< NPE
> final int inp1__unboxed0 = inp1_.intValue(); // <<< NPE
> final int v0 = inp0__unboxed0 / inp1__unboxed0;
> final java.math.BigDecimal v1 = new java.math.BigDecimal(
> v0);
> final java.math.BigDecimal v2 = v ? (java.math.BigDecimal) null :
> org.apache.calcite.runtime.SqlFunctions.sround(v1, 2);
> return v2;{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3128) Joining two tables producing only NULLs will return 0 rows
[
https://issues.apache.org/jira/browse/CALCITE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16873015#comment-16873015
]
Danny Chan commented on CALCITE-3128:
-
[~mgelbana] Thanks for your test case, i have applied a PR
[https://github.com/apache/calcite/pull/1279]
[~donnyzone] i also checked the Postgres and the behavior is same as you
proposed. While i think the MYSQL 5.6 behavior is wrong.
> Joining two tables producing only NULLs will return 0 rows
> --
>
> Key: CALCITE-3128
> URL: https://issues.apache.org/jira/browse/CALCITE-3128
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Muhammad Gelbana
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following queries will return 0 rows while they're expected to ruturn
> rows with NULLs in them.
> {code:sql}
> SELECT *
> FROM (SELECT NULLIF(5, 5)) a, (SELECT NULLIF(5, 5)) b
> {code}
> {code:sql}
> SELECT *
> FROM (VALUES (NULLIF(5, 5)), (NULLIF(5, 5))) a, (VALUES (NULLIF(5, 5)),
> (NULLIF(5, 5))) b
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-3128) Joining two tables producing only NULLs will return 0 rows
[
https://issues.apache.org/jira/browse/CALCITE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3128:
---
Assignee: Danny Chan
> Joining two tables producing only NULLs will return 0 rows
> --
>
> Key: CALCITE-3128
> URL: https://issues.apache.org/jira/browse/CALCITE-3128
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Muhammad Gelbana
>Assignee: Danny Chan
>Priority: Major
>
> The following queries will return 0 rows while they're expected to ruturn
> rows with NULLs in them.
> {code:sql}
> SELECT *
> FROM (SELECT NULLIF(5, 5)) a, (SELECT NULLIF(5, 5)) b
> {code}
> {code:sql}
> SELECT *
> FROM (VALUES (NULLIF(5, 5)), (NULLIF(5, 5))) a, (VALUES (NULLIF(5, 5)),
> (NULLIF(5, 5))) b
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3140) Multiple failures when executing slow tests
[
https://issues.apache.org/jira/browse/CALCITE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871920#comment-16871920
]
Danny Chan commented on CALCITE-3140:
-
For "separately" i mean to run them in the IDEA test suit, they are all passed.
I did hang on {{MaterializationTest, but have not digged the details yet.}}
> Multiple failures when executing slow tests
> ---
>
> Key: CALCITE-3140
> URL: https://issues.apache.org/jira/browse/CALCITE-3140
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0, 1.19.0, 1.20.0
> Environment: Windows 10 Pro, jdk1.8.0_202, maven 3.6.1 with 8G memory
> allocated to the VM (Xmx).
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Fix For: 1.21.0
>
>
> Running slow tests using the following command
> {noformat}
> mvn clean install -Dcalcite.test.slow -fn
> {noformat}
> finishes with the following errors.
> {noformat}
> [ERROR] Tests run: 12620, Failures: 0, Errors: 6, Skipped: 6310, Time
> elapsed: 2,522.456 s <<< FAILURE! - in org.apache.calcite.test.FoodmartTest
> (SOE + Timeouts)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 10.448 s <<< FAILURE! - in org.apache.calcite.sql.test.SqlOperatorTest
> [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 49.711 s <<< FAILURE! - in org.apache.calcite.test.CoreQuidemTest
> (Intermittent failures/Plan comparison)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 19.759 s <<< FAILURE! - in org.apache.calcite.test.CalciteSqlOperatorTest
> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 7, Time elapsed:
> 8.132 s <<< FAILURE! - in org.apache.calcite.adapter.tpcds.TpcdsTest (Stale
> plans)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3128) Joining two tables producing only NULLs will return 0 rows
[
https://issues.apache.org/jira/browse/CALCITE-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871051#comment-16871051
]
Danny Chan commented on CALCITE-3128:
-
While i execute the query in MYSQL 5.6:
{code:sql}
SELECT *FROM (SELECT NULLIF(5, 5)) a, (SELECT NULLIF(5, 5)) b
{code}
the result are (null) which is same from query:
{code:java}
SELECT *FROM (SELECT NULLIF(5, 5)) a
{code}
Does this behavior is expected ?
> Joining two tables producing only NULLs will return 0 rows
> --
>
> Key: CALCITE-3128
> URL: https://issues.apache.org/jira/browse/CALCITE-3128
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Muhammad Gelbana
>Priority: Major
>
> The following queries will return 0 rows while they're expected to ruturn
> rows with NULLs in them.
> {code:sql}
> SELECT *
> FROM (SELECT NULLIF(5, 5)) a, (SELECT NULLIF(5, 5)) b
> {code}
> {code:sql}
> SELECT *
> FROM (VALUES (NULLIF(5, 5)), (NULLIF(5, 5))) a, (VALUES (NULLIF(5, 5)),
> (NULLIF(5, 5))) b
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3140) Multiple failures when executing slow tests
[
https://issues.apache.org/jira/browse/CALCITE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871046#comment-16871046
]
Danny Chan commented on CALCITE-3140:
-
[~zabetak] I ran them separately, and there is no error for all these tests,
but if i run with the parameters:
{code:java}
mvn clean install -Dcalcite.test.slow -fn
{code}
I will hang at MaterializationTest, so this failure is very about the
environment to execute. My env is
{code:java}
Mac Mojave 10.14
Maven version: 3.5.3
Java version: 1.8.0_161, vendor: Oracle Corporation{code}
> Multiple failures when executing slow tests
> ---
>
> Key: CALCITE-3140
> URL: https://issues.apache.org/jira/browse/CALCITE-3140
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0, 1.19.0, 1.20.0
> Environment: Windows 10 Pro, jdk1.8.0_202, maven 3.6.1 with 8G memory
> allocated to the VM (Xmx).
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Fix For: 1.21.0
>
>
> Running slow tests using the following command
> {noformat}
> mvn clean install -Dcalcite.test.slow -fn
> {noformat}
> finishes with the following errors.
> {noformat}
> [ERROR] Tests run: 12620, Failures: 0, Errors: 6, Skipped: 6310, Time
> elapsed: 2,522.456 s <<< FAILURE! - in org.apache.calcite.test.FoodmartTest
> (SOE + Timeouts)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 10.448 s <<< FAILURE! - in org.apache.calcite.sql.test.SqlOperatorTest
> [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 49.711 s <<< FAILURE! - in org.apache.calcite.test.CoreQuidemTest
> (Intermittent failures/Plan comparison)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 19.759 s <<< FAILURE! - in org.apache.calcite.test.CalciteSqlOperatorTest
> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 7, Time elapsed:
> 8.132 s <<< FAILURE! - in org.apache.calcite.adapter.tpcds.TpcdsTest (Stale
> plans)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-771) Use materialization for scan-project-sort query
[
https://issues.apache.org/jira/browse/CALCITE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871020#comment-16871020
]
Danny Chan commented on CALCITE-771:
Thanks, [~donnyzone], i have left some review comments.
> Use materialization for scan-project-sort query
> ---
>
> Key: CALCITE-771
> URL: https://issues.apache.org/jira/browse/CALCITE-771
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> If a materialization is defined for scan-project-sort over another table,
> then scan-project and scan-project-sort queries should be able to use it.
> For example, given the following schema (in pseudo-DDL):
> {code}
> CREATE TABLE Emp (empid, deptno, gender);
> CREATE MATERIALIZATION I_Emp_Deptno AS
> SELECT deptno, empid FROM Emp ORDER BY deptno;
> {code}
> Calcite should rewrite the query
> {code}SELECT deptno FROM Emp ORDER BY deptno{code}
> to
> {code}
> Project(deptno)
> TableScan(table=[I_Emp_Deptno])
> {code}
> Note that no sort is necessary.
> This rewrite is important for using secondary indexes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3140) Multiple failures when executing slow tests
[
https://issues.apache.org/jira/browse/CALCITE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16870400#comment-16870400
]
Danny Chan commented on CALCITE-3140:
-
Thanks [~zabetak] for reporting this.
> Multiple failures when executing slow tests
> ---
>
> Key: CALCITE-3140
> URL: https://issues.apache.org/jira/browse/CALCITE-3140
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0, 1.19.0, 1.20.0
> Environment: Windows 10 Pro, jdk1.8.0_202, maven 3.6.1 with 8G memory
> allocated to the VM (Xmx).
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Fix For: 1.21.0
>
>
> Running slow tests using the following command
> {noformat}
> mvn clean install -Dcalcite.test.slow -fn
> {noformat}
> finishes with the following errors.
> {noformat}
> [ERROR] Tests run: 12620, Failures: 0, Errors: 6, Skipped: 6310, Time
> elapsed: 2,522.456 s <<< FAILURE! - in org.apache.calcite.test.FoodmartTest
> (SOE + Timeouts)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 10.448 s <<< FAILURE! - in org.apache.calcite.sql.test.SqlOperatorTest
> [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 49.711 s <<< FAILURE! - in org.apache.calcite.test.CoreQuidemTest
> (Intermittent failures/Plan comparison)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 19.759 s <<< FAILURE! - in org.apache.calcite.test.CalciteSqlOperatorTest
> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 7, Time elapsed:
> 8.132 s <<< FAILURE! - in org.apache.calcite.adapter.tpcds.TpcdsTest (Stale
> plans)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3132) Simply expressions in LogicalFilter generated for subqueries with quantified predicate
[
https://issues.apache.org/jira/browse/CALCITE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866589#comment-16866589
]
Danny Chan commented on CALCITE-3132:
-
Thanks, [~vgarg], the later 2 AND predicates are indeed redundant, please do it
!
> Simply expressions in LogicalFilter generated for subqueries with quantified
> predicate
> --
>
> Key: CALCITE-3132
> URL: https://issues.apache.org/jira/browse/CALCITE-3132
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: sub-query
>
> Following test in RelOptRulesTest generate plan with LogicalFilter
> {code}
> @Test public void testSome() {
> final String sql = "select * from emp e1\n"
> + " where e1.empno > SOME (select deptno from dept)";
> checkSubQuery(sql).withLateDecorrelation(true).check();
> }
> {code}
> {code}
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalProject(EMPNO=[$0], ENAME=[$1], JOB=[$2], MGR=[$3], HIREDATE=[$4],
> SAL=[$5], COMM=[$6], DEPTNO=[$7], SLACKER=[$8])
> LogicalFilter(condition=[OR(AND(IS TRUE(>($0, $9)), <>($10, 0)),
> AND(>($10, $11), null, <>($10, 0), IS NOT TRUE(>($0, $9))), AND(>($0, $9),
> <>($10, 0), IS NOT TRUE(>($0, $9)), <=($10, $11)))])
> LogicalJoin(condition=[true], joinType=[inner])
> LogicalTableScan(table=[[CATALOG, SALES, EMP]])
> LogicalAggregate(group=[{}], m=[MIN($0)], c=[COUNT()], d=[COUNT($0)])
> LogicalProject(DEPTNO=[$0])
> LogicalTableScan(table=[[CATALOG, SALES, DEPT]])
> {code}
> Note that LogicalFilter has condition which should be further simplified
> {code}
> (condition=[OR(AND(IS TRUE(>($0, $9)), <>($10, 0)), AND(>($10, $11), null,
> <>($10, 0), IS NOT TRUE(>($0, $9))), AND(>($0, $9), <>($10, 0), IS NOT
> TRUE(>($0, $9)), <=($10, $11)))])
> {code}
> If {{AND(IS TRUE(>($0, $9)), <>($10, 0))}} is true so will be {{AND(IS
> TRUE(>($0, $9)), <>($10, 0))}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (CALCITE-3126) Remove deprecated SemiJoin usage completely
Danny Chan created CALCITE-3126: --- Summary: Remove deprecated SemiJoin usage completely Key: CALCITE-3126 URL: https://issues.apache.org/jira/browse/CALCITE-3126 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.19.0 Reporter: Danny Chan Assignee: Danny Chan Fix For: 1.21.0 As a following of CALCITE-2969, Julian Hyde has keep some deprecated usage of SemiJoin class while supress the warnings[1], they are expected to be removed from version 1.21 [1] [https://github.com/apache/calcite/commit/ef0c4c6ddb7553e2e858259007d8ca1cb3692101] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862010#comment-16862010
]
Danny Chan edited comment on CALCITE-3121 at 6/12/19 11:20 AM:
---
If we change the sql to
{code:java}
String sql = "select n.n_regionkey from ( select * from "
+ "( select n_regionkey, n_nationkey from sales.customer) t) n where
n.n_nationkey >1 ";
{code}
The query will not hang, so i believe it is the ProjectMergeRule that can not
handle the pattern correctly,
{code:java}
project
|-filter
|---project(star(*))
|-scan{code}
was (Author: danny0405):
If we change the sql to
{code:java}
String sql = "select n.n_regionkey from ( select * from "
+ "( select n_regionkey, n_nationkey from sales.customer) t) n where
n.n_nationkey >1 ";
{code}
The query will not hang, so i believe it the ProjectMergeRule that can not
handle the pattern correctly,
{code:java}
project
|-filter
|---project(star(*))
|-scan{code}
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862010#comment-16862010
]
Danny Chan commented on CALCITE-3121:
-
If we change the sql to
{code:java}
String sql = "select n.n_regionkey from ( select * from "
+ "( select n_regionkey, n_nationkey from sales.customer) t) n where
n.n_nationkey >1 ";
{code}
The query will not hang, so i believe it the ProjectMergeRule that can not
handle the pattern correctly,
{code:java}
project
|-filter
|---project(star(*))
|-scan{code}
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16861715#comment-16861715
]
Danny Chan commented on CALCITE-3118:
-
Okey, finally i believe this is a bug
For this case, it happens with VolcanoRuleCall#getOperand0().solveOrder as [1,
0, 2],
for a tree with asending match mode like:
{code:java}
Birel(0)
/ \
left(1) right(2){code}
The VolcanoRuleCall firstly match right(2); then it start to match Birel(0),
obviously it is the successor of right(2), but when we do check
{code:java}
final RelSubset input =
(RelSubset) rel.getInput(previousOperand.ordinalInParent);
List inputRels = input.set.getRelsFromAllSubsets();
if (!inputRels.contains(previous)) {
continue;
}{code}
It got passed only because right(2) is the child of Birel(0), but as to the
sloveOrder, right(2) expects to be the second child of Birel(0), but with this
successfull match, it is the first child of Birel(0).
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched RelNode as a child *with the* *expected
> child ordinal*.
> However, there is a bug in this child ordinal check:
> {noformat}
> 333if (ascending && operand.childPolicy !=
> RelOptRuleOperandChildPolicy.UNORDERED) {
> 334 // We know that the previous operand was *a* child of its parent,
> 335 // but now check that it is the *correct* child.
> 336 final RelSubset input =
> 337 (RelSubset) rel.getInput(previousOperand.ordinalInParent);
> 338 List inputRels = input.set.getRelsFromAllSubsets();
> 339 if (!inputRels.contains(previous)) {
> 340continue;
> 341 }
> 342}
> {noformat}
> We intend to make sure that "previous" is in Subset "input". However line 338
> looked at RelNodes from the entire RelSet, rather than RelNodes only in
> Subset "input". As a result, in some cases, incorrect parent is not skipped
> as expected and is matched incorrectly.
> The unit test case included is a case that triggers this bug, where a second
> child RelNode incorrectly get matched as the first child of the parent
> RelNode.
> --
> Here's a detailed explanation of the test case that triggers the bug
> We constructed a RelNode structure:
> {noformat}
> PhysBiRel
> / \
> Subset1 Subset2
> | |
> leftPhyrightPhy
> {noformat}
> Where PhysBiRel has two inputs: leftPhy and rightPhy, both are logically
> equivalent, but with different traits (Convention in this test case), and
> thus are in different subsets.
> (Two Rels in two subsets in the same RelSet is a necessary condition to
> trigger this bug. )
> A rule AssertOperandsDifferentRule is constructed as follows:
> {noformat}
> operand(PhysBiRel.class,
> operand(PhysLeafRel.class, any()),
> operand(PhysLeafRel.class, any()))
> {noformat}
> Obviously the correct match would be:
> {noformat}
> PhysBiRel
> / \
> leftPhyrightPhy
> {noformat}
> However, with the bug, another match is also returned:
> {noformat}
> PhysBiRel
> / \
> rightPhyrightPhy
> {noformat}
> *This is wrong because rightPhy is not PhysBiRel's first input at all, and
> should not match as parent operand's first child.*
>
> Here's how the incorrect match occurs.
> 1. When rightPhy of class PhysLeafRel is registered, we attempt to match it
> to both the left and right child operands of AssertOperandsDifferentRule
> above. This is expected.
> 2. When matched to the right child operand, it eventually leads to the
> correct match result above.
> 3. When matched to the left child operand, it should have skipped/failed
> matching the parent operand to PhysBiRel because rightPhy is *NOT*
> PhysBiRel's first input. But because of the bug, the match succeeded. After
> parent is matched, then it attempt to match the right child operand, and
> again matched the rightPhy. As a result, both child operand end up matching
> the same RelNode rightPhy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859828#comment-16859828
]
Danny Chan commented on CALCITE-2744:
-
Thanks [~julianhyde], i still apply the fix because i think the original name
"MY_AVG" name is misleading. Fixed in
[340e21a6a|https://github.com/apache/calcite/commit/340e21a6a93e0766acf3fa292c4ecd86d5f5f8ef]
!
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at
[jira] [Commented] (CALCITE-2822) Allow multi-join rules to be used with any project/filter
[
https://issues.apache.org/jira/browse/CALCITE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859694#comment-16859694
]
Danny Chan commented on CALCITE-2822:
-
The master branch build fails from this
[https://github.com/apache/calcite/commit/0cb51c64715dd45f8bbcd533cc9096b4bb7b70af]
commit, with failed case:
CalciteSqlOperatorTest#testSimilarToOperator
{code:java}
tester.checkFails(
"'yd3223' similar to '[:LOWER:]{2}[:DIGIT:]{,5}'",
"Illegal repetition near index 20\n"
+ "\\[\\:LOWER\\:\\]\\{2\\}\\[\\:DIGIT\\:\\]\\{,5\\}\n"
+ " \\^",
true);
{code}
I have no idea if this failure comes from this commit, [~siddteotia] can you
take a look ?
> Allow multi-join rules to be used with any project/filter
> -
>
> Key: CALCITE-2822
> URL: https://issues.apache.org/jira/browse/CALCITE-2822
> Project: Calcite
> Issue Type: Task
> Components: core
>Reporter: Siddharth Teotia
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Allow multi join rules to be used with any subclass of Project or Filter
> instead of having just LogicalProject and LogicalFilter
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859670#comment-16859670
]
Danny Chan commented on CALCITE-2744:
-
[~julianhyde] I fired a fix in [https://github.com/apache/calcite/pull/1263],
can you take a look ?
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at
[jira] [Commented] (CALCITE-3005) Implement string functions: LEFT, RIGHT
[
https://issues.apache.org/jira/browse/CALCITE-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859664#comment-16859664
]
Danny Chan commented on CALCITE-3005:
-
An compile warning of JavaCC throws:
{code:java}
Warning: Choice conflict involving two expansions at
line 6350, column 13 and line 6353, column 13 respectively.
A common prefix is: "LEFT"
Consider using a lookahead of 2 for earlier expansion.
{code}
We need to fix this.
> Implement string functions: LEFT, RIGHT
> ---
>
> Key: CALCITE-3005
> URL: https://issues.apache.org/jira/browse/CALCITE-3005
> Project: Calcite
> Issue Type: New Feature
>Reporter: Forward Xu
>Assignee: Forward Xu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Some string functions including LEFT, RIGHT are not implemented now. It would
> be great if these functions can be implemented.
> [{{LEFT(_str_}},_{{len}}_)|https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_left]
> Returns the leftmost _{{len}}_ characters from the string _{{str}}_, or
> {{NULL}} if any argument is {{NULL}}.
> [{{RIGHT(_str_}},_{{len}}_)|https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_right]
> Returns the rightmost _{{len}}_ characters from the string _{{str}}_, or
> {{NULL}} if any argument is {{NULL}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859104#comment-16859104
]
Danny Chan commented on CALCITE-3121:
-
It seems that after some planning rules matched, In ProjectMergeRule, the top
project reference a wrong RelSubSet as input, so it just add new project over
and over again, and never ends up.
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (CALCITE-3121) VolcanoPlanner hangs due to removing ORDER BY from sub-query
[
https://issues.apache.org/jira/browse/CALCITE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reassigned CALCITE-3121:
---
Assignee: Danny Chan
> VolcanoPlanner hangs due to removing ORDER BY from sub-query
>
>
> Key: CALCITE-3121
> URL: https://issues.apache.org/jira/browse/CALCITE-3121
> Project: Calcite
> Issue Type: Bug
>Reporter: Bohdan Kazydub
>Assignee: Danny Chan
>Priority: Blocker
>
> After the fix for CALCITE-2798 some queries hang during planning in
> VolcanoPlanner (similar issue was reported in CALCITE-2223).
> Here is a test case which should be added to the {{RelOptRulesTest}} class:
> {code:java}
> @Test public void testSubQueryWithOrderByHang() {
> String sql = "select n.n_regionkey from ( select * from "
> + "( select * from sales.customer) t order by t.n_regionkey) n where
> n.n_nationkey >1 ";
> VolcanoPlanner planner = new VolcanoPlanner(null, null);
> planner.addRelTraitDef(ConventionTraitDef.INSTANCE);
> Tester dynamicTester = createDynamicTester().withDecorrelation(true)
> .withClusterFactory(
> relOptCluster -> RelOptCluster.create(planner,
> relOptCluster.getRexBuilder()));
> RelRoot root = dynamicTester.convertSqlToRel(sql);
> String planBefore = NL + RelOptUtil.toString(root.rel);
> getDiffRepos().assertEquals("planBefore", "${planBefore}", planBefore);
> PushProjector.ExprCondition exprCondition = expr -> {
> if (expr instanceof RexCall) {
> RexCall call = (RexCall) expr;
> return
> "item".equals(call.getOperator().getName().toLowerCase(Locale.ROOT));
> }
> return false;
> };
> RuleSet ruleSet =
> RuleSets.ofList(
> FilterProjectTransposeRule.INSTANCE,
> FilterMergeRule.INSTANCE,
> ProjectMergeRule.INSTANCE,
> new ProjectFilterTransposeRule(Project.class, Filter .class,
> RelFactories.LOGICAL_BUILDER, exprCondition),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_FILTER_RULE,
> EnumerableRules.ENUMERABLE_SORT_RULE,
> EnumerableRules.ENUMERABLE_LIMIT_RULE,
> EnumerableRules.ENUMERABLE_TABLE_SCAN_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> root.rel.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode relAfter = program.run(planner, root.rel, toTraits,
> Collections.emptyList(), Collections.emptyList());
> String planAfter = NL + RelOptUtil.toString(relAfter);
> getDiffRepos().assertEquals("planAfter", "${planAfter}", planAfter);
> }
> {code}
> Please note that if {{LIMIT }} is added to the sub-query with order by
> (so it is not removed due to the fix for CALCITE-2798) the test succeeds.
> Though the issue with hanging is more general, I think that if it wouldn't be
> fixed, the fix for CALCITE-2798 should be reverted to reduce cases when
> planner may hang.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859078#comment-16859078
]
Danny Chan edited comment on CALCITE-3118 at 6/8/19 2:20 AM:
-
I'm -1 on this fix.
Just like you said, _AssertOperandsDifferentRule's_ match pattern is:
{code:java}
PhysBiRel
/ \
leftPhy rightPhy{code}
Firstly you construct a RelNode tree like what the pattern describes, then you
register that the `leftPhy` and `rightPhy` are equivalent. It's not that
surprising you got a tree like:
{code:java}
PhysBiRel
/ \
rightPhy rightPhy
{code}
This tree also satisfied _AssertOperandsDifferentRule_ patttern.
This is Volcano Planner so the possibility space boost. BTW, the tree you build
manually would still be matched.
We have no contract that the `RelOptRule` must match same convention nodes
which would reduce many promotion possibilities.
was (Author: danny0405):
I'm -1 on this fix.
Just like you said, _AssertOperandsDifferentRule's_ match pattern is:
{code:java}
PhysBiRel
/ \
leftPhy rightPhy{code}
Firstly you construct a RelNode tree like what the pattern describes, then you
register that the `leftPhy` and `rightPhy` are equivalent. It's not that
surprising you got a tree like:
{code:java}
PhysBiRel
/ \
rightPhy rightPhy
{code}
This tree also satisfied _AssertOperandsDifferentRule_ patttern.
This is Volcano Planner so the possibility space boost. BTW, the tree you build
manually would still be matched.
We have contract that the `RelOptRule` must match same convention nodes which
would reduce many promotion possibilities.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where a second child
> RelNode get matched as first child of the parent RelNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16859078#comment-16859078
]
Danny Chan commented on CALCITE-3118:
-
I'm -1 on this fix.
Just like you said, _AssertOperandsDifferentRule's_ match pattern is:
{code:java}
PhysBiRel
/ \
leftPhy rightPhy{code}
Firstly you construct a RelNode tree like what the pattern describes, then you
register that the `leftPhy` and `rightPhy` are equivalent. It's not that
surprising you got a tree like:
{code:java}
PhysBiRel
/ \
rightPhy rightPhy
{code}
This tree also satisfied _AssertOperandsDifferentRule_ patttern.
This is Volcano Planner so the possibility space boost. BTW, the tree you build
manually would still be matched.
We have contract that the `RelOptRule` must match same convention nodes which
would reduce many promotion possibilities.
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where a second child
> RelNode get matched as first child of the parent RelNode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3118) VolcanoRuleCall match parent child ordinal not properly checked
[
https://issues.apache.org/jira/browse/CALCITE-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858325#comment-16858325
]
Danny Chan commented on CALCITE-3118:
-
I don't know why you make {{leftPhy}} and {{rightPhy}} equivalent, If they
were, the behavior is correct without your fix, because RelOptRule only matches
the RelNode class, not the convention, if you really want to match specific
convention, add a predicate on it :)
> VolcanoRuleCall match parent child ordinal not properly checked
> ---
>
> Key: CALCITE-3118
> URL: https://issues.apache.org/jira/browse/CALCITE-3118
> Project: Calcite
> Issue Type: Bug
>Reporter: Botong Huang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In VolcanoRuleCall.matchRecurse(), when ascending (child operand is matched,
> looking for parent operand match), we want to check that the matched parent
> indeed has the previously matched child RelNode as a child with the expected
> ordinal. However, there is a bug in this check. As a result, some incorrect
> parent is not skipped as expected and matched incorrectly. See unit test
> included in PR for a case that triggers this bug, where the same RelNode get
> matched for two operands at the same time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-3117) Distinguish Calcite's public APIs
[ https://issues.apache.org/jira/browse/CALCITE-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858282#comment-16858282 ] Danny Chan commented on CALCITE-3117: - The parser and sql node should be split out of the calcite-core, user usually only want to parse their sql and get a AST. > Distinguish Calcite's public APIs > - > > Key: CALCITE-3117 > URL: https://issues.apache.org/jira/browse/CALCITE-3117 > Project: Calcite > Issue Type: Task >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Minor > > The goal of this issue is to identify and mark Calcite's public APIs. This > can have bring various benefits to the project: > * provide more flexibility to developers; > * speed-up reviews; > * enhance backward compatibility; > * facilitate the life of end-users who will be exposed to fewer classes and > interfaces. > Let's list here packages, interfaces, and classes that should be part of the > public API. As a first step let's focus exclusively on completing the list > and not argue about others proposals. It will be easier to solve > disagreements afterwards as soon as we have a rather complete list. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2829) Use consistent types when processing ranges
[ https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858261#comment-16858261 ] Danny Chan edited comment on CALCITE-2829 at 6/7/19 2:40 AM: - Sure, the more test cases the better, i would link this issue with CALCITE-2302, thanks for your test cases. was (Author: danny0405): Sure, the more test cases the better, i would link this issue with CALCITE-2302 > Use consistent types when processing ranges > --- > > Key: CALCITE-2829 > URL: https://issues.apache.org/jira/browse/CALCITE-2829 > Project: Calcite > Issue Type: Bug >Reporter: Siddharth Teotia >Assignee: Juhwan Kim >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Range expressions like = 'literal' AND < 'literal' trigger > ClassCastException as literal are implicitly casted differently between =/<> > operators and other comparison operators. Apply the same casting rules for > comparison to =/<> calls when processing ranges, so that all the terms have > the same type for literals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2829) Use consistent types when processing ranges
[ https://issues.apache.org/jira/browse/CALCITE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858261#comment-16858261 ] Danny Chan commented on CALCITE-2829: - Sure, the more test cases the better, i would link this issue with CALCITE-2302 > Use consistent types when processing ranges > --- > > Key: CALCITE-2829 > URL: https://issues.apache.org/jira/browse/CALCITE-2829 > Project: Calcite > Issue Type: Bug >Reporter: Siddharth Teotia >Assignee: Juhwan Kim >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Range expressions like = 'literal' AND < 'literal' trigger > ClassCastException as literal are implicitly casted differently between =/<> > operators and other comparison operators. Apply the same casting rules for > comparison to =/<> calls when processing ranges, so that all the terms have > the same type for literals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3114) Support websocket transport
[ https://issues.apache.org/jira/browse/CALCITE-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857429#comment-16857429 ] Danny Chan commented on CALCITE-3114: - What do you mean by websocket transport ? > Support websocket transport > --- > > Key: CALCITE-3114 > URL: https://issues.apache.org/jira/browse/CALCITE-3114 > Project: Calcite > Issue Type: Wish >Reporter: Leo Tu >Priority: Minor > > it would be greatly improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[
https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857366#comment-16857366
]
Danny Chan commented on CALCITE-2280:
-
This is actually what we have to do in Flink custom parser, for computation
engines like Flink, one big topic is the Hive compatibility, for example, if we
want to add "partition" grammar like
{code:java}
insert into T partition(col1=vl) select from S;
{code}
What we have to do is to copy the whole java cc parse block for insert
statement and add our custom parse logic.
We also have to add our SqlConformance Hive, but the SqlConformanceEnum is
enum, so i also have to hard code the conformance in the parser template.
> Liberal "babel" parser that accepts all SQL dialects
>
>
> Key: CALCITE-2280
> URL: https://issues.apache.org/jira/browse/CALCITE-2280
> Project: Calcite
> Issue Type: Bug
> Components: babel
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.17.0
>
>
> Create a parser that accepts all SQL dialects.
> It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery.
> If you have preferred dialects, please let us know in the comments section.
> (If you're willing to work on a particular dialect, even better!)
> We would do this in a new module, inheriting and extending the parser in the
> same way that the DDL parser in the "server" module does.
> This would be a messy and difficult project, because we would have to comply
> with the rules of each parser (and its set of built-in functions) rather than
> writing the rules as we would like them to be. That's why I would keep it out
> of the core parser. But it would also have large benefits.
> This would be new territory Calcite: as a tool for manipulating/understanding
> SQL, not (necessarily) for relational algebra or execution.
> Some possible uses:
> * analyze query lineage (what tables and columns are used in a query);
> * translate from one SQL dialect to another (using the JDBC adapter to
> generate SQL in the target dialect);
> * a "deep" compatibility mode (much more comprehensive than the current
> compatibility mode) where Calcite could pretend to be, say, Oracle;
> * SQL parser as a service: a REST call gives a SQL query, and returns a JSON
> or XML document with the parse tree.
> If you can think of interesting uses, please discuss in the comments.
> There are similarities with Uber's
> [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can
> collaborate, or make use of their test cases.
> We will need a lot of sample queries. If you are able to contribute sample
> queries for particular dialects, please discuss in the comments section. It
> would be good if the sample queries are based on a familiar schema (e.g.
> scott or foodmart) but we can be flexible about this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857220#comment-16857220
]
Danny Chan commented on CALCITE-2744:
-
Fixed in
[https://github.com/apache/calcite/commit/7c319dbbce7957d622dcc47707f293ddaa5bcb34],
thanks for your PR, [~godfreyhe] !
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at
[jira] [Resolved] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-2744.
-
Resolution: Fixed
Fix Version/s: 1.20.0
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at
[jira] [Commented] (CALCITE-2968) New AntiJoin relational expression
[ https://issues.apache.org/jira/browse/CALCITE-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856680#comment-16856680 ] Danny Chan commented on CALCITE-2968: - [~rubenql] I have also reviewed this PR and +1 for merging this into 1.20.0, thanks for your great work ! > New AntiJoin relational expression > -- > > Key: CALCITE-2968 > URL: https://issues.apache.org/jira/browse/CALCITE-2968 > Project: Calcite > Issue Type: New Feature >Reporter: Ruben Quesada Lopez >Assignee: Ruben Quesada Lopez >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The goal of this ticket is to follow the discussion started in CALCITE-2920 > (and its [PR|https://github.com/apache/calcite/pull/1110]). > With the implementation of CALCITE-2920, RelBuilder has now a method to > create an AntiJoin. However, since, for the moment, there is no "AntiJoin" > expression, the only possibility to build an AntiJoin is using a > LogicalCorrelate with SemiJoinType.ANTI, and that's exactly what this method > does, for now. > The idea would be to "generalize" this with the creation of a new AntiJoin > (a.k.a. AntiSemiJoin) relational expression. With the implementation of > CALCITE-2969 (re-design of join-like expressions), it seems clear that the > new AntiJoin expression will be a *LogicalJoin with JoinType=ANTI*. Also, it > will have to be verified that: > - this new AntiJoin works fine in EnumerableConvention with all types of join > condition (equi-join, partial equi-join, non equi-join) > - this new AntiJoin can be correctly transformed into a Correlate (type ANTI) > via JoinToCorrelateRule -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3111) Allow custom implementations of Correlate in RelDecorrelator
[ https://issues.apache.org/jira/browse/CALCITE-3111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856621#comment-16856621 ] Danny Chan commented on CALCITE-3111: - Thanks [~zabetak], but what is the meaning to support nodes like physical ? We can decorrelate the logical nodes then the decorrelation work is done ! The correlate variables will never appear in the following planning phrase. > Allow custom implementations of Correlate in RelDecorrelator > > > Key: CALCITE-3111 > URL: https://issues.apache.org/jira/browse/CALCITE-3111 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Juhwan Kim >Assignee: Juhwan Kim >Priority: Minor > > Currently, RelDecorrelator code only works for LogicalCorrelate. > Decorrelating through Calcite would become much more flexible if it allows > using custom implementations of Correlate. This would require refactoring all > logical rels used in RelDecorrelator to the abstract ones(e.g > LogicalCorrelate -> Correlate). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2280) Liberal "babel" parser that accepts all SQL dialects
[ https://issues.apache.org/jira/browse/CALCITE-2280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856620#comment-16856620 ] Danny Chan commented on CALCITE-2280: - [~julianhyde] The Babel parser can mix-in all kinds of sql dialect, but it also needs to have the ability to specific the parser to one sql dialect, for the engines like Apache Flink or Apache Drill, supports all kinds of sql dialect is not that necessary and will confusing the user if the we give the parser grammar but not implement them in query execution. Maybe we should not always have a SqlConformance level, but make the dialect configurable and easily to extend will bring much user friendliness. > Liberal "babel" parser that accepts all SQL dialects > > > Key: CALCITE-2280 > URL: https://issues.apache.org/jira/browse/CALCITE-2280 > Project: Calcite > Issue Type: Bug > Components: babel >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: 1.17.0 > > > Create a parser that accepts all SQL dialects. > It would accept common dialects such as Oracle, MySQL, PostgreSQL, BigQuery. > If you have preferred dialects, please let us know in the comments section. > (If you're willing to work on a particular dialect, even better!) > We would do this in a new module, inheriting and extending the parser in the > same way that the DDL parser in the "server" module does. > This would be a messy and difficult project, because we would have to comply > with the rules of each parser (and its set of built-in functions) rather than > writing the rules as we would like them to be. That's why I would keep it out > of the core parser. But it would also have large benefits. > This would be new territory Calcite: as a tool for manipulating/understanding > SQL, not (necessarily) for relational algebra or execution. > Some possible uses: > * analyze query lineage (what tables and columns are used in a query); > * translate from one SQL dialect to another (using the JDBC adapter to > generate SQL in the target dialect); > * a "deep" compatibility mode (much more comprehensive than the current > compatibility mode) where Calcite could pretend to be, say, Oracle; > * SQL parser as a service: a REST call gives a SQL query, and returns a JSON > or XML document with the parse tree. > If you can think of interesting uses, please discuss in the comments. > There are similarities with Uber's > [QueryParser|https://eng.uber.com/queryparser/] tool. Maybe we can > collaborate, or make use of their test cases. > We will need a lot of sample queries. If you are able to contribute sample > queries for particular dialects, please discuss in the comments section. It > would be good if the sample queries are based on a familiar schema (e.g. > scott or foodmart) but we can be flexible about this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856331#comment-16856331
]
Danny Chan commented on CALCITE-2744:
-
[~julianhyde] I think there is some problem with the original PR, so i fired a
new one!
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at
[jira] [Commented] (CALCITE-3101) PushDownJoinConditions is not always a valid transformation
[
https://issues.apache.org/jira/browse/CALCITE-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856291#comment-16856291
]
Danny Chan commented on CALCITE-3101:
-
+1, project a boolean used for filtering seems a pattern that can be promoted,
but we should not introduce the complexity into pushDownJoinConditions.
> PushDownJoinConditions is not always a valid transformation
> ---
>
> Key: CALCITE-3101
> URL: https://issues.apache.org/jira/browse/CALCITE-3101
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.16.0
> Environment: Java app pointing to Oracle database.
>Reporter: Paul Jackson
>Priority: Major
>
> SqlToRelConverter can create a plan that is invalid when converted back to
> SQL in cases where the expression that is pushed to the projection returns a
> Boolean. The following example pushes IS NOT NULL to a select. Several SQL
> dialects do not support this. Oracle, for example, sees IS NOT NULL as a
> condition rather than an expression. It returns a Boolean data type, which is
> not supported. Likewise, Microsoft SQL Server does not support IS NOT NULL in
> a projection expression.
> Steps to reproduce (Oracle):
> DDL:
> {code:java}
> CREATE TABLE "EMP" (
> "empno" INTEGER PRIMARY KEY NOT NULL,
> "ename" VARCHAR(100),
> "deptno" INTEGER);{code}
> Start with this query:
> {code:java}
> SELECT "EMP"."empno", "t"."ename" "ename0"
> FROM "EMP"
> INNER JOIN "EMP" "t"
> ON "EMP"."deptno" = "t"."deptno" AND "t"."ename" IS NOT NULL{code}
> Parse using {{SqlToRelConverter.convertQuery()}}. At this point in the stack
> trace:
> {noformat}
> org.apache.calcite.plan.RelOptUtil.pushDownJoinConditions(RelOptUtil.java:3222)
> org.apache.calcite.sql2rel.SqlToRelConverter.createJoin(SqlToRelConverter.java:2414)
> org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2056)
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:641)
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:622)
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3057)
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:558)
> {noformat}
> the {{RelNode}} is:
> {noformat}
> LogicalJoin(condition=[AND(=($3, $8), IS NOT NULL($6))], joinType=[inner])
> JdbcTableScan(table=[[XYZ, EMP]])
> JdbcTableScan(table=[[XYZ, EMP]])
> {noformat}
> After {{pushDownJoinConditions}} the {{RelNode}} is:
> {noformat}
> LogicalJoin(condition=[AND(=($3, $8), $10)], joinType=[inner])
> JdbcTableScan(table=[[XYZ, EMP]])
> LogicalProject(empno=[$0], ename=[$1], job=[$2], deptno=[$3], etype=[$4],
> $f5=[IS NOT NULL($1)])
> JdbcTableScan(table=[[XYZ, EMP]])
> {noformat}
> Which leads to invalid SQL ("ORA-00923: FROM keyword not found where
> expected"):
> {code:java}
> SELECT "EMP"."empno", "t"."ename" "ename0"
> FROM "XYZ"."EMP"
> INNER JOIN (
> SELECT "empno", "ename", "job", "deptno", "etype", "ename" IS NOT NULL
> "$f5"
> FROM "XYZ"."EMP") "t"
> ON "EMP"."deptno" = "t"."deptno" AND "t"."$f5"
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856237#comment-16856237
]
Danny Chan edited comment on CALCITE-2744 at 6/5/19 2:17 AM:
-
I digged into the code and found that it is not because the agg call use
multiple parameters, it is because the final output map for the aggregate is
totally wrong. Say, We have a "combinedMap" as the mapping between the old
input and the new project:
{code:java}
Aggregate
| new project - - - -
| new_input combinedMap
| old_input - - - -
{code}
This "combinedMap" can not be used as map of the old aggregate and the new
generated aggregate. Actually, what we need to map it the old grouping keys,
the agg calls and the constant group key.
was (Author: danny0405):
I digged into the code and found that it is not because the agg call use
multiple parameters, it is because the final output map for the aggregate is
totally wrong. Say, We have a "combinedMap" as the mapping between the old
input and the new project:
{code:java}
Aggregate
| new project - - - -
| new_input combinedMap
| old_input - - - -
{code}
This "combinedMap" can not be used as map of the old aggregate and the new
generated aggregate.
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
[jira] [Commented] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856237#comment-16856237
]
Danny Chan commented on CALCITE-2744:
-
I digged into the code and found that it is not because the agg call use
multiple parameters, it is because the final output map for the aggregate is
totally wrong. Say, We have a "combinedMap" as the mapping between the old
input and the new project:
{code:java}
Aggregate
| new project - - - -
| new_input combinedMap
| old_input - - - -
{code}
This "combinedMap" can not be used as map of the old aggregate and the new
generated aggregate.
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
[jira] [Updated] (CALCITE-2744) RelDecorrelator use wrong output map for aggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-2744:
Summary: RelDecorrelator use wrong output map for aggregate decorrelate
(was: Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join
query with a multi-param aggregate function in subquery)
> RelDecorrelator use wrong output map for aggregate decorrelate
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at
[jira] [Updated] (CALCITE-2744) RelDecorrelator use wrong output map for LogicalAggregate decorrelate
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-2744:
Summary: RelDecorrelator use wrong output map for LogicalAggregate
decorrelate (was: RelDecorrelator use wrong output map for aggregate
decorrelate)
> RelDecorrelator use wrong output map for LogicalAggregate decorrelate
> -
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at
[jira] [Reopened] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan reopened CALCITE-2744:
-
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.17.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at
[jira] [Updated] (CALCITE-2744) Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query with a multi-param aggregate function in subquery
[
https://issues.apache.org/jira/browse/CALCITE-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan updated CALCITE-2744:
Affects Version/s: (was: 1.17.0)
1.20.0
> Throws RuntimeException in RelDecorrelator when optimizing a Semi-Join query
> with a multi-param aggregate function in subquery
> --
>
> Key: CALCITE-2744
> URL: https://issues.apache.org/jira/browse/CALCITE-2744
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Wei Zhong
>Assignee: Danny Chan
>Priority: Major
>
> when running the following test in org.apache.calcite.test.RelOptRulesTest,
> we will get a RuntimeException:
> {code:java}
> @Test public void testDecorrelateWith2ParamsAgg() {
> final HepProgram preProgram =
> HepProgram.builder()
> .addRuleInstance(FilterProjectTransposeRule.INSTANCE)
> .addRuleInstance(FilterJoinRule.FILTER_ON_JOIN)
> .addRuleInstance(ProjectMergeRule.INSTANCE)
> .build();
> final HepProgram program =
> HepProgram.builder()
> .addRuleInstance(SemiJoinRule.PROJECT)
> .build();
> final String sql = "select * from dept where exists (\n"
> + "select UDF_AVG(sal, sal + 1) from sales.emp\n"
> + "where emp.deptno = dept.deptno\n"
> + "and emp.sal > 100)";
> Sql sqlTest = sql(sql)
> .withDecorrelation(true)
> .withTrim(true)
> .withPre(preProgram)
> .with(program);
> sqlTest.check();
> }
> {code}
> UDF_AVG code in MockSqlOperatorTable:
> {code:java}
> public static void addRamp(MockSqlOperatorTable opTab) {
> // Don't use anonymous inner classes. They can't be instantiated
> // using reflection when we are deserializing from JSON.
> opTab.addOperator(new RampFunction());
> opTab.addOperator(new DedupFunction());
> opTab.addOperator(UDF_AVG);
> }
> public static final SqlFunction UDF_AVG = new SqlAggFunction(
> "UDF_AVG",
> null,
> SqlKind.OTHER_FUNCTION,
> ReturnTypes.AVG_AGG_FUNCTION,
> null,
> OperandTypes.family(SqlTypeFamily.NUMERIC, SqlTypeFamily.NUMERIC),
> SqlFunctionCategory.NUMERIC,
> false,
> false,
> Optionality.FORBIDDEN) {
> @Override public boolean isDeterministic() {
> return false;
> }
> };
> {code}
> The RuntimeExcpetion detail:
> {code:java}
> java.lang.RuntimeException: While invoking method 'public
> org.apache.calcite.sql2rel.RelDecorrelator$Frame
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateRel(org.apache.calcite.rel.logical.LogicalProject)'
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:527)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.getInvoke(RelDecorrelator.java:613)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:254)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:214)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelateQuery(SqlToRelConverter.java:3037)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.decorrelate(SqlToRelConverter.java:481)
> at
> org.apache.calcite.test.SqlToRelTestBase$TesterImpl.convertSqlToRel(SqlToRelTestBase.java:615)
> at
> org.apache.calcite.test.RelOptTestBase.checkPlanning(RelOptTestBase.java:177)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:334)
> at org.apache.calcite.test.RelOptTestBase$Sql.check(RelOptTestBase.java:317)
> at
> org.apache.calcite.test.RelOptRulesTest.testDecorrelateWith2ParamsAgg(RelOptRulesTest.java:4286)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at
[jira] [Commented] (CALCITE-2969) Improve design of join-like relational expressions
[ https://issues.apache.org/jira/browse/CALCITE-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853520#comment-16853520 ] Danny Chan commented on CALCITE-2969: - Thanks [~rubenql], i would recheck the doc to see if i missed something. > Improve design of join-like relational expressions > -- > > Key: CALCITE-2969 > URL: https://issues.apache.org/jira/browse/CALCITE-2969 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.19.0 >Reporter: Stamatis Zampetakis >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 31h > Remaining Estimate: 0h > > The existing join-like (Join, SemiJoin, Correlate, etc.) logical and physical > relational expressions have a few design issues which make some parts of the > codebase complicated and difficult to understand. > The goal of this ticket is to improve the design of the respective > expressions based on the discussion in the dev list (see thread [Join, > SemiJoin, > Correlate|https://mail-archives.apache.org/mod_mbox/calcite-dev/201903.mbox/%3C8EEA04A0-4A77-4283-BD20-B019E19AE126%40apache.org%3E]). > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3102) Deprecation warnings following [CALCITE-2969]
[ https://issues.apache.org/jira/browse/CALCITE-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853515#comment-16853515 ] Danny Chan commented on CALCITE-3102: - Sorry for that [~julianhyde], i should have waited for your review :) It's not that bad cause RelBuilder will not new SemiJoin instance anymore, but i agree with you that removing the code in 1.21 is more acceptable. > Deprecation warnings following [CALCITE-2969] > - > > Key: CALCITE-3102 > URL: https://issues.apache.org/jira/browse/CALCITE-3102 > Project: Calcite > Issue Type: Bug >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Following CALCITE-2969 there are a couple of dozen deprecation warnings. Our > policy is to tolerate no deprecation warnings. So we either need to stop > using deprecated APIs or silence the warnings. > This is a blocker for 1.20. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3100) cast(? as DATE) won't work with PreparedStatement
[
https://issues.apache.org/jira/browse/CALCITE-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852857#comment-16852857
]
Danny Chan commented on CALCITE-3100:
-
This is generated code:
{code:java}
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator =
interpreter.enumerator();
public void reset() {
inputEnumerator.reset();
}
public boolean moveNext() {
while (inputEnumerator.moveNext()) {
// where exception throws out
final Integer inp9_ = (Integer) ((Object[])
inputEnumerator.current())[9];
if (inp9_ != null && (Integer) root.get("?0") != null && inp9_ ==
((Integer) root.get("?0")).intValue()) {
return true;
}
}
return false;
}
{code}
> cast(? as DATE) won't work with PreparedStatement
> -
>
> Key: CALCITE-3100
> URL: https://issues.apache.org/jira/browse/CALCITE-3100
> Project: Calcite
> Issue Type: Bug
>Reporter: Li Xian
>Assignee: Danny Chan
>Priority: Minor
>
> When I do something like
>
> {code:java}
> Connection connection =
> DriverManager.getConnection("jdbc:calcite:", info);
> PreparedStatement ps = connection.prepareStatement("select * from EMPS where
> JOINEDAT=cast(? as DATE)");
> ps.setString(1, "1996-08-03");
> ps.execute();{code}
>
> It will produce error like below
>
> {code:java}
> java.sql.SQLException: exception while executing query: java.lang.String
> cannot be cast to java.lang.Integer
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:577)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.execute(AvaticaPreparedStatement.java:239)
> at org.apache.calcite.test.CsvTest.testDate(CsvTest.java:185)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Integer
> at Baz$1$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:675)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:95)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:184)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:573)
> ... 24 more
> {code}
>
> It looks like that the 'cast' clause
[jira] [Commented] (CALCITE-2973) Allow theta joins that have equi conditions to be executed using a hash join algorithm
[ https://issues.apache.org/jira/browse/CALCITE-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852772#comment-16852772 ] Danny Chan commented on CALCITE-2973: - +1 to skip this patch for 1.20 version. > Allow theta joins that have equi conditions to be executed using a hash join > algorithm > -- > > Key: CALCITE-2973 > URL: https://issues.apache.org/jira/browse/CALCITE-2973 > Project: Calcite > Issue Type: New Feature > Components: core >Affects Versions: 1.19.0 >Reporter: Lai Zhou >Priority: Minor > Labels: pull-request-available > Fix For: 1.20.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Now the EnumerableMergeJoinRule only supports an inner and equi join. > If users make a theta-join query for a large dataset (such as 1*1), > the nested-loop join process will take dozens of time than the sort-merge > join process . > So if we can apply merge-join or hash-join rule for a theta join, it will > improve the performance greatly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (CALCITE-3100) cast(? as DATE) won't work with PreparedStatement
[
https://issues.apache.org/jira/browse/CALCITE-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852760#comment-16852760
]
Danny Chan commented on CALCITE-3100:
-
I have tried all other data types, it seems that all of the arguments would be
force casted to Integer first(except for the Integer type), it's not as
expected, and i'm figuring where the logic is...:(
> cast(? as DATE) won't work with PreparedStatement
> -
>
> Key: CALCITE-3100
> URL: https://issues.apache.org/jira/browse/CALCITE-3100
> Project: Calcite
> Issue Type: Bug
>Reporter: Li Xian
>Assignee: Danny Chan
>Priority: Minor
>
> When I do something like
>
> {code:java}
> Connection connection =
> DriverManager.getConnection("jdbc:calcite:", info);
> PreparedStatement ps = connection.prepareStatement("select * from EMPS where
> JOINEDAT=cast(? as DATE)");
> ps.setString(1, "1996-08-03");
> ps.execute();{code}
>
> It will produce error like below
>
> {code:java}
> java.sql.SQLException: exception while executing query: java.lang.String
> cannot be cast to java.lang.Integer
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:577)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.execute(AvaticaPreparedStatement.java:239)
> at org.apache.calcite.test.CsvTest.testDate(CsvTest.java:185)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
> java.lang.Integer
> at Baz$1$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:675)
> at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:95)
> at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:90)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:184)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:573)
> ... 24 more
> {code}
>
> It looks like that the 'cast' clause doesn't make effect here with the '?'
> placeholder parameter.
> I am able to reproduce this error by adding the below testcase to
> org.apache.calcite.test.CsvTest and run it
> {code:java}
> @Test
> public void testDate() throws Exception {
> Properties info = new Properties();
> info.put("model", jsonPath("model"));
> Connection connection =
> DriverManager.getConnection("jdbc:calcite:", info);
> PreparedStatement ps = connection.prepareStatement("select * from EMPS
> where JOINEDAT=cast(? as DATE)");
> ps.setString(1, "1996-08-03");
> ps.execute();
> }
> {code}
--
This message was sent by Atlassian JIRA
