[jira] [Commented] (CALCITE-3507) Mismatched type AssertionError or incorrect logical plan on correlated query
[
https://issues.apache.org/jira/browse/CALCITE-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975601#comment-16975601
]
Danny Chen commented on CALCITE-3507:
-
Thanks for reporting this [~tc260], supply a test case in RelOptRulesTest would
help a lot to clarify this problem ~
> Mismatched type AssertionError or incorrect logical plan on correlated query
>
>
> Key: CALCITE-3507
> URL: https://issues.apache.org/jira/browse/CALCITE-3507
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.21.0
>Reporter: Tiangang Chen
>Priority: Major
>
> Calcite fails to convert correlated SQL query to logical plan because of
> AssertionError (type mismatch) during decorrelateQuery. Forcing types to
> match results in incorrect logical plan.
> Stack trace:
>
> {code:java}
> Exception in thread "main" java.lang.AssertionError: mismatched type $1
> VARCHAR
> at
> org.apache.calcite.rex.RexUtil$FixNullabilityShuttle.visitInputRef(RexUtil.java:2540)
> at
> org.apache.calcite.rex.RexUtil$FixNullabilityShuttle.visitInputRef(RexUtil.java:2518)
> at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
> at org.apache.calcite.rex.RexShuttle.visitList(RexShuttle.java:149)
> at org.apache.calcite.rex.RexShuttle.visitCall(RexShuttle.java:101)
> at org.apache.calcite.rex.RexShuttle.visitCall(RexShuttle.java:34)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191)
> at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:277)
> at org.apache.calcite.rex.RexShuttle.mutate(RexShuttle.java:239)
> at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:257)
> at org.apache.calcite.rex.RexUtil.fixUp(RexUtil.java:1635)
> at
> org.apache.calcite.rel.rules.FilterJoinRule.perform(FilterJoinRule.java:249)
> at
> org.apache.calcite.rel.rules.FilterJoinRule$FilterIntoJoinRule.onMatch(FilterJoinRule.java:383)
> at
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319)
> at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560)
> at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:419)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256)
> at
> org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215)
> at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:250)
> at
> org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:215)
> at org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:259)
> at RunWithPlanner.main(RunWithPlanner.java:17)
> {code}
>
>
> Code to reproduce:
> {code:java}
> SchemaPlus rootSchema = Frameworks.createRootSchema(true);
> rootSchema.add("LIKES", new AbstractTable() {
> public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
> RelDataTypeFactory.Builder builder = typeFactory.builder();
> builder.add("DRINKER", new BasicSqlType(new RelDataTypeSystemImpl() {
> }, SqlTypeName.VARCHAR));
> builder.add("BEER", new BasicSqlType(new RelDataTypeSystemImpl() {
> }, SqlTypeName.VARCHAR));
> return builder.build();
> }
> });
> rootSchema.add("SERVES", new AbstractTable() {
> public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
> RelDataTypeFactory.Builder builder = typeFactory.builder();
> builder.add("BAR", new BasicSqlType(new RelDataTypeSystemImpl() {
> }, SqlTypeName.VARCHAR));
> builder.add("BEER", new BasicSqlType(new RelDataTypeSystemImpl() {
> }, SqlTypeName.VARCHAR));
> builder.add("PRICE", new BasicSqlType(new RelDataTypeSystemImpl() {
> }, SqlTypeName.DECIMAL));
> return builder.build();
> }
> });
> rootSchema.add("DRINKER", new AbstractTable() {
> public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
> RelDataTypeFactory.Builder builder = typeFactory.builder();
> builder.add("NAME", new BasicSqlType(new RelDataTypeSystemImpl() {
> }, SqlTypeName.VARCHAR));
> builder.add("ADDRESS", new BasicSqlType(new RelDataTypeSystemImpl() {
> }, SqlTypeName.VARCHAR));
> return builder.build();
> }
> });
> rootSchema.add("FREQUENTS", new AbstractTable() {
> public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
> RelDataTypeFactory.Builder builder = typeFactory.builder();
> builder.add("DRINKER", new BasicSqlType(new RelDataTypeSystemImpl() {
> },
[jira] [Commented] (CALCITE-3508) Strengthen outer Join to inner if it is under a Filter that discards null values
[
https://issues.apache.org/jira/browse/CALCITE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975599#comment-16975599
]
Danny Chen commented on CALCITE-3508:
-
I agree with Julian, we may need a method like `Strong.isNotFalse()` to decide
that a filter condition would never returns null or false value.
> Strengthen outer Join to inner if it is under a Filter that discards null
> values
>
>
> Key: CALCITE-3508
> URL: https://issues.apache.org/jira/browse/CALCITE-3508
> Project: Calcite
> Issue Type: Improvement
>Reporter: Scott Reynolds
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner
> Join when the nullable side contains a filter IS_NOT_NULL. Below is the code.
> {code:java}
> for (RexNode filter : aboveFilters) {
> if (joinType.generatesNullsOnLeft()
> && Strong.isNotTrue(filter, leftBitmap)) {
> joinType = joinType.cancelNullsOnLeft();
> }
> if (joinType.generatesNullsOnRight()
> && Strong.isNotTrue(filter, rightBitmap)) {
> joinType = joinType.cancelNullsOnRight();
> }
> if (!joinType.isOuterJoin()) {
> break;
> }
> }
> {code}
> This code looks at the filter to determine if it is always true, then it can
> alter the join type by removing the null on that side.
> We can see this in the following test RelOptRules#testStrengthenJoinType,
> which executes the following SQL that transforms from a LEFT OUTER JOIN to an
> INNER JOIN
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.deptno is not null and emp.sal > 100
> {code}
> This ticket is about broadening the application of this rule to a sql like
> the following:
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.sal > 100
> {code}
> This originally came up on the mailing list:
> [https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
> and in that thread it was pointed out that there are filters that prevent
> this from being applied:
> {code:sql}
> SELECT b.title
> FROM Book b
> LEFT JOIN Author a ON b.author = a.id
> WHERE a.name <> 'Victor'
> {code}
> This means we need to ensure we that the OUTER JOIN doesn't contain – for
> lack of a different term – negation filters. If there is a negation – like
> NOT_EQUAL – the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (CALCITE-3508) Strengthen outer Join to inner if it is under a Filter that discards null values
[
https://issues.apache.org/jira/browse/CALCITE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975520#comment-16975520
]
Julian Hyde edited comment on CALCITE-3508 at 11/16/19 1:53 AM:
I changed the subject, because "FILTER clause" is an ambiguous - some people
might think it referred to, say, "SUM\(x) FILTER (WHERE y > 0)".
was (Author: julianhyde):
I changed the subject, because "FILTER clause" is an ambiguous - some people
might think it referred to, say, "SUM(x) FILTER (WHERE y > 0)".
> Strengthen outer Join to inner if it is under a Filter that discards null
> values
>
>
> Key: CALCITE-3508
> URL: https://issues.apache.org/jira/browse/CALCITE-3508
> Project: Calcite
> Issue Type: Improvement
>Reporter: Scott Reynolds
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner
> Join when the nullable side contains a filter IS_NOT_NULL. Below is the code.
> {code:java}
> for (RexNode filter : aboveFilters) {
> if (joinType.generatesNullsOnLeft()
> && Strong.isNotTrue(filter, leftBitmap)) {
> joinType = joinType.cancelNullsOnLeft();
> }
> if (joinType.generatesNullsOnRight()
> && Strong.isNotTrue(filter, rightBitmap)) {
> joinType = joinType.cancelNullsOnRight();
> }
> if (!joinType.isOuterJoin()) {
> break;
> }
> }
> {code}
> This code looks at the filter to determine if it is always true, then it can
> alter the join type by removing the null on that side.
> We can see this in the following test RelOptRules#testStrengthenJoinType,
> which executes the following SQL that transforms from a LEFT OUTER JOIN to an
> INNER JOIN
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.deptno is not null and emp.sal > 100
> {code}
> This ticket is about broadening the application of this rule to a sql like
> the following:
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.sal > 100
> {code}
> This originally came up on the mailing list:
> [https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
> and in that thread it was pointed out that there are filters that prevent
> this from being applied:
> {code:sql}
> SELECT b.title
> FROM Book b
> LEFT JOIN Author a ON b.author = a.id
> WHERE a.name <> 'Victor'
> {code}
> This means we need to ensure we that the OUTER JOIN doesn't contain – for
> lack of a different term – negation filters. If there is a negation – like
> NOT_EQUAL – the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3508) Strengthen outer Join to inner if it is under a Filter that discards null values
[
https://issues.apache.org/jira/browse/CALCITE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975520#comment-16975520
]
Julian Hyde commented on CALCITE-3508:
--
I changed the subject, because "FILTER clause" is an ambiguous - some people
might think it referred to, say, "SUM(x) FILTER (WHERE y > 0)".
> Strengthen outer Join to inner if it is under a Filter that discards null
> values
>
>
> Key: CALCITE-3508
> URL: https://issues.apache.org/jira/browse/CALCITE-3508
> Project: Calcite
> Issue Type: Improvement
>Reporter: Scott Reynolds
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner
> Join when the nullable side contains a filter IS_NOT_NULL. Below is the code.
> {code:java}
> for (RexNode filter : aboveFilters) {
> if (joinType.generatesNullsOnLeft()
> && Strong.isNotTrue(filter, leftBitmap)) {
> joinType = joinType.cancelNullsOnLeft();
> }
> if (joinType.generatesNullsOnRight()
> && Strong.isNotTrue(filter, rightBitmap)) {
> joinType = joinType.cancelNullsOnRight();
> }
> if (!joinType.isOuterJoin()) {
> break;
> }
> }
> {code}
> This code looks at the filter to determine if it is always true, then it can
> alter the join type by removing the null on that side.
> We can see this in the following test RelOptRules#testStrengthenJoinType,
> which executes the following SQL that transforms from a LEFT OUTER JOIN to an
> INNER JOIN
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.deptno is not null and emp.sal > 100
> {code}
> This ticket is about broadening the application of this rule to a sql like
> the following:
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.sal > 100
> {code}
> This originally came up on the mailing list:
> [https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
> and in that thread it was pointed out that there are filters that prevent
> this from being applied:
> {code:sql}
> SELECT b.title
> FROM Book b
> LEFT JOIN Author a ON b.author = a.id
> WHERE a.name <> 'Victor'
> {code}
> This means we need to ensure we that the OUTER JOIN doesn't contain – for
> lack of a different term – negation filters. If there is a negation – like
> NOT_EQUAL – the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3508) Strengthen outer Join to inner if it is under a Filter that discards null values
[
https://issues.apache.org/jira/browse/CALCITE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde updated CALCITE-3508:
-
Summary: Strengthen outer Join to inner if it is under a Filter that
discards null values (was: Strengthen Outer Joins based on FILTER clauses)
> Strengthen outer Join to inner if it is under a Filter that discards null
> values
>
>
> Key: CALCITE-3508
> URL: https://issues.apache.org/jira/browse/CALCITE-3508
> Project: Calcite
> Issue Type: Improvement
>Reporter: Scott Reynolds
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner
> Join when the nullable side contains a filter IS_NOT_NULL. Below is the code.
> {code:java}
> for (RexNode filter : aboveFilters) {
> if (joinType.generatesNullsOnLeft()
> && Strong.isNotTrue(filter, leftBitmap)) {
> joinType = joinType.cancelNullsOnLeft();
> }
> if (joinType.generatesNullsOnRight()
> && Strong.isNotTrue(filter, rightBitmap)) {
> joinType = joinType.cancelNullsOnRight();
> }
> if (!joinType.isOuterJoin()) {
> break;
> }
> }
> {code}
> This code looks at the filter to determine if it is always true, then it can
> alter the join type by removing the null on that side.
> We can see this in the following test RelOptRules#testStrengthenJoinType,
> which executes the following SQL that transforms from a LEFT OUTER JOIN to an
> INNER JOIN
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.deptno is not null and emp.sal > 100
> {code}
> This ticket is about broadening the application of this rule to a sql like
> the following:
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.sal > 100
> {code}
> This originally came up on the mailing list:
> [https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
> and in that thread it was pointed out that there are filters that prevent
> this from being applied:
> {code:sql}
> SELECT b.title
> FROM Book b
> LEFT JOIN Author a ON b.author = a.id
> WHERE a.name <> 'Victor'
> {code}
> This means we need to ensure we that the OUTER JOIN doesn't contain – for
> lack of a different term – negation filters. If there is a negation – like
> NOT_EQUAL – the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3508) Strengthen Outer Joins based on FILTER clauses
[
https://issues.apache.org/jira/browse/CALCITE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975517#comment-16975517
]
Julian Hyde commented on CALCITE-3508:
--
I'm surprised that this doesn't use {{class Strong}}. Did you consider using
it, the way that {{JoinProjectTransposeRule}} and other code does.
The email archive link isn't very precise. Did you mean [this
email|https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/%3CCAGzrZ38hh%2B8B8TG8gVHUfVKunJZ4L%2BK7rmrinQxpwOHcdJbzGQ%40mail.gmail.com%3E]?
> Strengthen Outer Joins based on FILTER clauses
> --
>
> Key: CALCITE-3508
> URL: https://issues.apache.org/jira/browse/CALCITE-3508
> Project: Calcite
> Issue Type: Improvement
>Reporter: Scott Reynolds
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner
> Join when the nullable side contains a filter IS_NOT_NULL. Below is the code.
> {code:java}
> for (RexNode filter : aboveFilters) {
> if (joinType.generatesNullsOnLeft()
> && Strong.isNotTrue(filter, leftBitmap)) {
> joinType = joinType.cancelNullsOnLeft();
> }
> if (joinType.generatesNullsOnRight()
> && Strong.isNotTrue(filter, rightBitmap)) {
> joinType = joinType.cancelNullsOnRight();
> }
> if (!joinType.isOuterJoin()) {
> break;
> }
> }
> {code}
> This code looks at the filter to determine if it is always true, then it can
> alter the join type by removing the null on that side.
> We can see this in the following test RelOptRules#testStrengthenJoinType,
> which executes the following SQL that transforms from a LEFT OUTER JOIN to an
> INNER JOIN
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.deptno is not null and emp.sal > 100
> {code}
> This ticket is about broadening the application of this rule to a sql like
> the following:
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.sal > 100
> {code}
> This originally came up on the mailing list:
> [https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
> and in that thread it was pointed out that there are filters that prevent
> this from being applied:
> {code:sql}
> SELECT b.title
> FROM Book b
> LEFT JOIN Author a ON b.author = a.id
> WHERE a.name <> 'Victor'
> {code}
> This means we need to ensure we that the OUTER JOIN doesn't contain – for
> lack of a different term – negation filters. If there is a negation – like
> NOT_EQUAL – the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3508) Strengthen Outer Joins based on FILTER clauses
[
https://issues.apache.org/jira/browse/CALCITE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Reynolds updated CALCITE-3508:
Summary: Strengthen Outer Joins based on FILTER clauses (was: Strengthen
JOINs when Filter enforces the nullable side(s) to non-nullable)
> Strengthen Outer Joins based on FILTER clauses
> --
>
> Key: CALCITE-3508
> URL: https://issues.apache.org/jira/browse/CALCITE-3508
> Project: Calcite
> Issue Type: Improvement
>Reporter: Scott Reynolds
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner
> Join when the nullable side contains a filter IS_NOT_NULL. Below is the code.
> {code:java}
> for (RexNode filter : aboveFilters) {
> if (joinType.generatesNullsOnLeft()
> && Strong.isNotTrue(filter, leftBitmap)) {
> joinType = joinType.cancelNullsOnLeft();
> }
> if (joinType.generatesNullsOnRight()
> && Strong.isNotTrue(filter, rightBitmap)) {
> joinType = joinType.cancelNullsOnRight();
> }
> if (!joinType.isOuterJoin()) {
> break;
> }
> }
> {code}
> This code looks at the filter to determine if it is always true, then it can
> alter the join type by removing the null on that side.
> We can see this in the following test RelOptRules#testStrengthenJoinType,
> which executes the following SQL that transforms from a LEFT OUTER JOIN to an
> INNER JOIN
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.deptno is not null and emp.sal > 100
> {code}
> This ticket is about broadening the application of this rule to a sql like
> the following:
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.sal > 100
> {code}
> This originally came up on the mailing list:
> [https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
> and in that thread it was pointed out that there are filters that prevent
> this from being applied:
> {code:sql}
> SELECT b.title
> FROM Book b
> LEFT JOIN Author a ON b.author = a.id
> WHERE a.name <> 'Victor'
> {code}
> This means we need to ensure we that the OUTER JOIN doesn't contain – for
> lack of a different term – negation filters. If there is a negation – like
> NOT_EQUAL – the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3508) Strengthen JOINs when Filter enforces the nullable side(s) to non-nullable
[
https://issues.apache.org/jira/browse/CALCITE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-3508:
Labels: pull-request-available (was: )
> Strengthen JOINs when Filter enforces the nullable side(s) to non-nullable
> --
>
> Key: CALCITE-3508
> URL: https://issues.apache.org/jira/browse/CALCITE-3508
> Project: Calcite
> Issue Type: Improvement
>Reporter: Scott Reynolds
>Priority: Major
> Labels: pull-request-available
>
> Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner
> Join when the nullable side contains a filter IS_NOT_NULL. Below is the code.
> {code:java}
> for (RexNode filter : aboveFilters) {
> if (joinType.generatesNullsOnLeft()
> && Strong.isNotTrue(filter, leftBitmap)) {
> joinType = joinType.cancelNullsOnLeft();
> }
> if (joinType.generatesNullsOnRight()
> && Strong.isNotTrue(filter, rightBitmap)) {
> joinType = joinType.cancelNullsOnRight();
> }
> if (!joinType.isOuterJoin()) {
> break;
> }
> }
> {code}
> This code looks at the filter to determine if it is always true, then it can
> alter the join type by removing the null on that side.
> We can see this in the following test RelOptRules#testStrengthenJoinType,
> which executes the following SQL that transforms from a LEFT OUTER JOIN to an
> INNER JOIN
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.deptno is not null and emp.sal > 100
> {code}
> This ticket is about broadening the application of this rule to a sql like
> the following:
> {code:sql}
> select *
> from dept left join emp on dept.deptno = emp.deptno
> where emp.sal > 100
> {code}
> This originally came up on the mailing list:
> [https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
> and in that thread it was pointed out that there are filters that prevent
> this from being applied:
> {code:sql}
> SELECT b.title
> FROM Book b
> LEFT JOIN Author a ON b.author = a.id
> WHERE a.name <> 'Victor'
> {code}
> This means we need to ensure we that the OUTER JOIN doesn't contain – for
> lack of a different term – negation filters. If there is a negation – like
> NOT_EQUAL – the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3508) Strengthen JOINs when Filter enforces the nullable side(s) to non-nullable
Scott Reynolds created CALCITE-3508:
---
Summary: Strengthen JOINs when Filter enforces the nullable
side(s) to non-nullable
Key: CALCITE-3508
URL: https://issues.apache.org/jira/browse/CALCITE-3508
Project: Calcite
Issue Type: Improvement
Reporter: Scott Reynolds
Today, FilterJoinRule given an Outer Join the rule strengthens it to Inner Join
when the nullable side contains a filter IS_NOT_NULL. Below is the code.
{code:java}
for (RexNode filter : aboveFilters) {
if (joinType.generatesNullsOnLeft()
&& Strong.isNotTrue(filter, leftBitmap)) {
joinType = joinType.cancelNullsOnLeft();
}
if (joinType.generatesNullsOnRight()
&& Strong.isNotTrue(filter, rightBitmap)) {
joinType = joinType.cancelNullsOnRight();
}
if (!joinType.isOuterJoin()) {
break;
}
}
{code}
This code looks at the filter to determine if it is always true, then it can
alter the join type by removing the null on that side.
We can see this in the following test RelOptRules#testStrengthenJoinType, which
executes the following SQL that transforms from a LEFT OUTER JOIN to an INNER
JOIN
{code:sql}
select *
from dept left join emp on dept.deptno = emp.deptno
where emp.deptno is not null and emp.sal > 100
{code}
This ticket is about broadening the application of this rule to a sql like the
following:
{code:sql}
select *
from dept left join emp on dept.deptno = emp.deptno
where emp.sal > 100
{code}
This originally came up on the mailing list:
[https://mail-archives.apache.org/mod_mbox/calcite-dev/201909.mbox/browser]
and in that thread it was pointed out that there are filters that prevent this
from being applied:
{code:sql}
SELECT b.title
FROM Book b
LEFT JOIN Author a ON b.author = a.id
WHERE a.name <> 'Victor'
{code}
This means we need to ensure we that the OUTER JOIN doesn't contain – for lack
of a different term – negation filters. If there is a negation – like NOT_EQUAL
– the JOIN cannot be strengthened.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-3507) Mismatched type AssertionError or incorrect logical plan on correlated query
Tiangang Chen created CALCITE-3507:
--
Summary: Mismatched type AssertionError or incorrect logical plan
on correlated query
Key: CALCITE-3507
URL: https://issues.apache.org/jira/browse/CALCITE-3507
Project: Calcite
Issue Type: Bug
Affects Versions: 1.21.0
Reporter: Tiangang Chen
Calcite fails to convert correlated SQL query to logical plan because of
AssertionError (type mismatch) during decorrelateQuery. Forcing types to match
results in incorrect logical plan.
Stack trace:
{code:java}
Exception in thread "main" java.lang.AssertionError: mismatched type $1 VARCHAR
at
org.apache.calcite.rex.RexUtil$FixNullabilityShuttle.visitInputRef(RexUtil.java:2540)
at
org.apache.calcite.rex.RexUtil$FixNullabilityShuttle.visitInputRef(RexUtil.java:2518)
at org.apache.calcite.rex.RexInputRef.accept(RexInputRef.java:112)
at org.apache.calcite.rex.RexShuttle.visitList(RexShuttle.java:149)
at org.apache.calcite.rex.RexShuttle.visitCall(RexShuttle.java:101)
at org.apache.calcite.rex.RexShuttle.visitCall(RexShuttle.java:34)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:191)
at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:277)
at org.apache.calcite.rex.RexShuttle.mutate(RexShuttle.java:239)
at org.apache.calcite.rex.RexShuttle.apply(RexShuttle.java:257)
at org.apache.calcite.rex.RexUtil.fixUp(RexUtil.java:1635)
at
org.apache.calcite.rel.rules.FilterJoinRule.perform(FilterJoinRule.java:249)
at
org.apache.calcite.rel.rules.FilterJoinRule$FilterIntoJoinRule.onMatch(FilterJoinRule.java:383)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:560)
at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:419)
at
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:256)
at
org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:215)
at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:202)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelate(RelDecorrelator.java:250)
at
org.apache.calcite.sql2rel.RelDecorrelator.decorrelateQuery(RelDecorrelator.java:215)
at org.apache.calcite.prepare.PlannerImpl.rel(PlannerImpl.java:259)
at RunWithPlanner.main(RunWithPlanner.java:17)
{code}
Code to reproduce:
{code:java}
SchemaPlus rootSchema = Frameworks.createRootSchema(true);
rootSchema.add("LIKES", new AbstractTable() {
public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
RelDataTypeFactory.Builder builder = typeFactory.builder();
builder.add("DRINKER", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
builder.add("BEER", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
return builder.build();
}
});
rootSchema.add("SERVES", new AbstractTable() {
public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
RelDataTypeFactory.Builder builder = typeFactory.builder();
builder.add("BAR", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
builder.add("BEER", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
builder.add("PRICE", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.DECIMAL));
return builder.build();
}
});
rootSchema.add("DRINKER", new AbstractTable() {
public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
RelDataTypeFactory.Builder builder = typeFactory.builder();
builder.add("NAME", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
builder.add("ADDRESS", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
return builder.build();
}
});
rootSchema.add("FREQUENTS", new AbstractTable() {
public RelDataType getRowType(final RelDataTypeFactory typeFactory) {
RelDataTypeFactory.Builder builder = typeFactory.builder();
builder.add("DRINKER", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
builder.add("BAR", new BasicSqlType(new RelDataTypeSystemImpl() {
}, SqlTypeName.VARCHAR));
builder.add("TIMES_A_WEEK", new BasicSqlType(new
RelDataTypeSystemImpl() {
}, SqlTypeName.SMALLINT));
return builder.build();
}
});
String SAMPLE_QUERY = "SELECT Frequents.drinker, Frequents.bar\n" +
"FROM Frequents\n" +
"WHERE NOT EXISTS(SELECT * FROM Likes, Serves\n" +
"WHERE Likes.drinker = Frequents.drinker\n" +
"AND Serves.bar = Frequents.bar\n" +
[jira] [Updated] (CALCITE-3491) VolcanoPlanner.completeConversion() is bypassed by "if (true)"
[
https://issues.apache.org/jira/browse/CALCITE-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haisheng Yuan updated CALCITE-3491:
---
Issue Type: Improvement (was: Bug)
> VolcanoPlanner.completeConversion() is bypassed by "if (true)"
> --
>
> Key: CALCITE-3491
> URL: https://issues.apache.org/jira/browse/CALCITE-3491
> Project: Calcite
> Issue Type: Improvement
>Reporter: Xiening Dai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Not very clear about why we need VolcanoPlanner.completeConversion(). It's
> currently bypassed as below[1] -
>if (true) {
> return rel;
> }
> If the method is not needed, we should just remove it.
> [1]
> https://github.com/apache/calcite/blob/f90e5d7b48848191c1a70e6c0f78eece93433806/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L1096
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-3491) VolcanoPlanner.completeConversion() is bypassed by "if (true)"
[
https://issues.apache.org/jira/browse/CALCITE-3491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haisheng Yuan resolved CALCITE-3491.
Fix Version/s: 1.22.0
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/301bbe9e132346f60587e8c21c474f7c51b236c3.
> VolcanoPlanner.completeConversion() is bypassed by "if (true)"
> --
>
> Key: CALCITE-3491
> URL: https://issues.apache.org/jira/browse/CALCITE-3491
> Project: Calcite
> Issue Type: Improvement
>Reporter: Xiening Dai
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Not very clear about why we need VolcanoPlanner.completeConversion(). It's
> currently bypassed as below[1] -
>if (true) {
> return rel;
> }
> If the method is not needed, we should just remove it.
> [1]
> https://github.com/apache/calcite/blob/f90e5d7b48848191c1a70e6c0f78eece93433806/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoPlanner.java#L1096
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3501) Improve EnumerableTableFunctionScan's implement() for table value function windowing
[ https://issues.apache.org/jira/browse/CALCITE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated CALCITE-3501: -- Description: In order to support table value function windowing, EnumerableTableFunctionScan's implement() should have an implementation which generates code that iterates input, calls the RexCall and generates output (can refer to the implementation of EnumerableCalc). For example, assume the input of table function is: col1 timestamp, col2 int, col3 char enumerable table function scan should emit col1 timestamp, col2 int, col3 char, wstart timestamp, wend timestamp was: In order to support table value function windowing, EnumerableTableFunctionScan's implement() should have an implementation which generates code that iterates input, calls the RexCall and generates output (can refer to the implementation of EnumerableCalc). For example, assume the input of table function is: col1 timestamp, col2 int, col3 char enumerable table function scan should emit col1 timestamp, col2 int, col3 char, window_start timestamp, window_end timestamp > Improve EnumerableTableFunctionScan's implement() for table value function > windowing > > > Key: CALCITE-3501 > URL: https://issues.apache.org/jira/browse/CALCITE-3501 > Project: Calcite > Issue Type: Sub-task >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > > In order to support table value function windowing, > EnumerableTableFunctionScan's implement() should have an implementation which > generates code that iterates input, calls the RexCall and generates output > (can refer to the implementation of EnumerableCalc). > For example, assume the input of table function is: > col1 timestamp, col2 int, col3 char > enumerable table function scan should emit > col1 timestamp, col2 int, col3 char, wstart timestamp, wend timestamp -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3506) ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is used
[
https://issues.apache.org/jira/browse/CALCITE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975267#comment-16975267
]
Josh Elser commented on CALCITE-3506:
-
{quote}I don't longer have the exception, so it would be useful to know why the
cast from Float to long is needed. This doesn't mean this is the solution, but
I just wanted to understand if this was the real issue in my system.
{quote}
I wouldn't be surprised if it's unnecessary/wrong at this point, just wanted to
point out that I didn't think a real query would actually hit this code path
(which begs the question about the failure you saw).
If you can figure out a query which causes the original server-side issue, you
can always turn on TRACE (i think) for Avatica and see the data flowing over
the wire which should help understand what was actually serialized.
> ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is
> used
>
>
> Key: CALCITE-3506
> URL: https://issues.apache.org/jira/browse/CALCITE-3506
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.12.0
>Reporter: Enrique Saurez
>Priority: Major
>
> am using Apache Calcite-Avatica version 1.12 (but the relevant code
> sections are not different from the master branch), and I am getting
> the following exception on the client side (but the actual error in on
> the server side):
> ||Exception||
> |org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) :
> Remote driver error: ClassCastException: java.lang.Long cannot be cast
> to java.lang.Float
> at org.apache.calcite.avatica.Helper.createException(Helper.java:54)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:557)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.getCustomerByName(Payment.java:400)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.run(Payment.java:221)
> at
> com.oltpbenchmark.benchmarks.tpcc.TPCCWorker.executeWork(TPCCWorker.java:74)
> at com.oltpbenchmark.api.Worker.doWork(Worker.java:386)
> at com.oltpbenchmark.api.Worker.run(Worker.java:296)
> at java.lang.Thread.run(Thread.java:748)
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Float
> at
> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:594)
> at
> org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:799)
> at
> org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:985)
> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:971)
> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:936)
> at
> org.apache.calcite.avatica.remote.Service$ResultSetResponse.serialize(Service.java:841)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1158)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1113)
> at
> org.apache.calcite.avatica.remote.ProtobufTranslationImpl.serializeResponse(ProtobufTranslationImpl.java:348)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:57)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:31)
> at
> org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:95)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:46)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:127)
> at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> [org.eclipse.jetty.io|http://org.eclipse.jetty.io/].AbstractConnection$2.run(AbstractConnection.java:544)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:748)|
> From the code, it seems like when the function
[jira] [Commented] (CALCITE-3506) ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is used
[
https://issues.apache.org/jira/browse/CALCITE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975250#comment-16975250
]
Enrique Saurez commented on CALCITE-3506:
-
I agree that my test is not something useful to add to the codebase, I just
wanted to show an example of how an exception like mine could be triggered. My
problem is that it is hard for me to create a better test case, because in my
use case I am only using Avatica, and not the "normal" backend database. And I
haven't being able to find the exact query in my system that is triggering this
exception (just because of lack of visibility in my current test environment, I
will try to invest a little more hours into this, to try to give you something
more concrete), but I have a feeling that I am treating Floats differently that
the backend used in your unit test.
After changing the Avatica code from:
{code:java}
writeToProtoWithType(builder, ((Float) o).longValue(), Common.Rep.FLOAT);
{code}
to
{code:java}
writeToProtoWithType(builder, (Float) o, Common.Rep.FLOAT);
{code}
I don't longer have the exception, so it would be useful to know why the cast
from Float to long is needed. This doesn't mean this is the solution, but I
just wanted to understand if this was the real issue in my system.
I am also having trouble setting up the Avatica codebase with my IDE (Intellij
IDEA), so it is hard for me to step-through your unit test and then can check
the path in my backend. I am still not too familiar with Avatica codebase.
Thanks again for helping :)
> ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is
> used
>
>
> Key: CALCITE-3506
> URL: https://issues.apache.org/jira/browse/CALCITE-3506
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.12.0
>Reporter: Enrique Saurez
>Priority: Major
>
> am using Apache Calcite-Avatica version 1.12 (but the relevant code
> sections are not different from the master branch), and I am getting
> the following exception on the client side (but the actual error in on
> the server side):
> ||Exception||
> |org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) :
> Remote driver error: ClassCastException: java.lang.Long cannot be cast
> to java.lang.Float
> at org.apache.calcite.avatica.Helper.createException(Helper.java:54)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:557)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.getCustomerByName(Payment.java:400)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.run(Payment.java:221)
> at
> com.oltpbenchmark.benchmarks.tpcc.TPCCWorker.executeWork(TPCCWorker.java:74)
> at com.oltpbenchmark.api.Worker.doWork(Worker.java:386)
> at com.oltpbenchmark.api.Worker.run(Worker.java:296)
> at java.lang.Thread.run(Thread.java:748)
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Float
> at
> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:594)
> at
> org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:799)
> at
> org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:985)
> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:971)
> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:936)
> at
> org.apache.calcite.avatica.remote.Service$ResultSetResponse.serialize(Service.java:841)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1158)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1113)
> at
> org.apache.calcite.avatica.remote.ProtobufTranslationImpl.serializeResponse(ProtobufTranslationImpl.java:348)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:57)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:31)
> at
> org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:95)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:46)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:127)
> at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
> at
>
[jira] [Commented] (CALCITE-3506) ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is used
[
https://issues.apache.org/jira/browse/CALCITE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975243#comment-16975243
]
Josh Elser commented on CALCITE-3506:
-
I've spent some time messing around with this. TypedValue is a mess because
it's trying to reconcile all of these different representations of the same
data.
I can see the issue as you describe it, but the code snippet you suggest to add
to TypedValueTest doesn't really test what the code itself is actually doing.
It would be better to get a unit test reproducing the issue using a {{Frame}}
or a real query. For example, the following test which I added to
RemoteMetaTest.java passes without your change:
{code:java}
@Test public void testFloats() throws Exception {
final float floatValue = 3.14159f;
ConnectionSpec.getDatabaseLock().lock();
try (final Connection conn = DriverManager.getConnection(url);
final Statement stmt = conn.createStatement()) {
stmt.execute("DROP TABLE IF EXISTS testFloats");
stmt.execute("CREATE TABLE testFloats(id integer primary key, f float)");
try (final PreparedStatement pstmt = conn.prepareStatement("INSERT INTO
testFloats values(?,?)")) {
pstmt.setInt(1, 1);
pstmt.setFloat(2, floatValue);
assertEquals(1, pstmt.executeUpdate());
}
ResultSet results = stmt.executeQuery("SELECT * from testFloats");
assertNotNull(results);
assertTrue(results.next());
assertEquals(1, results.getInt(1));
float actual = results.getFloat(2);
assertTrue("Expected float values to be equal, but was " + actual,
floatValue == actual);
} finally {
ConnectionSpec.getDatabaseLock().unlock();
}
} {code}
> ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is
> used
>
>
> Key: CALCITE-3506
> URL: https://issues.apache.org/jira/browse/CALCITE-3506
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.12.0
>Reporter: Enrique Saurez
>Priority: Major
>
> am using Apache Calcite-Avatica version 1.12 (but the relevant code
> sections are not different from the master branch), and I am getting
> the following exception on the client side (but the actual error in on
> the server side):
> ||Exception||
> |org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) :
> Remote driver error: ClassCastException: java.lang.Long cannot be cast
> to java.lang.Float
> at org.apache.calcite.avatica.Helper.createException(Helper.java:54)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:557)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.getCustomerByName(Payment.java:400)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.run(Payment.java:221)
> at
> com.oltpbenchmark.benchmarks.tpcc.TPCCWorker.executeWork(TPCCWorker.java:74)
> at com.oltpbenchmark.api.Worker.doWork(Worker.java:386)
> at com.oltpbenchmark.api.Worker.run(Worker.java:296)
> at java.lang.Thread.run(Thread.java:748)
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Float
> at
> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:594)
> at
> org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:799)
> at
> org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:985)
> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:971)
> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:936)
> at
> org.apache.calcite.avatica.remote.Service$ResultSetResponse.serialize(Service.java:841)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1158)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1113)
> at
> org.apache.calcite.avatica.remote.ProtobufTranslationImpl.serializeResponse(ProtobufTranslationImpl.java:348)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:57)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:31)
> at
> org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:95)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:46)
> at
>
[jira] [Commented] (CALCITE-3477) Decimal type should not be assigned from other types
[
https://issues.apache.org/jira/browse/CALCITE-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975238#comment-16975238
]
Rui Wang commented on CALCITE-3477:
---
at least based on the implicit type cast design, these types are allowed to
cast to decimal. See:
https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit
> Decimal type should not be assigned from other types
>
>
> Key: CALCITE-3477
> URL: https://issues.apache.org/jira/browse/CALCITE-3477
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Kirils Mensikovs
>Assignee: Feng Zhu
>Priority: Major
> Labels: geospatial, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Geospatial function with integer parameter fails. The expected behavior is to
> cast automatically all number values to BigDecimal.
> {{Example: 'select ST_MAKEPOINT(1.0, 1)'}}
> Return:
> {code:java}
> Error: Error while executing SQL "select ST_MAKEPOINT(1.0, 1)": Error while
> compiling generated Java code:
> public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root) {
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable =
> org.apache.calcite.linq4j.Linq4j.asEnumerable(new Integer[] {
> 0});
> return new org.apache.calcite.linq4j.AbstractEnumerable(){
> public org.apache.calcite.linq4j.Enumerator enumerator() {
> return new org.apache.calcite.linq4j.Enumerator(){
> public final org.apache.calcite.linq4j.Enumerator inputEnumerator
> = _inputEnumerable.enumerator();
> public void reset() {
> inputEnumerator.reset();
> }
> public boolean moveNext() {
> return inputEnumerator.moveNext();
> }
> public void close() {
> inputEnumerator.close();
> }
> public Object current() {
> final java.math.BigDecimal v =
> $L4J$C$new_java_math_BigDecimal_1_0_;
> return org.apache.calcite.runtime.GeoFunctions.ST_MakePoint(v,
> 1);
> }
> static final java.math.BigDecimal
> $L4J$C$new_java_math_BigDecimal_1_0_ = new java.math.BigDecimal(
> "1.0");
> };
> }
> };
> }
> public Class getElementType() {
> return org.apache.calcite.runtime.GeoFunctions.Geom.class;
> } (state=,code=0)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3506) ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is used
[
https://issues.apache.org/jira/browse/CALCITE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975227#comment-16975227
]
Danny Chen commented on CALCITE-3506:
-
Thanks for the reporting, i would take a look tomorrow ~
> ClassCastException in TypedValue.writetoProtoWithType when a boxed Float is
> used
>
>
> Key: CALCITE-3506
> URL: https://issues.apache.org/jira/browse/CALCITE-3506
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.12.0
>Reporter: Enrique Saurez
>Priority: Major
>
> am using Apache Calcite-Avatica version 1.12 (but the relevant code
> sections are not different from the master branch), and I am getting
> the following exception on the client side (but the actual error in on
> the server side):
> ||Exception||
> |org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) :
> Remote driver error: ClassCastException: java.lang.Long cannot be cast
> to java.lang.Float
> at org.apache.calcite.avatica.Helper.createException(Helper.java:54)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:557)
> at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.getCustomerByName(Payment.java:400)
> at
> com.oltpbenchmark.benchmarks.tpcc.procedures.Payment.run(Payment.java:221)
> at
> com.oltpbenchmark.benchmarks.tpcc.TPCCWorker.executeWork(TPCCWorker.java:74)
> at com.oltpbenchmark.api.Worker.doWork(Worker.java:386)
> at com.oltpbenchmark.api.Worker.run(Worker.java:296)
> at java.lang.Thread.run(Thread.java:748)
> java.lang.ClassCastException: java.lang.Long cannot be cast to
> java.lang.Float
> at
> org.apache.calcite.avatica.remote.TypedValue.writeToProtoWithType(TypedValue.java:594)
> at
> org.apache.calcite.avatica.remote.TypedValue.toProto(TypedValue.java:799)
> at
> org.apache.calcite.avatica.Meta$Frame.serializeScalar(Meta.java:985)
> at org.apache.calcite.avatica.Meta$Frame.parseColumn(Meta.java:971)
> at org.apache.calcite.avatica.Meta$Frame.toProto(Meta.java:936)
> at
> org.apache.calcite.avatica.remote.Service$ResultSetResponse.serialize(Service.java:841)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1158)
> at
> org.apache.calcite.avatica.remote.Service$ExecuteResponse.serialize(Service.java:1113)
> at
> org.apache.calcite.avatica.remote.ProtobufTranslationImpl.serializeResponse(ProtobufTranslationImpl.java:348)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:57)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.encode(ProtobufHandler.java:31)
> at
> org.apache.calcite.avatica.remote.AbstractHandler.apply(AbstractHandler.java:95)
> at
> org.apache.calcite.avatica.remote.ProtobufHandler.apply(ProtobufHandler.java:46)
> at
> org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:127)
> at
> org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> [org.eclipse.jetty.io|http://org.eclipse.jetty.io/].AbstractConnection$2.run(AbstractConnection.java:544)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:748)|
> From the code, it seems like when the function "writeToProtoWithType"
> is called from "toProto", there is a boxing conversion from long
> (primitive) to Long (object), this is because
> for floats, "toProto" is using "((Float) o).longValue()" which returns
> a long. Then in "writeToProtoWithType" is being casted to float, which
> I think causes the CastClassException.
> If I add this code to the testFloat() function in the
> "core/src/test/java/org/apache/calcite/avatica/remote/TypedValueTest.java"
> file:
> {code:java}
> Common.TypedValue.Builder builder = Common.TypedValue.newBuilder();
> Common.Rep val = TypedValue.toProto(builder,
[jira] [Resolved] (CALCITE-3498) Unnest operation's ordinality should be deterministic
[
https://issues.apache.org/jira/browse/CALCITE-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruben Q L resolved CALCITE-3498.
Fix Version/s: 1.22.0
Resolution: Fixed
> Unnest operation's ordinality should be deterministic
> -
>
> Key: CALCITE-3498
> URL: https://issues.apache.org/jira/browse/CALCITE-3498
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
> Environment: **_**_**
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Current now, the compution of unnest operation's ordinality is conducted in
> *_current()_* method (
> [https://github.com/apache/calcite/blob/3853118b4d1d48f7d3970b5488baf7ca78d5028e/core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java#L2764).]
> Consequently, when the method is called, the ordinality will be incremented,
> leading to incorrect and un-deterministic result.
> We can use a simple query for illustration.
> {code:java}
> Query:
> ==
> select v, o
> from unnest(array[100,200]) with ordinality as t1(v, o)
> where v > 1
> Expected Result
> ==
> V=100; O=1
> V=200; O=2
> {code}
> However, we get the follow incorrect result.
> {code:java}
> V=100; O=2
> V=200; O=4
> {code}
> We can infer to the code generated. It can be seen that
> *inputEnumerator.current()* has been called two times, one for filter (v>1)
> and one for select (v, o).
> {code:java}
> public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root) {
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable = ..
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable0 =
> child.selectMany(org.apache.calcite.runtime.SqlFunctions.flatProduct(new
> int[] {
> -1}, true, new
> org.apache.calcite.runtime.SqlFunctions.FlatProductInputType[] {
> org.apache.calcite.runtime.SqlFunctions.FlatProductInputType.SCALAR}));
> return new org.apache.calcite.linq4j.AbstractEnumerable(){
> public org.apache.calcite.linq4j.Enumerator enumerator() {
> return new org.apache.calcite.linq4j.Enumerator(){
> public final org.apache.calcite.linq4j.Enumerator inputEnumerator
> = _inputEnumerable0.enumerator();
> public void reset() {
> inputEnumerator.reset();
> }public boolean moveNext() {
> while (inputEnumerator.moveNext()) {
> if
> (org.apache.calcite.runtime.SqlFunctions.toInt(((org.apache.calcite.runtime.FlatLists.ComparableList)
> inputEnumerator.current()).get(0)) > 1) {
> return true;
> }
> }
> return false;
> }public void close() {
> inputEnumerator.close();
> }public Object current() {
> final org.apache.calcite.runtime.FlatLists.ComparableList
> current = (org.apache.calcite.runtime.FlatLists.ComparableList)
> inputEnumerator.current();
> return new Object[] {
> current.get(0),
> current.get(1)};
> } };
> }};
> }
> public Class getElementType() {
> return java.lang.Object[].class;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3498) Unnest operation's ordinality should be deterministic
[
https://issues.apache.org/jira/browse/CALCITE-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974918#comment-16974918
]
Ruben Q L commented on CALCITE-3498:
Fixed via
[https://github.com/apache/calcite/commit/17d0684445b5ce4224e09728d4fd89cf41004d19]
Thanks for the PR [~donnyzone] !
> Unnest operation's ordinality should be deterministic
> -
>
> Key: CALCITE-3498
> URL: https://issues.apache.org/jira/browse/CALCITE-3498
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.21.0
> Environment: **_**_**
>Reporter: Feng Zhu
>Assignee: Feng Zhu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Current now, the compution of unnest operation's ordinality is conducted in
> *_current()_* method (
> [https://github.com/apache/calcite/blob/3853118b4d1d48f7d3970b5488baf7ca78d5028e/core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java#L2764).]
> Consequently, when the method is called, the ordinality will be incremented,
> leading to incorrect and un-deterministic result.
> We can use a simple query for illustration.
> {code:java}
> Query:
> ==
> select v, o
> from unnest(array[100,200]) with ordinality as t1(v, o)
> where v > 1
> Expected Result
> ==
> V=100; O=1
> V=200; O=2
> {code}
> However, we get the follow incorrect result.
> {code:java}
> V=100; O=2
> V=200; O=4
> {code}
> We can infer to the code generated. It can be seen that
> *inputEnumerator.current()* has been called two times, one for filter (v>1)
> and one for select (v, o).
> {code:java}
> public org.apache.calcite.linq4j.Enumerable bind(final
> org.apache.calcite.DataContext root) {
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable = ..
> final org.apache.calcite.linq4j.Enumerable _inputEnumerable0 =
> child.selectMany(org.apache.calcite.runtime.SqlFunctions.flatProduct(new
> int[] {
> -1}, true, new
> org.apache.calcite.runtime.SqlFunctions.FlatProductInputType[] {
> org.apache.calcite.runtime.SqlFunctions.FlatProductInputType.SCALAR}));
> return new org.apache.calcite.linq4j.AbstractEnumerable(){
> public org.apache.calcite.linq4j.Enumerator enumerator() {
> return new org.apache.calcite.linq4j.Enumerator(){
> public final org.apache.calcite.linq4j.Enumerator inputEnumerator
> = _inputEnumerable0.enumerator();
> public void reset() {
> inputEnumerator.reset();
> }public boolean moveNext() {
> while (inputEnumerator.moveNext()) {
> if
> (org.apache.calcite.runtime.SqlFunctions.toInt(((org.apache.calcite.runtime.FlatLists.ComparableList)
> inputEnumerator.current()).get(0)) > 1) {
> return true;
> }
> }
> return false;
> }public void close() {
> inputEnumerator.close();
> }public Object current() {
> final org.apache.calcite.runtime.FlatLists.ComparableList
> current = (org.apache.calcite.runtime.FlatLists.ComparableList)
> inputEnumerator.current();
> return new Object[] {
> current.get(0),
> current.get(1)};
> } };
> }};
> }
> public Class getElementType() {
> return java.lang.Object[].class;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
