[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Orlov updated IGNITE-20164:
--
Ignite Flags: (was: Docs Required,Release Notes Required)
> Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce
> aggregates.
> --
>
> Key: IGNITE-20164
> URL: https://issues.apache.org/jira/browse/IGNITE-20164
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Maksim Zhuravkov
>Assignee: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> RelCollation propagation does not take into account remapping of group keys
> between MAP/REDUCE phases, hence causes errors in queries that are expected
> to use sort-based MAP/REDUCE - RelCollation uses the same keys on both
> phases. Example:
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]),
> IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]),
> IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), < HERE
> IgniteExchange(distribution=[single]),
> IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]),
> IgniteSort(sort0=[$1], dir0=[ASC]),
> IgniteTableScan(table=[[PUBLIC, TESTME40]],
> requiredColumns=[{1, 2}]),
> {code}
> Error:
> {code:java}
> Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
> cast to class java.lang.Comparable (java.util.ArrayList and
> java.lang.Comparable are in module java.base of loader 'bootstrap')
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
> at
> java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
> at
> java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
> at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
> at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
> at
> org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
> ... 7 more
> {code}
> The query below works because position of column b does not change after MAP
> phase.
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]),
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]),
> IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]),
> IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]),
> IgniteExchange(distribution=[single]),
> IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]),
> IgniteSort(sort0=[$0], dir0=[ASC]),
> IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
> $t0]], requiredColumns=[{1, 2}]),
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Description:
RelCollation propagation does not take into account remapping of group keys
between MAP/REDUCE phases, hence causes errors in queries that are expected to
use sort-based MAP/REDUCE - RelCollation uses the same keys on both phases.
Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), < HERE
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
IgniteSort(sort0=[$1], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], requiredColumns=[{1,
2}]),
{code}
Error:
{code:java}
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
cast to class java.lang.Comparable (java.util.ArrayList and
java.lang.Comparable are in module java.base of loader 'bootstrap')
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
at
java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
at
java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
at
org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
... 7 more
{code}
The query below works because position of column b does not change after MAP
phase.
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteSort(sort0=[$0], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
$t0]], requiredColumns=[{1, 2}]),
{code}
was:
RelCollation propagation does not take into account remapping of group keys
between MAP/REDUCE phases, hence causes errors in queries that are expected to
use sort-based MAP/REDUCE - RelCollation uses the same keys on both phases.
Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Description:
RelCollation propagation does not take into account remapping of group keys
between MAP/REDUCE phases, hence causes errors in queries that are expected to
use sort-based MAP/REDUCE - RelCollation uses the same keys on both phases.
Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
IgniteSort(sort0=[$1], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], requiredColumns=[{1,
2}]),
{code}
Error:
{code:java}
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
cast to class java.lang.Comparable (java.util.ArrayList and
java.lang.Comparable are in module java.base of loader 'bootstrap')
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
at
java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
at
java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
at
org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
... 7 more
{code}
The query below works because position of column b does not change after MAP
phase.
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteSort(sort0=[$0], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
$t0]], requiredColumns=[{1, 2}]),
{code}
was:
RelCollation propagation does not take into account remapping of group keys
between MAP/REDUCE phases, hence causes errors in queries that are expected to
use sort-based MAP/REDUCE (RelCollation uses incorrect fields). Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{1}],
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Description:
RelCollation propagation does not take into account remapping of group keys
between MAP/REDUCE phases, hence causes errors in queries that are expected to
use sort-based MAP/REDUCE (RelCollation uses incorrect fields). Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
IgniteSort(sort0=[$1], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], requiredColumns=[{1,
2}]),
{code}
Error:
{code:java}
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
cast to class java.lang.Comparable (java.util.ArrayList and
java.lang.Comparable are in module java.base of loader 'bootstrap')
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
at
java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
at
java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
at
org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
... 7 more
{code}
The query below works because position of column b does not change after MAP
phase.
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteSort(sort0=[$0], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
$t0]], requiredColumns=[{1, 2}]),
{code}
was:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Summary: Sql. Incorrect propagation of RelCollation trait for Sort-based
map/reduce aggregates. (was: Sql. Incorrect propagation of RelCollation trait
for Sort-based Map/reduce aggregates.)
> Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce
> aggregates.
> --
>
> Key: IGNITE-20164
> URL: https://issues.apache.org/jira/browse/IGNITE-20164
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
>
> RelCollation propagation does not take remapping for group keys between
> MAP/REDUCE phases into account, hence causes errors in queries that are
> expected to use sort-based MAP/REDUCE. Example:
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]),
> IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]),
> IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]),
> IgniteExchange(distribution=[single]),
> IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]),
> IgniteSort(sort0=[$1], dir0=[ASC]),
> IgniteTableScan(table=[[PUBLIC, TESTME40]],
> requiredColumns=[{1, 2}]),
> {code}
> Error:
> {code:java}
> Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
> cast to class java.lang.Comparable (java.util.ArrayList and
> java.lang.Comparable are in module java.base of loader 'bootstrap')
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
> at
> java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
> at
> java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
> at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
> at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
> at
> org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
> ... 7 more
> {code}
> The query below works because position of column b does not change after MAP
> phase.
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]),
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]),
> IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]),
> IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]),
> IgniteExchange(distribution=[single]),
> IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]),
> IgniteSort(sort0=[$0], dir0=[ASC]),
> IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
> $t0]], requiredColumns=[{1, 2}]),
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Description:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]),
IgniteSort(sort0=[$1], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], requiredColumns=[{1,
2}]),
{code}
Error:
{code:java}
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
cast to class java.lang.Comparable (java.util.ArrayList and
java.lang.Comparable are in module java.base of loader 'bootstrap')
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
at
java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
at
java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
at
org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
... 7 more
{code}
The query below works because position of column b does not change after MAP
phase.
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]),
IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]),
IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteExchange(distribution=[single]),
IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]),
IgniteSort(sort0=[$0], dir0=[ASC]),
IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
$t0]], requiredColumns=[{1, 2}]),
{code}
was:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 209
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 208
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), id = 207
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Description:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
{code:java}
// Some comments here
public String getFoo()
{
return foo;
}
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 209
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 208
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), id = 207
IgniteExchange(distribution=[single]), id = 206
IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), id = 205
IgniteSort(sort0=[$1], dir0=[ASC]), id = 204
IgniteTableScan(table=[[PUBLIC, TESTME40]], requiredColumns=[{1,
2}]), id = 194
{code}
Error:
{code:java}
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
cast to class java.lang.Comparable (java.util.ArrayList and
java.lang.Comparable are in module java.base of loader 'bootstrap')
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
at
java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
at
java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
at
org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
... 7 more
{code}
The query below works because position of column b does not change after MAP
phase.
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]), id = 350
IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]), id = 349
IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]), id = 348
IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]), id = 347
IgniteExchange(distribution=[single]), id = 346
IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]), id = 345
IgniteSort(sort0=[$0], dir0=[ASC]), id = 344
IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
$t0]], requiredColumns=[{1, 2}]), id = 334
{code}
was:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases, hence brakes queries that are expected to use sort-based
MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Description:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 209
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 208
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), id = 207
IgniteExchange(distribution=[single]), id = 206
IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), id = 205
IgniteSort(sort0=[$1], dir0=[ASC]), id = 204
IgniteTableScan(table=[[PUBLIC, TESTME40]], requiredColumns=[{1,
2}]), id = 194
{code}
Error:
{code:java}
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
cast to class java.lang.Comparable (java.util.ArrayList and
java.lang.Comparable are in module java.base of loader 'bootstrap')
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
at
java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
at
java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
at
org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
... 7 more
{code}
The query below works because position of column b does not change after MAP
phase.
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]), id = 350
IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]), id = 349
IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]), id = 348
IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]), id = 347
IgniteExchange(distribution=[single]), id = 346
IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]), id = 345
IgniteSort(sort0=[$0], dir0=[ASC]), id = 344
IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
$t0]], requiredColumns=[{1, 2}]), id = 334
{code}
was:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 209
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Description:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 209
IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)], EXPR$1=[COUNT($0)],
collation=[[]]), id = 208
IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), id = 207
IgniteExchange(distribution=[single]), id = 206
IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
collation=[[1]]), id = 205
IgniteSort(sort0=[$1], dir0=[ASC]), id = 204
IgniteTableScan(table=[[PUBLIC, TESTME40]], requiredColumns=[{1,
2}]), id = 194
{code}
Error:
{code:java}
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
cast to class java.lang.Comparable (java.util.ArrayList and
java.lang.Comparable are in module java.base of loader 'bootstrap')
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
at
org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
at
java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
at
java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
at
org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
at
org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
... 7 more
{code}
The query below works because position of column b does not change after MAP
phase.
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
Plan:
{code:java}
IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]), id = 350
IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]), id = 349
IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)], EXPR$1=[$SUM0($1)],
collation=[[]]), id = 348
IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]), id = 347
IgniteExchange(distribution=[single]), id = 346
IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
collation=[[0]]), id = 345
IgniteSort(sort0=[$0], dir0=[ASC]), id = 344
IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
$t0]], requiredColumns=[{1, 2}]), id = 334
{code}
was:
RelCollation propagation does not take remapping for group keys between
MAP/REDUCE phases into account, hence causes errors in queries that are
expected to use sort-based MAP/REDUCE. Example:
{code:java}
String[] rules = {
"MapReduceHashAggregateConverterRule",
"ColocatedHashAggregateConverterRule",
"ColocatedSortAggregateConverterRule"
};
sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
.disableRules(rules)
.returns(3L, 2L)
.check();
{code}
{code:java}
// Some comments here
public String getFoo()
{
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Labels: ignite-3 (was: )
> Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce
> aggregates.
> --
>
> Key: IGNITE-20164
> URL: https://issues.apache.org/jira/browse/IGNITE-20164
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
>
> RelCollation propagation does not take remapping for group keys between
> MAP/REDUCE phases, hence brakes queries that are expected to use sort-based
> MAP/REDUCE. Example:
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> {code:java}
> // Some comments here
> public String getFoo()
> {
> return foo;
> }
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]), id = 209
> IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]), id = 208
> IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), id = 207
> IgniteExchange(distribution=[single]), id = 206
> IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), id = 205
> IgniteSort(sort0=[$1], dir0=[ASC]), id = 204
> IgniteTableScan(table=[[PUBLIC, TESTME40]],
> requiredColumns=[{1, 2}]), id = 194
> {code}
> Error:
> {code:java}
> Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
> cast to class java.lang.Comparable (java.util.ArrayList and
> java.lang.Comparable are in module java.base of loader 'bootstrap')
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
> at
> java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
> at
> java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
> at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
> at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
> at
> org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
> ... 7 more
> {code}
> The query below works because position of column b does not change after MAP
> phase.
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]), id = 350
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]), id = 349
> IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]), id = 348
> IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]), id = 347
> IgniteExchange(distribution=[single]), id = 346
> IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]), id = 345
> IgniteSort(sort0=[$0], dir0=[ASC]), id = 344
> IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
> $t0]], requiredColumns=[{1, 2}]), id = 334
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Affects Version/s: 3.0.0-beta1
> Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce
> aggregates.
> --
>
> Key: IGNITE-20164
> URL: https://issues.apache.org/jira/browse/IGNITE-20164
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
>
> RelCollation propagation does not take remapping for group keys between
> MAP/REDUCE phases, hence brakes queries that are expected to use sort-based
> MAP/REDUCE. Example:
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> {code:java}
> // Some comments here
> public String getFoo()
> {
> return foo;
> }
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]), id = 209
> IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]), id = 208
> IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), id = 207
> IgniteExchange(distribution=[single]), id = 206
> IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), id = 205
> IgniteSort(sort0=[$1], dir0=[ASC]), id = 204
> IgniteTableScan(table=[[PUBLIC, TESTME40]],
> requiredColumns=[{1, 2}]), id = 194
> {code}
> Error:
> {code:java}
> Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
> cast to class java.lang.Comparable (java.util.ArrayList and
> java.lang.Comparable are in module java.base of loader 'bootstrap')
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
> at
> java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
> at
> java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
> at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
> at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
> at
> org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
> ... 7 more
> {code}
> The query below works because position of column b does not change after MAP
> phase.
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]), id = 350
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]), id = 349
> IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]), id = 348
> IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]), id = 347
> IgniteExchange(distribution=[single]), id = 346
> IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]), id = 345
> IgniteSort(sort0=[$0], dir0=[ASC]), id = 344
> IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
> $t0]], requiredColumns=[{1, 2}]), id = 334
> {code}
--
This message was sent by Atlassian Jira
[jira] [Updated] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-20164:
--
Component/s: sql
> Sql. Incorrect propagation of RelCollation trait for Sort-based Map/reduce
> aggregates.
> --
>
> Key: IGNITE-20164
> URL: https://issues.apache.org/jira/browse/IGNITE-20164
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Major
>
> RelCollation propagation does not take remapping for group keys between
> MAP/REDUCE phases, hence brakes queries that are expected to use sort-based
> MAP/REDUCE. Example:
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> {code:java}
> // Some comments here
> public String getFoo()
> {
> return foo;
> }
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]), id = 210
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]), id = 209
> IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]), id = 208
> IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), id = 207
> IgniteExchange(distribution=[single]), id = 206
> IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), id = 205
> IgniteSort(sort0=[$1], dir0=[ASC]), id = 204
> IgniteTableScan(table=[[PUBLIC, TESTME40]],
> requiredColumns=[{1, 2}]), id = 194
> {code}
> Error:
> {code:java}
> Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
> cast to class java.lang.Comparable (java.util.ArrayList and
> java.lang.Comparable are in module java.base of loader 'bootstrap')
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
> at
> java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
> at
> java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
> at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
> at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
> at
> org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
> ... 7 more
> {code}
> The query below works because position of column b does not change after MAP
> phase.
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]), id = 350
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]), id = 349
> IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]), id = 348
> IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]), id = 347
> IgniteExchange(distribution=[single]), id = 346
> IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]), id = 345
> IgniteSort(sort0=[$0], dir0=[ASC]), id = 344
> IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
> $t0]], requiredColumns=[{1, 2}]), id = 334
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
