[
https://issues.apache.org/jira/browse/CALCITE-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294186#comment-14294186
]
Vladimir Sitnikov edited comment on CALCITE-569 at 1/27/15 9:06 PM:
--------------------------------------------------------------------
{quote} to ban collations on logical RelNodes{quote}
I do not suggest to ban collations on logical RelNodes.
My point is as follows:
1) Current deducer of project collations is broken
2) Thanks to that broken logic it almost never identified a "non-required sort"
3) Users got correct results
If we heal collation deducer, we might unexpectedly get _wrong data_.
For instance, consider JdbcCalcRule. It blindly adds "select+where" and that is
it.
Does it ensure collation trait via order by? No. It can result in arbitrary
shuffle by the database engine.
The same goes for Enumerable.
Do you really mean to fix all the Calc/Project/etc converters?
It would be a non-trivial patch
That is why I suggest the following:
1) Heal collation deducer (so method works as expected, without
ArrayIndexOut...)
2) Skip collation deducer when producing logical relations.
3) We can preserve collation for well-known operations (e.g. EnumerableProject,
etc).
#2 will make Calcite a bit "dumb" (it might preform non-required sorts),
however it would result in _correct_ result. I do not like "wrong results" kind
of issues from the database engines.
Later we could revisit the approach and make Calcite smarter in terms of
collation preserving and collation non-preserving Projects.
was (Author: vladimirsitnikov):
{quote} to ban collations on logical RelNodes{quote}
I do not suggest to ban collations on logical RelNodes.
My point is as follows:
1) Current deducer of project collations is broken
2) Thanks to that broken logic it almost never identified a "non-required sort"
3) Users got correct results
If we heal collation deducer, we might unexpectedly get _wrong data_.
For instance, consider JdbcCalcRule. It blindly adds "select+where" and that is
it.
Does it ensure collation trait via order by?
Same goes for Enumerable.
Do you really mean to fix all the Calc/Project/etc converters?
It would be a non-trivial patch
That is why I suggest the following:
1) Heal collation deducer (so method works as expected, without
ArrayIndexOut...)
2) Skip collation deducer when producing logical relations.
#2 will make Calcite a bit "dumb" (it might preform non-required sorts),
however it would result in _correct_ result.
Later we could revisit the approach and make Calcite more smart in terms of
collation preserving and collation non-preserving Projects.
> ArrayIndexOutOfBoundsException when deducing collation
> ------------------------------------------------------
>
> Key: CALCITE-569
> URL: https://issues.apache.org/jira/browse/CALCITE-569
> Project: Calcite
> Issue Type: Bug
> Affects Versions: 1.0.0-incubating
> Reporter: Aman Sinha
> Assignee: Julian Hyde
> Attachments:
> 0001-CALCITE-569-Create-a-Project-with-empty-collation-if.patch,
> 0001-Properly-track-collation-trait-for-select-a-from-.-o.patch,
> 0001-Properly-track-collation-trait-for-select-a-from-.-o.patch,
> PlanTest.java.diff
>
>
> If a subquery has an ORDER BY on a column that is not in the SELECT list and
> the outer query does another ORDER BY, Calcite encounters an
> ArrayIndexOutOfBoundException when deducing collation.
> In PlannerTest, I created a simple test by first adding the following traits:
> {code}
> List<RelTraitDef> traitDefs = new ArrayList<RelTraitDef>();
> traitDefs.add(ConventionTraitDef.INSTANCE);
> traitDefs.add(RelCollationTraitDef.INSTANCE);
> {code}
> And ran the following query:
> {code}
> select t.psPartkey from (select ps.psPartkey from `tpch`.`partsupp` ps order
> by ps.psSupplyCost) t order by t.psPartkey"
> {code}
> {code}
> java.lang.ArrayIndexOutOfBoundsException: -1
> at
> org.apache.calcite.rex.RexProgram.deduceCollations(RexProgram.java:589)
> at org.apache.calcite.rex.RexProgram.getCollations(RexProgram.java:558)
> at
> org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2685)
> at
> org.apache.calcite.plan.RelOptUtil.createProject(RelOptUtil.java:2623)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3571)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:613)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:568)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2929)
> at
> org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:526)
> at org.apache.calcite.prepare.PlannerImpl.convert(PlannerImpl.java:189)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
