[
https://issues.apache.org/jira/browse/CALCITE-3997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108920#comment-17108920
]
Ruben Q L commented on CALCITE-3997:
------------------------------------
[~FatLittle], usually this is the case, and I am also using already
ProjectMergeRule; but for this specific scenario it does not work (obviously,
the initial plan is a bit more complex than what I have described, with
operators in the middle of both LogicalProjects, and several optimizations /
transformations leading to the final result of two EnumerableCalc together).
In any case, I think the key point here is not discussing the peculiarities of
my plan, but the regression introduced by this patch (which IMHO is more
important than just my example). Previously, there were some rules (e.g.
CalcMergeRule) that were fired in Volcano planner using the "base" operator
(e.g. Calc), and now they will only fire on the logical operator (LogicalCal).
I just pointed out that regression, and we have a quick fix. However, the main
issue remains, what happens to those downstream projects that require rules to
be fired on the "base" operators or, like in my specific example, in the
enumerable operators? What is the approach now? If we look at the definition of
some rules (like CalcMergeRule or ProjectMergeRule), they should work in that
case (because they are defined to be fired on the "base" operator), however
with the introduction of the TransformationRule interface, they will not (in
Volcano planner). This makes these rule definitions somehow misleading. So,
coming back to my question, what happens to those projects which require
CalcMergeRule to work at enumerable operators level? Do they need to copy-paste
CalcMergeRule code and definer their own EnumerableCalcMergeRule (which does
not implement TransformationRule)? Because this is the only workaround that I
can see so far (maybe I am missing something); it is possible but not ideal in
terms of code re-usability.
I am no saying this patch is wrong, I am just saying this is a breaking change
that can (like in my example) break real scenarios. Of course we can break
things; if it is justified and documented with the new release. And maybe we
come to the conclusion that what is wrong is not the direction of this patch,
but my use case in particular, and I'd need to find and apply a workaround for
my specific case (like the one described above). That is fine, I just wanted to
open the discussion about this regression and see what conclusion we reach all
together :)
> Problem with MERGE JOIN: java.lang.AssertionError: cannot merge join: left
> input is not sorted on left keys
> -----------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-3997
> URL: https://issues.apache.org/jira/browse/CALCITE-3997
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.23.0
> Reporter: Enrico Olivelli
> Priority: Blocker
> Fix For: 1.23.0
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> I have a couple of problems with HerdDB.
> 1) JOIN order unsorted columns in presence of a WHERE over other columns
> This is my case:
> CREATE TABLE tblspace1.table1 (k1 string primary key,n1 int,s1 string)
> CREATE TABLE tblspace1.table3 (k1 string primary key,n3 int,s3 string)
> SELECT t1.k1 as first, t2.k1 as second
> FROM tblspace1.table1 t1
> INNER JOIN tblspace1.table3 t2 ON t1.k1=t2.k1
> WHERE t1.n1 + 1 = t2.n3
> In this case for table1 and table3 no column is physically sorted (no column
> with a collation)
> I have this Planner error:
> java.lang.AssertionError: cannot merge join: left input is not sorted on left
> keys
> at
> org.apache.calcite.rel.metadata.RelMdCollation.mergeJoin(RelMdCollation.java:457)
> at
> org.apache.calcite.rel.metadata.RelMdCollation.collations(RelMdCollation.java:153)
> at GeneratedMetadataHandler_Collation.collations_$(Unknown Source)
> at GeneratedMetadataHandler_Collation.collations(Unknown Source)
> at
> org.apache.calcite.rel.metadata.RelMetadataQuery.collations(RelMetadataQuery.java:539)
> at
> org.apache.calcite.rel.metadata.RelMdCollation.project(RelMdCollation.java:273)
> at
> org.apache.calcite.rel.logical.LogicalProject.lambda$create$0(LogicalProject.java:122)
> at org.apache.calcite.plan.RelTraitSet.replaceIfs(RelTraitSet.java:242)
> at
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:121)
> at
> org.apache.calcite.rel.logical.LogicalProject.create(LogicalProject.java:111)
> at
> org.apache.calcite.rel.core.RelFactories$ProjectFactoryImpl.createProject(RelFactories.java:172)
> at org.apache.calcite.tools.RelBuilder.project_(RelBuilder.java:1464)
> at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1258)
> at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1230)
> at org.apache.calcite.tools.RelBuilder.project(RelBuilder.java:1219)
> at
> org.apache.calcite.plan.RelOptUtil.pushDownJoinConditions(RelOptUtil.java:3620)
> at
> org.apache.calcite.rel.rules.JoinPushExpressionsRule.onMatch(JoinPushExpressionsRule.java:59)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:221)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:519)
> at herddb.sql.CalcitePlanner.runPlanner(CalcitePlanner.java:535)
> at herddb.sql.CalcitePlanner.translate(CalcitePlanner.java:292)
> If I remove the "WHERE" clause then no error is reported.
> we have many other test cases about JOINs and this one is the only one that
> fails
> This is the failing test case on HerdDB
> https://github.com/diennea/herddb/blob/vote-calcite-123/herddb-core/src/test/java/herddb/core/SimpleJoinTest.java#L522
> We are using the default set of rules Programs.ofRules(Programs.RULE_SET)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
