[
https://issues.apache.org/jira/browse/CALCITE-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maryann Xue updated CALCITE-1018:
---------------------------------
Attachment: CALCITE-1018.repro.diff
I have finally come up with a test case in JdbcTest that easily reproduces this
issue and demonstrates the few other things I have stated above. Apply
CALCITE-1019.patch together with this attached file, you'll see:
1) The rule doesn't fire.
2) After adding CALCITE-1018.01.patch, the test will fail due to infinite loop.
3) Apply the temporary fix, the test case will pass. Again, I am not suggesting
using this temporary fix. It rather just serves as a demonstration of the
problem.
This file also removes the original test case for SortJoinTransposeRule, the
comment for which was wrong by saying that the rule did not take effect because
"Limit and sort are different Enumerable operators, thus it does not apply".
The old test case couldn't see the effect of SortJoinTransposeRule because
EnumerableLimit didn't have a rowCount that reflects "limit", and even if it
did the extra sorting cost of emp table might be bigger than the gain the limit
could bring.
> SortJoinTransposeRule not firing due to getMaxRowCount(RelSubset) returning
> null
> --------------------------------------------------------------------------------
>
> Key: CALCITE-1018
> URL: https://issues.apache.org/jira/browse/CALCITE-1018
> Project: Calcite
> Issue Type: Bug
> Reporter: Maryann Xue
> Attachments: CALCITE-1018.01.patch, CALCITE-1018.repro.diff
>
>
> getMaxRowCount(RelSubset) defaults to getMaxRowCount(RelNode) and returns
> null, so if the Join rel has its outer child rel as a RelSubset, the
> SortJoinTransposeRule will fail the checkInputForCollationAndLimit check and
> will not proceed to push down a limit through join.
> Before fix of CALCITE-995 and CALCITE-987, getMaxRowCount(RelSubset) would
> return positive infinity, which would cause similar situation but an opposite
> effect as firing the rule infinitely when the join's outer child is a
> RelSubset.
> Neither of the above situation was reflected in the test cases for
> SortJoinTransposeRule in Calcite, since the RelSubset condition was not
> covered by RelOptRule test which uses HepPlanner running with just one or no
> more than a couple of rules together.
> Basically we need a more accurate way to get max row count for RelSubset
> (similarly for HexVertex as well), otherwise checkInputForCollationAndLimit
> would either always fail or always succeed for a Limit over a Join over a
> RelSubset. But I assume, to be real accurate, we'd have to introduce a
> similar mechanism to one that computes bestCost in RelSubset, which I doubt
> would be worth it.
> Another way, which might seem a little ugly, is to add something like
> "isSortPushedThrough()" in Sort rel, similar to "isSemiJoinDone()" in Join,
> in order to avoid the rule being fired infinitely.
> In my Phoenix project, I applied a temporary fix, but it proved to work:
> {code}
> public class PhoenixRelMdMaxRowCount {
> public static final RelMetadataProvider SOURCE =
> ReflectiveRelMetadataProvider.reflectiveSource(
> BuiltInMethod.MAX_ROW_COUNT.method, new
> PhoenixRelMdMaxRowCount());
> private PhoenixRelMdMaxRowCount() {
> }
> public Double getMaxRowCount(RelSubset rel) {
> for (RelNode node : rel.getRels()) {
> if (node instanceof Sort) {
> Sort sort = (Sort) node;
> if (sort.fetch != null) {
> return (double) RexLiteral.intValue(sort.fetch);
> }
> }
> }
>
> return Double.POSITIVE_INFINITY;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
