alex-plekhanov commented on code in PR #13389:
URL: https://github.com/apache/ignite/pull/13389#discussion_r4324587626
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdColumnOrigins.java:
##########
@@ -303,6 +322,38 @@ private <T> void getOperands(RexNode rn, Class<T> cls,
Set<T> res) {
}
}
+ /**
+ * Provides column origins for a subset.
+ *
+ * <p>Origins are resolved through the original (logical) expression of
the set rather than through the current best
+ * plan: the best expression changes during optimization, and metadata
depending on it makes row count estimates
Review Comment:
> metadata depending on it makes row count estimates of the same expression
unstable, which in turn leads to inconsistent costs and cyclic best plans in
Volcano memo
Column origin (unlike row count estimation) should be calculated identically
for all plans.
Here we do not calculate row count. At least comment is incorrect.
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdColumnOrigins.java:
##########
@@ -281,7 +288,19 @@ public class IgniteMdColumnOrigins implements
MetadataHandler<BuiltInMetadata.Co
return res;
}
- return
Collections.singleton(rel.columnOriginsByRelLocalRef(iOutputColumn));
Review Comment:
What the difference with the current implementation?
Let's keep it as is, but replace `requiredColumns().toArray()[colIdx]` with
`requiredColumns().indexOf(colIdx)` in `columnOriginsByRelLocalRef` for little
optimization.
##########
modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/planner/JoinRowCountEstimationTest.java:
##########
@@ -0,0 +1,347 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.query.calcite.planner;
+
+import java.math.BigDecimal;
+import java.util.function.Predicate;
+import java.util.regex.Pattern;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.sql.SqlExplainFormat;
+import org.apache.calcite.sql.SqlExplainLevel;
+import org.apache.calcite.util.ImmutableIntList;
+import
org.apache.ignite.internal.processors.query.calcite.rel.IgniteCorrelatedNestedLoopJoin;
+import org.apache.ignite.internal.processors.query.calcite.schema.IgniteSchema;
+import
org.apache.ignite.internal.processors.query.calcite.trait.IgniteDistributions;
+import org.apache.ignite.internal.util.typedef.internal.CU;
+import org.hamcrest.BaseMatcher;
+import org.hamcrest.CoreMatchers;
+import org.hamcrest.Description;
+import org.hamcrest.Matcher;
+import org.junit.Test;
Review Comment:
Replace with JUnit 5 class
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdPercentageOriginalRows.java:
##########
Review Comment:
Change to:
```
public static final RelMetadataProvider SOURCE =
ReflectiveRelMetadataProvider.reflectiveSource(
new IgniteMdPercentageOriginalRows(),
BuiltInMetadata.PercentageOriginalRows.Handler.class);
```
as for other Metadata?
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -45,14 +56,24 @@
/** */
@SuppressWarnings("unused") // actually all methods are used by runtime
generated classes
public class IgniteMdRowCount extends RelMdRowCount {
+ /** */
+ private static final double NON_EQUI_COEFF = 0.7;
+
+ /** */
+ public static final double EQUI_COEFF = 0.8;
+
/** */
public static final RelMetadataProvider SOURCE =
- ReflectiveRelMetadataProvider.reflectiveSource(
- BuiltInMethod.ROW_COUNT.method, new IgniteMdRowCount());
+ ReflectiveRelMetadataProvider.reflectiveSource(new IgniteMdRowCount(),
BuiltInMetadata.RowCount.Handler.class);
+
+ /** */
+ public Double getRowCount(IgniteCorrelatedNestedLoopJoin rel,
RelMetadataQuery mq) {
+ return rel.estimateRowCount(mq);
+ }
/** {@inheritDoc} */
@Override public Double getRowCount(Join rel, RelMetadataQuery mq) {
- return rel.estimateRowCount(mq);
Review Comment:
Why not leave `rel.estimateRowCount(mq);`? For all nodes except CNLJ it
calls `joinRowCount(mq, rel);`, for CNLJ it calls `correlatedJoinRowCount`, so
looks like `rel.estimateRowCount(mq);`:
- makes code more predictable (joinRowCount and correlatedJoinRowCount
called only from one place in code).
- makes method `getRowCount(IgniteCorrelatedNestedLoopJoin rel,
RelMetadataQuery mq)` redundant and code simple.
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -72,48 +116,272 @@ public class IgniteMdRowCount extends RelMdRowCount {
mq.getRowCount(rel.getLeft()));
}
+ JoinInfo joinInfo = rel.analyzeCondition();
+
+ if (joinInfo.pairs().isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
// Row count estimates of 0 will be rounded up to 1.
// So, use maxRowCount where the product is very small.
- final Double left = mq.getRowCount(rel.getLeft());
- final Double right = mq.getRowCount(rel.getRight());
+ final Double leftRowCnt = mq.getRowCount(rel.getLeft());
+ final Double rightRowCnt = multiply(mq.getRowCount(rel.getRight()),
rightAdjust);
- if (left == null || right == null)
+ if (leftRowCnt == null || rightRowCnt == null)
return null;
- if (left <= 1D || right <= 1D) {
+ if (leftRowCnt <= 1D || rightRowCnt <= 1D) {
Double max = mq.getMaxRowCount(rel);
if (max != null && max <= 1D)
return max;
}
- JoinInfo joinInfo = rel.analyzeCondition();
+ Map<Integer, KeyColumnOrigin> columnsFromLeft = resolveOrigins(mq,
rel.getLeft(), joinInfo.leftKeys);
+ Map<Integer, KeyColumnOrigin> columnsFromRight = resolveOrigins(mq,
rel.getRight(), joinInfo.rightKeys);
+
+ if (columnsFromLeft.isEmpty() || columnsFromRight.isEmpty()) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ Map<TablesPair, JoinContext> joinCtxts = new HashMap<>();
+ for (IntPair joinKeys : joinInfo.pairs()) {
+ KeyColumnOrigin leftKey = columnsFromLeft.get(joinKeys.source);
+ KeyColumnOrigin rightKey = columnsFromRight.get(joinKeys.target);
+ if (leftKey == null || rightKey == null) {
+ continue;
+ }
+
+ joinCtxts.computeIfAbsent(
+ new TablesPair(
+ leftKey.origin.getOriginTable(),
+ rightKey.origin.getOriginTable()
+ ),
+ key -> {
+ IgniteTable leftTable = key.left.unwrap(IgniteTable.class);
+ IgniteTable rightTable =
key.right.unwrap(IgniteTable.class);
+
+ assert leftTable != null && rightTable != null;
+
+ int leftPkSize = leftTable.distribution().getKeys().size();
+ int rightPkSize =
rightTable.distribution().getKeys().size();
+
+ return new JoinContext(leftPkSize, rightPkSize);
+ }
+ ).countKeys(leftKey, rightKey);
+ }
+
+ if (joinCtxts.isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
+ Iterator<JoinContext> it = joinCtxts.values().iterator();
+ JoinContext joinCtx = it.next();
+ while (it.hasNext()) {
+ JoinContext nextCtx = it.next();
+ if (nextCtx.joinType().strength > joinCtx.joinType().strength) {
+ joinCtx = nextCtx;
+ }
+
+ if (joinCtx.joinType().strength ==
JoiningRelationType.PK_ON_PK.strength) {
+ break;
+ }
+ }
+
+ if (joinCtx.joinType() == JoiningRelationType.UNKNOWN) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ double postFiltrationAdjustment = 1.0;
+
+ switch (rel.getJoinType()) {
+ case INNER:
+ case SEMI:
+ // Extra join keys as well as non-equi conditions serves as
post-filtration,
+ // therefore we need to adjust final result with a little
factor.
+ if (joinCtxts.size() != 1 || !joinInfo.isEqui())
+ postFiltrationAdjustment = NON_EQUI_COEFF;
+
+ break;
+ default:
+ break;
+ }
+
+ double baseRowCnt = 0.0;
+ Double percentageAdjustment = null;
+ if (joinCtx.joinType() == JoiningRelationType.PK_ON_PK) {
+ postFiltrationAdjustment = 1.0;
+
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ postFiltrationAdjustment = EQUI_COEFF;
Review Comment:
Why do we redefine post filtration here? If we have two tables and first is
a subset of second, if no additional (non equi) conditions are defined, no
additional EQUI_COEF is required, row count will be firstTableRowCoun *
percentOfTotalRowsFromSecondTable (if data from second table are selected
randomly). If there are any non-equi conditions, row count should be reduced by
NON_EQUI_COEFF
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -72,48 +116,272 @@ public class IgniteMdRowCount extends RelMdRowCount {
mq.getRowCount(rel.getLeft()));
}
+ JoinInfo joinInfo = rel.analyzeCondition();
+
+ if (joinInfo.pairs().isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
// Row count estimates of 0 will be rounded up to 1.
// So, use maxRowCount where the product is very small.
- final Double left = mq.getRowCount(rel.getLeft());
- final Double right = mq.getRowCount(rel.getRight());
+ final Double leftRowCnt = mq.getRowCount(rel.getLeft());
+ final Double rightRowCnt = multiply(mq.getRowCount(rel.getRight()),
rightAdjust);
- if (left == null || right == null)
+ if (leftRowCnt == null || rightRowCnt == null)
return null;
- if (left <= 1D || right <= 1D) {
+ if (leftRowCnt <= 1D || rightRowCnt <= 1D) {
Double max = mq.getMaxRowCount(rel);
if (max != null && max <= 1D)
return max;
}
- JoinInfo joinInfo = rel.analyzeCondition();
+ Map<Integer, KeyColumnOrigin> columnsFromLeft = resolveOrigins(mq,
rel.getLeft(), joinInfo.leftKeys);
+ Map<Integer, KeyColumnOrigin> columnsFromRight = resolveOrigins(mq,
rel.getRight(), joinInfo.rightKeys);
+
+ if (columnsFromLeft.isEmpty() || columnsFromRight.isEmpty()) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ Map<TablesPair, JoinContext> joinCtxts = new HashMap<>();
+ for (IntPair joinKeys : joinInfo.pairs()) {
+ KeyColumnOrigin leftKey = columnsFromLeft.get(joinKeys.source);
+ KeyColumnOrigin rightKey = columnsFromRight.get(joinKeys.target);
+ if (leftKey == null || rightKey == null) {
+ continue;
+ }
+
+ joinCtxts.computeIfAbsent(
+ new TablesPair(
+ leftKey.origin.getOriginTable(),
+ rightKey.origin.getOriginTable()
+ ),
+ key -> {
+ IgniteTable leftTable = key.left.unwrap(IgniteTable.class);
+ IgniteTable rightTable =
key.right.unwrap(IgniteTable.class);
+
+ assert leftTable != null && rightTable != null;
+
+ int leftPkSize = leftTable.distribution().getKeys().size();
+ int rightPkSize =
rightTable.distribution().getKeys().size();
+
+ return new JoinContext(leftPkSize, rightPkSize);
+ }
+ ).countKeys(leftKey, rightKey);
+ }
+
+ if (joinCtxts.isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
+ Iterator<JoinContext> it = joinCtxts.values().iterator();
+ JoinContext joinCtx = it.next();
+ while (it.hasNext()) {
+ JoinContext nextCtx = it.next();
+ if (nextCtx.joinType().strength > joinCtx.joinType().strength) {
+ joinCtx = nextCtx;
+ }
+
+ if (joinCtx.joinType().strength ==
JoiningRelationType.PK_ON_PK.strength) {
+ break;
+ }
+ }
+
+ if (joinCtx.joinType() == JoiningRelationType.UNKNOWN) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ double postFiltrationAdjustment = 1.0;
+
+ switch (rel.getJoinType()) {
+ case INNER:
+ case SEMI:
+ // Extra join keys as well as non-equi conditions serves as
post-filtration,
+ // therefore we need to adjust final result with a little
factor.
+ if (joinCtxts.size() != 1 || !joinInfo.isEqui())
+ postFiltrationAdjustment = NON_EQUI_COEFF;
+
+ break;
+ default:
+ break;
+ }
+
+ double baseRowCnt = 0.0;
+ Double percentageAdjustment = null;
+ if (joinCtx.joinType() == JoiningRelationType.PK_ON_PK) {
+ postFiltrationAdjustment = 1.0;
Review Comment:
Redefining here is redundant, it always 1 except for INNER and SEMI join,
but for INNER and SEMI join it redefined again below (but shouldn't be).
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/CalciteQueryProcessor.java:
##########
@@ -145,8 +144,6 @@ public class CalciteQueryProcessor extends
GridProcessorAdapter implements Query
// See Apache Calcite ticket for more information, assertion is
incorrect.
// FRAMEWORK_CONFIG initialize SqlToRelConverter class. Assertions
should be disabled before class initialization.
SqlToRelConverter.class.getClassLoader().setClassAssertionStatus(SqlToRelConverter.class.getName(),
false);
- // TODO Workaround for
https://issues.apache.org/jira/browse/CALCITE-5421 and
https://issues.apache.org/jira/browse/CALCITE-7034
Review Comment:
Looks like we can remove previous workaraund too
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -72,48 +92,261 @@ public class IgniteMdRowCount extends RelMdRowCount {
mq.getRowCount(rel.getLeft()));
}
+ JoinInfo joinInfo = rel.analyzeCondition();
+
+ if (joinInfo.pairs().isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
// Row count estimates of 0 will be rounded up to 1.
// So, use maxRowCount where the product is very small.
- final Double left = mq.getRowCount(rel.getLeft());
- final Double right = mq.getRowCount(rel.getRight());
+ final Double leftRowCnt = mq.getRowCount(rel.getLeft());
+ final Double rightRowCnt = mq.getRowCount(rel.getRight());
- if (left == null || right == null)
+ if (leftRowCnt == null || rightRowCnt == null)
return null;
- if (left <= 1D || right <= 1D) {
+ if (leftRowCnt <= 1D || rightRowCnt <= 1D) {
Double max = mq.getMaxRowCount(rel);
if (max != null && max <= 1D)
return max;
}
- JoinInfo joinInfo = rel.analyzeCondition();
+ Map<Integer, KeyColumnOrigin> columnsFromLeft = resolveOrigins(mq,
rel.getLeft(), joinInfo.leftKeys);
+ Map<Integer, KeyColumnOrigin> columnsFromRight = resolveOrigins(mq,
rel.getRight(), joinInfo.rightKeys);
+
+ if (columnsFromLeft.isEmpty() || columnsFromRight.isEmpty())
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+
+ Map<TablesPair, JoinContext> joinCtxts = new HashMap<>();
+ for (IntPair joinKeys : joinInfo.pairs()) {
+ KeyColumnOrigin leftKey = columnsFromLeft.get(joinKeys.source);
+ KeyColumnOrigin rightKey = columnsFromRight.get(joinKeys.target);
+
+ if (leftKey == null || rightKey == null) {
+ continue;
+ }
+
+ joinCtxts.computeIfAbsent(
+ new TablesPair(
+ leftKey.origin.getOriginTable(),
+ rightKey.origin.getOriginTable()
+ ),
+ key -> {
+ IgniteTable leftTable = key.left.unwrap(IgniteTable.class);
+ IgniteTable rightTable =
key.right.unwrap(IgniteTable.class);
+
+ assert leftTable != null && rightTable != null;
+
+ int leftPkSize = leftTable.distribution().getKeys().size();
+ int rightPkSize =
rightTable.distribution().getKeys().size();
+
+ return new JoinContext(leftPkSize, rightPkSize);
+ }
+ ).countKeys(leftKey, rightKey);
+ }
+
+ if (joinCtxts.isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
+ Iterator<JoinContext> it = joinCtxts.values().iterator();
+ JoinContext joinCtx = it.next();
+ while (it.hasNext()) {
+ JoinContext nextCtx = it.next();
+ if (nextCtx.joinType().strength > joinCtx.joinType().strength) {
+ joinCtx = nextCtx;
+ }
+
+ if (joinCtx.joinType().strength ==
JoiningRelationType.PK_ON_PK.strength) {
+ break;
+ }
+ }
+
+ if (joinCtx.joinType() == JoiningRelationType.UNKNOWN) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
+ double postFiltrationAdjustment = 1.0;
+
+ switch (rel.getJoinType()) {
+ case INNER:
+ case SEMI:
+ // Extra join keys as well as non-equi conditions serves as
post-filtration,
+ // therefore we need to adjust final result with a little
factor.
+ if (joinCtxts.size() != 1 || !joinInfo.isEqui())
+ postFiltrationAdjustment = NON_EQUI_COEFF;
+
+ break;
+ default:
+ break;
+ }
+
+ double baseRowCnt = 0.0;
+ Double percentageAdjustment = null;
+ if (joinCtx.joinType() == JoiningRelationType.PK_ON_PK) {
+ postFiltrationAdjustment = EQUI_COEFF;
+
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ // Assume we have two fact tables SALES and RETURNS sharing
the same primary key. Every item
+ // can be sold, but only items which were sold can be returned
back, therefore
+ // size(SALES) > size(RETURNS). When joining SALES on RETURNS
by primary key, the estimated
+ // result size will be the same as the size of the smallest
table (RETURNS in this case),
+ // adjusted by the percentage of rows of the biggest table
(SALES in this case; percentage
+ // adjustment is required to account for predicates pushed
down to the table, e.g. we are
+ // interested in returns of items with certain category)
+ if (leftRowCnt > rightRowCnt) {
+ baseRowCnt = rightRowCnt;
+ percentageAdjustment =
mq.getPercentageOriginalRows(rel.getLeft());
+ }
+ else {
+ baseRowCnt = leftRowCnt;
+ percentageAdjustment =
mq.getPercentageOriginalRows(rel.getRight());
+ }
+ }
+ else if (rel.getJoinType() == JoinRelType.LEFT) {
+ baseRowCnt = leftRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.RIGHT) {
+ baseRowCnt = rightRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.FULL) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+
+ // Fall-back to calcite's implementation.
+ if (selectivity == null) {
+ return RelMdUtil.getJoinRowCount(mq, rel,
rel.getCondition());
+ }
+
+ baseRowCnt = rightRowCnt + leftRowCnt;
+ percentageAdjustment = 1.0 - selectivity;
+ }
+ }
+ else if (joinCtx.joinType() == JoiningRelationType.FK_ON_PK) {
+ // For foreign key joins the base table is the one which is joined
by non-primary key columns.
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ baseRowCnt = leftRowCnt;
+ // Need to adopt: IGNITE-23969, after it become possible to
change into: mq.getPercentageOriginalRows(rel.getRight());
+ percentageAdjustment = FK_WITH_PK_COEFF;
+ }
+ else if (rel.getJoinType() == JoinRelType.LEFT ||
rel.getJoinType() == JoinRelType.RIGHT) {
+ baseRowCnt = leftRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.FULL) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+
+ // Fall-back to calcite's implementation.
+ if (selectivity == null) {
+ return RelMdUtil.getJoinRowCount(mq, rel,
rel.getCondition());
+ }
+
+ baseRowCnt = rightRowCnt + leftRowCnt;
+ percentageAdjustment = 1.0 - selectivity;
+ }
+ }
+ else { // PK_ON_FK
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ baseRowCnt = rightRowCnt;
+ percentageAdjustment =
mq.getPercentageOriginalRows(rel.getLeft());
+ }
+ else if (rel.getJoinType() == JoinRelType.RIGHT ||
rel.getJoinType() == JoinRelType.LEFT) {
+ baseRowCnt = rightRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.FULL) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+
+ // Fall-back to calcite's implementation.
+ if (selectivity == null) {
+ return RelMdUtil.getJoinRowCount(mq, rel,
rel.getCondition());
+ }
+
+ baseRowCnt = rightRowCnt + leftRowCnt;
+ percentageAdjustment = 1.0 - selectivity;
+ }
+ }
+
+ if (percentageAdjustment == null) {
+ percentageAdjustment = 1.0; // No info, let's be conservative
+ }
+
+ return baseRowCnt * percentageAdjustment * postFiltrationAdjustment;
+ }
+
+ /** */
+ private static Map<Integer, KeyColumnOrigin>
resolveOrigins(RelMetadataQuery mq, RelNode joinShoulder, ImmutableIntList
keys) {
+ GridLeanMap<Integer, KeyColumnOrigin> origins = new GridLeanMap<>();
+
+ for (int i : keys) {
+ if (origins.containsKey(i)) {
+ continue;
+ }
+
+ RelColumnOrigin origin = mq.getColumnOrigin(joinShoulder, i);
+ if (origin == null) {
+ continue;
+ }
+ IgniteTable table =
origin.getOriginTable().unwrap(IgniteTable.class);
+ if (table == null || !table.distribution().function().affinity())
+ continue;
+
+ // Keys can relate to affinity not pk, just assumption here.
+ ImmutableIntList distKeys = table.distribution().getKeys();
Review Comment:
1. Check for columns description with `key()` is too huge?
2. For broadcast table and for tables with affinity column defined
estimation is wrong. I thing it's a relatively big part of all cases.
##########
modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/planner/JoinRowCountEstimationTest.java:
##########
@@ -0,0 +1,347 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.query.calcite.planner;
+
+import java.math.BigDecimal;
+import java.util.function.Predicate;
+import java.util.regex.Pattern;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.sql.SqlExplainFormat;
+import org.apache.calcite.sql.SqlExplainLevel;
+import org.apache.calcite.util.ImmutableIntList;
+import
org.apache.ignite.internal.processors.query.calcite.rel.IgniteCorrelatedNestedLoopJoin;
+import org.apache.ignite.internal.processors.query.calcite.schema.IgniteSchema;
+import
org.apache.ignite.internal.processors.query.calcite.trait.IgniteDistributions;
+import org.apache.ignite.internal.util.typedef.internal.CU;
+import org.hamcrest.BaseMatcher;
+import org.hamcrest.CoreMatchers;
+import org.hamcrest.Description;
+import org.hamcrest.Matcher;
+import org.junit.Test;
+
+import static
org.apache.ignite.internal.processors.query.calcite.metadata.IgniteMdSelectivity.COMPARISON_SELECTIVITY;
+import static
org.apache.ignite.internal.processors.query.calcite.metadata.IgniteMdSelectivity.EQUALS_SELECTIVITY;
+import static
org.apache.ignite.internal.processors.query.calcite.metadata.IgniteMdSelectivity.IS_NOT_NULL_SELECTIVITY;
+
+/**
+ * Tests to check row count estimation for join relation.
+ */
+public class JoinRowCountEstimationTest extends AbstractPlannerTest {
+ /** */
+ private static final int CATALOG_SALES_SIZE = 1_441_548;
+
+ /** */
+ private static final int CATALOG_RETURNS_SIZE = 144_067;
+
+ /** */
+ private static final int DATE_DIM_SIZE = 73_049;
+
+ /** */
+ private static final String SELECT = "SELECT /*+ DISABLE_RULE(" +
+ "'JoinCommuteRule', " +
+ "'MergeJoinConverter', " +
+ "'NestedLoopJoinConverter', " +
+ "'CorrelateToNestedLoopRule'" +
+ ") */ * ";
+
+ /** */
+ private IgniteSchema publicSchema;
+
+ /** {@inheritDoc} */
+ @Override public void setup() {
Review Comment:
Add annotation BeforeClass
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -72,48 +116,272 @@ public class IgniteMdRowCount extends RelMdRowCount {
mq.getRowCount(rel.getLeft()));
}
+ JoinInfo joinInfo = rel.analyzeCondition();
+
+ if (joinInfo.pairs().isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
// Row count estimates of 0 will be rounded up to 1.
// So, use maxRowCount where the product is very small.
- final Double left = mq.getRowCount(rel.getLeft());
- final Double right = mq.getRowCount(rel.getRight());
+ final Double leftRowCnt = mq.getRowCount(rel.getLeft());
+ final Double rightRowCnt = multiply(mq.getRowCount(rel.getRight()),
rightAdjust);
- if (left == null || right == null)
+ if (leftRowCnt == null || rightRowCnt == null)
return null;
- if (left <= 1D || right <= 1D) {
+ if (leftRowCnt <= 1D || rightRowCnt <= 1D) {
Double max = mq.getMaxRowCount(rel);
if (max != null && max <= 1D)
return max;
}
- JoinInfo joinInfo = rel.analyzeCondition();
+ Map<Integer, KeyColumnOrigin> columnsFromLeft = resolveOrigins(mq,
rel.getLeft(), joinInfo.leftKeys);
+ Map<Integer, KeyColumnOrigin> columnsFromRight = resolveOrigins(mq,
rel.getRight(), joinInfo.rightKeys);
+
+ if (columnsFromLeft.isEmpty() || columnsFromRight.isEmpty()) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ Map<TablesPair, JoinContext> joinCtxts = new HashMap<>();
+ for (IntPair joinKeys : joinInfo.pairs()) {
+ KeyColumnOrigin leftKey = columnsFromLeft.get(joinKeys.source);
+ KeyColumnOrigin rightKey = columnsFromRight.get(joinKeys.target);
+ if (leftKey == null || rightKey == null) {
+ continue;
+ }
+
+ joinCtxts.computeIfAbsent(
+ new TablesPair(
+ leftKey.origin.getOriginTable(),
+ rightKey.origin.getOriginTable()
+ ),
+ key -> {
+ IgniteTable leftTable = key.left.unwrap(IgniteTable.class);
+ IgniteTable rightTable =
key.right.unwrap(IgniteTable.class);
+
+ assert leftTable != null && rightTable != null;
+
+ int leftPkSize = leftTable.distribution().getKeys().size();
+ int rightPkSize =
rightTable.distribution().getKeys().size();
+
+ return new JoinContext(leftPkSize, rightPkSize);
+ }
+ ).countKeys(leftKey, rightKey);
+ }
+
+ if (joinCtxts.isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
+ Iterator<JoinContext> it = joinCtxts.values().iterator();
+ JoinContext joinCtx = it.next();
+ while (it.hasNext()) {
+ JoinContext nextCtx = it.next();
+ if (nextCtx.joinType().strength > joinCtx.joinType().strength) {
+ joinCtx = nextCtx;
+ }
+
+ if (joinCtx.joinType().strength ==
JoiningRelationType.PK_ON_PK.strength) {
+ break;
+ }
+ }
+
+ if (joinCtx.joinType() == JoiningRelationType.UNKNOWN) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ double postFiltrationAdjustment = 1.0;
+
+ switch (rel.getJoinType()) {
+ case INNER:
+ case SEMI:
+ // Extra join keys as well as non-equi conditions serves as
post-filtration,
+ // therefore we need to adjust final result with a little
factor.
+ if (joinCtxts.size() != 1 || !joinInfo.isEqui())
+ postFiltrationAdjustment = NON_EQUI_COEFF;
+
+ break;
+ default:
+ break;
+ }
+
+ double baseRowCnt = 0.0;
+ Double percentageAdjustment = null;
+ if (joinCtx.joinType() == JoiningRelationType.PK_ON_PK) {
+ postFiltrationAdjustment = 1.0;
+
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ postFiltrationAdjustment = EQUI_COEFF;
+ // Assume we have two fact tables SALES and RETURNS sharing
the same primary key. Every item
+ // can be sold, but only items which were sold can be returned
back, therefore
+ // size(SALES) > size(RETURNS). When joining SALES on RETURNS
by primary key, the estimated
+ // result size will be the same as the size of the smallest
table (RETURNS in this case),
+ // adjusted by the percentage of rows of the biggest table
(SALES in this case; percentage
+ // adjustment is required to account for predicates pushed
down to the table, e.g. we are
+ // interested in returns of items with certain category)
+ if (leftRowCnt > rightRowCnt) {
+ baseRowCnt = rightRowCnt;
+ percentageAdjustment =
mq.getPercentageOriginalRows(rel.getLeft());
+ }
+ else {
+ baseRowCnt = leftRowCnt;
+ percentageAdjustment = rightPercentage(mq, rel,
rightAdjust);
+ }
+ }
+ else if (rel.getJoinType() == JoinRelType.LEFT) {
+ baseRowCnt = leftRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.RIGHT) {
+ baseRowCnt = rightRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.FULL) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+
+ // Fall-back.
+ if (selectivity == null) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel,
rel.getCondition());
+ }
+
+ baseRowCnt = rightRowCnt + leftRowCnt;
+ percentageAdjustment = 1.0 - selectivity;
Review Comment:
For full join, if one table is subset of another:
rowCnt = leftRowCnt + rightRowCnt - intersetion,
intersection = min(part of left rows in right table, part of right rows in
left table) = min(leftRowCnt * rightPercentage, rightRowCnt * leftPercentage)
or intersect = leftRowCnt * rightRowCnt / Math.max(leftOrigCnt, rightOrigCnt);
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -157,4 +425,120 @@ public double getRowCount(IgniteLimit rel,
RelMetadataQuery mq) {
public double getRowCount(IgniteTableModify rel, RelMetadataQuery mq) {
return rel.estimateRowCount(mq);
}
+
+ /** */
+ private record TablesPair(RelOptTable left, RelOptTable right) { }
+
+ /** */
+ private static class JoinContext {
+ /** Used columns of primary key of table from left side. */
+ private final BitSet leftKeys;
+
+ /** Used columns of primary key of table from right side. */
+ private final BitSet rightKeys;
+
+ /**
+ * Used columns of primary key in both tables.
+ *
+ * <p>This bitset is initialized when PK of both tables has equal
columns count, and
+ * bits are cleared when join pair contains columns with equal
positions in PK of corresponding
+ * table. For example, having tables T1 and T2 with primary keys in
both tables defined as
+ * CONSTRAINT PRIMARY KEY (a, b), in case of query
+ * {@code SELECT ... FROM t1 JOIN t2 ON t1.a = t2.a AND t1.b = t2.b}
commonKeys will be initialized
+ * and cleared, but in case of query {@code SELECT ... FROM t1 JOIN t2
ON t1.a = t2.b AND t1.b = t2.a}
Review Comment:
It's strange. commonKeys are only required to detect PK_ON_PK join, why
`t1.a = t2.b AND t1.b = t2.a` is not PK_ON_PK if both `a` and `b` are PK in
both tables?
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -72,48 +116,272 @@ public class IgniteMdRowCount extends RelMdRowCount {
mq.getRowCount(rel.getLeft()));
}
+ JoinInfo joinInfo = rel.analyzeCondition();
+
+ if (joinInfo.pairs().isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
// Row count estimates of 0 will be rounded up to 1.
// So, use maxRowCount where the product is very small.
- final Double left = mq.getRowCount(rel.getLeft());
- final Double right = mq.getRowCount(rel.getRight());
+ final Double leftRowCnt = mq.getRowCount(rel.getLeft());
+ final Double rightRowCnt = multiply(mq.getRowCount(rel.getRight()),
rightAdjust);
- if (left == null || right == null)
+ if (leftRowCnt == null || rightRowCnt == null)
return null;
- if (left <= 1D || right <= 1D) {
+ if (leftRowCnt <= 1D || rightRowCnt <= 1D) {
Double max = mq.getMaxRowCount(rel);
if (max != null && max <= 1D)
return max;
}
- JoinInfo joinInfo = rel.analyzeCondition();
+ Map<Integer, KeyColumnOrigin> columnsFromLeft = resolveOrigins(mq,
rel.getLeft(), joinInfo.leftKeys);
+ Map<Integer, KeyColumnOrigin> columnsFromRight = resolveOrigins(mq,
rel.getRight(), joinInfo.rightKeys);
+
+ if (columnsFromLeft.isEmpty() || columnsFromRight.isEmpty()) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ Map<TablesPair, JoinContext> joinCtxts = new HashMap<>();
+ for (IntPair joinKeys : joinInfo.pairs()) {
+ KeyColumnOrigin leftKey = columnsFromLeft.get(joinKeys.source);
+ KeyColumnOrigin rightKey = columnsFromRight.get(joinKeys.target);
+ if (leftKey == null || rightKey == null) {
+ continue;
+ }
+
+ joinCtxts.computeIfAbsent(
+ new TablesPair(
+ leftKey.origin.getOriginTable(),
+ rightKey.origin.getOriginTable()
+ ),
+ key -> {
+ IgniteTable leftTable = key.left.unwrap(IgniteTable.class);
+ IgniteTable rightTable =
key.right.unwrap(IgniteTable.class);
+
+ assert leftTable != null && rightTable != null;
+
+ int leftPkSize = leftTable.distribution().getKeys().size();
+ int rightPkSize =
rightTable.distribution().getKeys().size();
+
+ return new JoinContext(leftPkSize, rightPkSize);
+ }
+ ).countKeys(leftKey, rightKey);
+ }
+
+ if (joinCtxts.isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
+ Iterator<JoinContext> it = joinCtxts.values().iterator();
+ JoinContext joinCtx = it.next();
+ while (it.hasNext()) {
+ JoinContext nextCtx = it.next();
+ if (nextCtx.joinType().strength > joinCtx.joinType().strength) {
+ joinCtx = nextCtx;
+ }
+
+ if (joinCtx.joinType().strength ==
JoiningRelationType.PK_ON_PK.strength) {
+ break;
+ }
+ }
+
+ if (joinCtx.joinType() == JoiningRelationType.UNKNOWN) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ double postFiltrationAdjustment = 1.0;
+
+ switch (rel.getJoinType()) {
+ case INNER:
+ case SEMI:
+ // Extra join keys as well as non-equi conditions serves as
post-filtration,
+ // therefore we need to adjust final result with a little
factor.
+ if (joinCtxts.size() != 1 || !joinInfo.isEqui())
+ postFiltrationAdjustment = NON_EQUI_COEFF;
+
+ break;
+ default:
+ break;
+ }
+
+ double baseRowCnt = 0.0;
+ Double percentageAdjustment = null;
+ if (joinCtx.joinType() == JoiningRelationType.PK_ON_PK) {
+ postFiltrationAdjustment = 1.0;
+
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ postFiltrationAdjustment = EQUI_COEFF;
Review Comment:
Why do we redefine post filtration here? If we have two tables and first is
a subset of second, if no additional (non equi) conditions are defined, no
additional EQUI_COEF is required, row count will be firstTableRowCoun *
percentOfTotalRowsFromSecondTable (if data from second table are selected
randomly). If there are any non-equi conditions, row count should be reduced by
NON_EQUI_COEFF (in case of other join types too)
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/metadata/IgniteMdRowCount.java:
##########
@@ -72,48 +116,272 @@ public class IgniteMdRowCount extends RelMdRowCount {
mq.getRowCount(rel.getLeft()));
}
+ JoinInfo joinInfo = rel.analyzeCondition();
+
+ if (joinInfo.pairs().isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
// Row count estimates of 0 will be rounded up to 1.
// So, use maxRowCount where the product is very small.
- final Double left = mq.getRowCount(rel.getLeft());
- final Double right = mq.getRowCount(rel.getRight());
+ final Double leftRowCnt = mq.getRowCount(rel.getLeft());
+ final Double rightRowCnt = multiply(mq.getRowCount(rel.getRight()),
rightAdjust);
- if (left == null || right == null)
+ if (leftRowCnt == null || rightRowCnt == null)
return null;
- if (left <= 1D || right <= 1D) {
+ if (leftRowCnt <= 1D || rightRowCnt <= 1D) {
Double max = mq.getMaxRowCount(rel);
if (max != null && max <= 1D)
return max;
}
- JoinInfo joinInfo = rel.analyzeCondition();
+ Map<Integer, KeyColumnOrigin> columnsFromLeft = resolveOrigins(mq,
rel.getLeft(), joinInfo.leftKeys);
+ Map<Integer, KeyColumnOrigin> columnsFromRight = resolveOrigins(mq,
rel.getRight(), joinInfo.rightKeys);
+
+ if (columnsFromLeft.isEmpty() || columnsFromRight.isEmpty()) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ Map<TablesPair, JoinContext> joinCtxts = new HashMap<>();
+ for (IntPair joinKeys : joinInfo.pairs()) {
+ KeyColumnOrigin leftKey = columnsFromLeft.get(joinKeys.source);
+ KeyColumnOrigin rightKey = columnsFromRight.get(joinKeys.target);
+ if (leftKey == null || rightKey == null) {
+ continue;
+ }
+
+ joinCtxts.computeIfAbsent(
+ new TablesPair(
+ leftKey.origin.getOriginTable(),
+ rightKey.origin.getOriginTable()
+ ),
+ key -> {
+ IgniteTable leftTable = key.left.unwrap(IgniteTable.class);
+ IgniteTable rightTable =
key.right.unwrap(IgniteTable.class);
+
+ assert leftTable != null && rightTable != null;
+
+ int leftPkSize = leftTable.distribution().getKeys().size();
+ int rightPkSize =
rightTable.distribution().getKeys().size();
+
+ return new JoinContext(leftPkSize, rightPkSize);
+ }
+ ).countKeys(leftKey, rightKey);
+ }
+
+ if (joinCtxts.isEmpty()) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel, rel.getCondition());
+ }
+
+ Iterator<JoinContext> it = joinCtxts.values().iterator();
+ JoinContext joinCtx = it.next();
+ while (it.hasNext()) {
+ JoinContext nextCtx = it.next();
+ if (nextCtx.joinType().strength > joinCtx.joinType().strength) {
+ joinCtx = nextCtx;
+ }
+
+ if (joinCtx.joinType().strength ==
JoiningRelationType.PK_ON_PK.strength) {
+ break;
+ }
+ }
+
+ if (joinCtx.joinType() == JoiningRelationType.UNKNOWN) {
+ // Use crude estimation instead.
+ return crudeEstimation(mq, joinInfo, rel, leftRowCnt, rightRowCnt);
+ }
+
+ double postFiltrationAdjustment = 1.0;
+
+ switch (rel.getJoinType()) {
+ case INNER:
+ case SEMI:
+ // Extra join keys as well as non-equi conditions serves as
post-filtration,
+ // therefore we need to adjust final result with a little
factor.
+ if (joinCtxts.size() != 1 || !joinInfo.isEqui())
+ postFiltrationAdjustment = NON_EQUI_COEFF;
+
+ break;
+ default:
+ break;
+ }
+
+ double baseRowCnt = 0.0;
+ Double percentageAdjustment = null;
+ if (joinCtx.joinType() == JoiningRelationType.PK_ON_PK) {
+ postFiltrationAdjustment = 1.0;
+
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ postFiltrationAdjustment = EQUI_COEFF;
+ // Assume we have two fact tables SALES and RETURNS sharing
the same primary key. Every item
+ // can be sold, but only items which were sold can be returned
back, therefore
+ // size(SALES) > size(RETURNS). When joining SALES on RETURNS
by primary key, the estimated
+ // result size will be the same as the size of the smallest
table (RETURNS in this case),
+ // adjusted by the percentage of rows of the biggest table
(SALES in this case; percentage
+ // adjustment is required to account for predicates pushed
down to the table, e.g. we are
+ // interested in returns of items with certain category)
+ if (leftRowCnt > rightRowCnt) {
+ baseRowCnt = rightRowCnt;
+ percentageAdjustment =
mq.getPercentageOriginalRows(rel.getLeft());
+ }
+ else {
+ baseRowCnt = leftRowCnt;
+ percentageAdjustment = rightPercentage(mq, rel,
rightAdjust);
+ }
+ }
+ else if (rel.getJoinType() == JoinRelType.LEFT) {
+ baseRowCnt = leftRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.RIGHT) {
+ baseRowCnt = rightRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.FULL) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+
+ // Fall-back.
+ if (selectivity == null) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel,
rel.getCondition());
+ }
+
+ baseRowCnt = rightRowCnt + leftRowCnt;
+ percentageAdjustment = 1.0 - selectivity;
+ }
+ }
+ else if (joinCtx.joinType() == JoiningRelationType.FK_ON_PK) {
+ // For foreign key joins the base table is the one which is joined
by non-primary key columns.
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+ baseRowCnt = leftRowCnt * selectivity;
+ percentageAdjustment = rightPercentage(mq, rel, rightAdjust);
+ }
+ else if (rel.getJoinType() == JoinRelType.LEFT ||
rel.getJoinType() == JoinRelType.RIGHT) {
+ baseRowCnt = leftRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.FULL) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+
+ // Fall-back.
+ if (selectivity == null) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel,
rel.getCondition());
+ }
+
+ baseRowCnt = rightRowCnt + leftRowCnt;
+ percentageAdjustment = 1.0 - selectivity;
+ }
+ }
+ else { // PK_ON_FK
+ if (rel.getJoinType() == JoinRelType.INNER || rel.getJoinType() ==
JoinRelType.SEMI) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+ baseRowCnt = rightRowCnt * selectivity;;
+ percentageAdjustment =
mq.getPercentageOriginalRows(rel.getLeft());
+ }
+ else if (rel.getJoinType() == JoinRelType.RIGHT ||
rel.getJoinType() == JoinRelType.LEFT) {
+ baseRowCnt = rightRowCnt;
+ }
+ else if (rel.getJoinType() == JoinRelType.FULL) {
+ Double selectivity = mq.getSelectivity(rel,
rel.getCondition());
+
+ // Fall-back.
+ if (selectivity == null) {
+ // Fall-back to calcite's implementation.
+ return RelMdUtil.getJoinRowCount(mq, rel,
rel.getCondition());
+ }
+
+ baseRowCnt = rightRowCnt + leftRowCnt;
+ percentageAdjustment = 1.0 - selectivity;
+ }
+ }
+
+ if (percentageAdjustment == null) {
+ // No info, let's be conservative.
+ percentageAdjustment = 1.0;
+ }
+
+ return baseRowCnt * percentageAdjustment * postFiltrationAdjustment;
+ }
+
+ /** Percentage of original rows of the right input, see {@link
#joinRowCount(RelMetadataQuery, Join, double)}. */
+ @Nullable private static Double rightPercentage(RelMetadataQuery mq, Join
rel, double rightAdjust) {
+ Double percentage = mq.getPercentageOriginalRows(rel.getRight());
+
+ return percentage == null ? null : Math.min(1.0, percentage *
rightAdjust);
+ }
+
+ /** */
+ private static Map<Integer, KeyColumnOrigin>
resolveOrigins(RelMetadataQuery mq, RelNode joinShoulder, ImmutableIntList
keys) {
Review Comment:
AFAIK shoulder - it's incorrect translation to English, not commonly used
term. Left/right hand, or left/right part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
