amansinha100 commented on a change in pull request #1744: Drill 7148 - Join
order, multi-col ndv and aggregate rowcount fixes for TPCH queries
URL: https://github.com/apache/drill/pull/1744#discussion_r274512764
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelMdDistinctRowCount.java
##########
@@ -208,8 +244,64 @@ public Double getDistinctRowCount(DrillJoinRelBase
joinRel, RelMetadataQuery mq,
Double leftDistRowCount = null;
Double rightDistRowCount = null;
double distRowCount = 1;
+ int gbyCols = 0;
ImmutableBitSet lmb = leftMask.build();
ImmutableBitSet rmb = rightMask.build();
+ PlannerSettings plannerSettings =
PrelUtil.getPlannerSettings(joinRel.getCluster().getPlanner());
+ if (((int)plannerSettings.getStatisticsJoinCardinalityMode() & 4) == 4) {
+ /*
+ * The NDV for a multi-column GBY key past a join is determined as follows:
+ * GBY(s1, s2, s3) = CNDV(s1)*CNDV(s2)*CNDV(s3)
+ * where CNDV is determined as follows:
+ * A) If sX is present as a join column (sX = tX) CNDV(sX) = MIN(NDV(sX),
NDV(tX)) where X =1, 2, 3, etc
+ * B) Otherwise, based on independence assumption CNDV(sX) = NDV(sX)
+ */
+ Set<ImmutableBitSet> joinFiltersSet = new HashSet<>();
+ for (RexNode filter : joinFilters) {
+ final RelOptUtil.InputFinder inputFinder =
RelOptUtil.InputFinder.analyze(filter);
+ joinFiltersSet.add(inputFinder.inputBitSet.build());
+ }
+ for (int idx = 0; idx < groupKey.length(); idx++) {
+ if (groupKey.get(idx)) {
+ // GBY key is present in some filter - now try options A) and B) as
described above
+ double ndvSGby = 0;
+ ImmutableBitSet sGby = getSingleGbyKey(groupKey, idx);
+ if (sGby != null) {
+ for (ImmutableBitSet jFilter : joinFiltersSet) {
Review comment:
Suppose the query is ' SELECT distinct a1 FROM t1, t2, t3 where t1.a1 =
t2.a2 and t1.a1 = t2.a3 '
In this case, the grouping key is involved in 2 joins .. can we do an
offline walk-through to see whether the top join's NDV is being used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
