jcamachor commented on a change in pull request #671: HIVE-21857
URL: https://github.com/apache/hive/pull/671#discussion_r296410050
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistinctRowCount.java
##########
@@ -61,17 +62,11 @@ private HiveRelMdDistinctRowCount() {
@Override
public Double getDistinctRowCount(RelNode rel, RelMetadataQuery mq,
ImmutableBitSet groupKey,
RexNode predicate) {
- if (rel instanceof HiveTableScan) {
- return getDistinctRowCount((HiveTableScan) rel, mq, groupKey, predicate);
- }
- /*
- * For now use Calcite' default formulas for propagating NDVs up the Query
- * Tree.
- */
- return super.getDistinctRowCount(rel, mq, groupKey, predicate);
+ return NumberUtil.multiply(mq.getRowCount(rel),
Review comment:
The logic in the super method was:
```
// REVIEW zfong 4/19/06 - Broadbase code does not take into
// consideration selectivity of predicates passed in. Also, they
// assume the rows are unique even if the table is not
boolean uniq = RelMdUtil.areColumnsDefinitelyUnique(mq, rel, groupKey);
if (uniq) {
return NumberUtil.multiply(mq.getRowCount(rel),
mq.getSelectivity(rel, predicate));
}
return null;
```
I cannot figure out the logic behind it... if columns are definitely unique,
return distinct row count, otherwise return null?
In any case, I thought about this again and I changed the logic. I left
implementation as it was (I removed the method because it was just calling to
the super method) and I changed slightly the logic in FilterEstimator so it can
handle null values. Basically, null is unknown and you just push those
predicates in the AND / OR to the very end since you do not know how selective
they are. This may happen when we find an operator in the tree that is not
recognized yet in RelMdDistinctRowCount (e.g. DruidQuery).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
