Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18543 )
Change subject: WIP: IMPALA-11301: Fix = and != selectivity for very low NDVs ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/18543/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/18543/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@267 PS1, Line 267: selectivity_ = 1.0 / (distinctValues + 1); If NDV == 1, this would now produce 0.5 instead of 1 which is not right. I suspect this change will change a lot of plans especially for predicates on boolean columns (ndv = 2). IMHO It would be better to restrict the fix to the non-equality predicates. http://gerrit.cloudera.org:8080/#/c/18543/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@273 PS1, Line 273: selectivity_ = 1.0 - 1.0 / (distinctValues + 1); Since the distinctValues == 1 is a special case which causes 0 to be produced with the old formula, could we just explicitly check for distinctValues == 1 and only in that case use the new formula ? For the other values, old formula should still be ok right ? Just want to see if a minimal change can be done since formula changes have to go through much more testing. -- To view, visit http://gerrit.cloudera.org:8080/18543 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6b5334a8d7d6ca46a450ff98ae03e5269faaa3c6 Gerrit-Change-Number: 18543 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Wed, 18 May 2022 17:21:40 +0000 Gerrit-HasComments: Yes
