Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18543 )
Change subject: IMPALA-11301: Fix extreme = and != selectivity for NDV=1 ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/18543/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/18543/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@267 PS1, Line 267: if (distinctValues == 1) distinctValues = 2; > Yeah the NOT col = 1 is a good use case to justify this. I am good with i Hmm..not sure what I meant by the last sentence in my comment above. Pls ignore that part. Looking at the Jenkins test output, I see several diffs in the hash join cardinality estimates for TPC-DS queries. I was worried about plan changes but wasn't expecting it for TPC-DS since I doubt there are columns with ndv=1. However, what might be happening is that the ndv gets adjusted based on filtering that was applied earlier in the plan. A join's cardinality estimate formula leverages the ndv coming from the child which may have been adjusted (reduced). We could look at all the plan diffs but it would need some time to validate functionally and with a performance run. Alternatively, we could limit the scope of this fix to the != predicates. -- To view, visit http://gerrit.cloudera.org:8080/18543 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6b5334a8d7d6ca46a450ff98ae03e5269faaa3c6 Gerrit-Change-Number: 18543 Gerrit-PatchSet: 2 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 19 May 2022 02:26:58 +0000 Gerrit-HasComments: Yes
