Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18543 )
Change subject: IMPALA-11301: Fix extreme != selectivity for NDV=1 ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/18543/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java File fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java: http://gerrit.cloudera.org:8080/#/c/18543/1/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java@267 PS1, Line 267: selectivity_ = 1.0 / (distinctValues + 1); > Hmm..not sure what I meant by the last sentence in my comment above. Pls i Chosen to only touch the != case in the end to avoid side effects. I became unsure about the correct solution - I think that we should try to avoid extreme small selectivities in general, for example we could assume that 1% of rows always survives the predicate (unless we can really rule this out, for example when ndv=row_count) to rule out that a much bigger table falling on the build side. Generally it is not a huge issue if mistakenly the build side will have more rows than the probe side as long as it fits to memory, but should really try to avoid extremely large build sides. -- To view, visit http://gerrit.cloudera.org:8080/18543 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6b5334a8d7d6ca46a450ff98ae03e5269faaa3c6 Gerrit-Change-Number: 18543 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 19 May 2022 14:57:10 +0000 Gerrit-HasComments: Yes
