Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18184 )
Change subject: IMPALA-10961: Implementing adaptive 3-way quicksort in sorter ...................................................................... Patch Set 13: > Patch Set 13: > > > > Patch Set 12: Verified+1 Code-Review+2 > > > > > > Looked at the failed build, the errors seem unrelated flaky tests > > (e.g. IMPALA-10754) > > > > According to > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/15800/testReport/junit/org.apache.impala.planner/CardinalityTest/testAggregationNodeGroupByCardinalityCapping/, > > it seems this patch introduced some non-determinism. > > This shouldn't be related, as the failed test is COUNT DISTINCT related, so > it shouldn't be affected by changes in sort. + the test is about FE > estimations, and shouldn't be affected by BE only changes. > > One possibility I see is that the changed order of elements could have > affected the tables written during dataload - but this shouldn't be a new > thing, as the equal elements were never deterministic during sorting. Thanks for the analysis Csaba! Yeah I think you are right. According to https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/planner/CardinalityTest.java#L42, we have {{CARDINALITY_TOLERANCE}} that is used to tune how tolerant we are with respect to a given cardinality estimate. The value is currently set to 0.05. Maybe we could consider increaseing this a bit, e.g., to 0.07. -- To view, visit http://gerrit.cloudera.org:8080/18184 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I81e7b36a04a43de3b83e6aeee49ca0943f0bf202 Gerrit-Change-Number: 18184 Gerrit-PatchSet: 13 Gerrit-Owner: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Mon, 21 Feb 2022 22:41:56 +0000 Gerrit-HasComments: No
