Jian Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19030 )
Change subject: IMPALA-10891: Avoid hash exchanges in more situations ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/19030/3/fe/src/main/java/org/apache/impala/analysis/SlotRef.java File fe/src/main/java/org/apache/impala/analysis/SlotRef.java: http://gerrit.cloudera.org:8080/#/c/19030/3/fe/src/main/java/org/apache/impala/analysis/SlotRef.java@198 PS3, Line 198: numDistinctValues_ / rootTable.getNumRows() > Aren't we too strict here? My understanding is that the NDV should be high +1. IMHO, instead of checking the factor of NDV/NumRows, a better solution is to check the factor of NDV/NumImpalaBackends since we only need to ensure data can be somehow evenly redistributed after the exchange operator. http://gerrit.cloudera.org:8080/#/c/19030/3/testdata/workloads/functional-planner/queries/PlannerTest/optimize_shuffle.test File testdata/workloads/functional-planner/queries/PlannerTest/optimize_shuffle.test: http://gerrit.cloudera.org:8080/#/c/19030/3/testdata/workloads/functional-planner/queries/PlannerTest/optimize_shuffle.test@1 PS3, Line 1: optimize > Can you mention what did we optimize in this query exactly? My understandin +1. It would be helpful if there were some micro-benchmark results to show the performance improvement and resource utilization reduction after applying the optimization. -- To view, visit http://gerrit.cloudera.org:8080/19030 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie627d07b729f1f584f589cd0251dfe2bb64a417f Gerrit-Change-Number: 19030 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaoqing Gao <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jian Zhang <[email protected]> Gerrit-Comment-Date: Mon, 26 Sep 2022 02:36:35 +0000 Gerrit-HasComments: Yes
