Riza Suminto has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20684 )
Change subject: IMPALA-12548: Fix behavior of AGG_MEM_CORRELATION_FACTOR ...................................................................... IMPALA-12548: Fix behavior of AGG_MEM_CORRELATION_FACTOR AGG_MEM_CORRELATION_FACTOR has a valid value between 0.0 to 1.0. Like JOIN_SELECTIVITY_CORRELATION_FACTOR option, the correlation factor here is meant to reflect the correlation coefficient between grouping columns. A high value of AGG_MEM_CORRELATION_FACTOR should mean a high correlation between grouping columns. However, the implementation of this query option behaves the opposite. 1.0 is interpreted as no correlation at all in the code, while <1.0 is interpreted as somewhat correlated. This patch fixes the behavior so that the planner lower memory estimate as AGG_MEM_CORRELATION_FACTOR go higher. Testing: - Fix and pass PlannerTest#testAggNodeMaxMemEstimate. - Add testAggNodeLowMemEstimate and testAggNodeHighMemEstimate. Change-Id: I6f81db32a1818abc257957f6de942b5c9f36211a Reviewed-on: http://gerrit.cloudera.org:8080/20684 Reviewed-by: Michael Smith <[email protected]> Reviewed-by: Kurt Deschler <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/planner/AggregationNode.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A testdata/workloads/functional-planner/queries/PlannerTest/agg-node-high-mem-estimate.test A testdata/workloads/functional-planner/queries/PlannerTest/agg-node-low-mem-estimate.test 5 files changed, 2,364 insertions(+), 12 deletions(-) Approvals: Michael Smith: Looks good to me, but someone else must approve Kurt Deschler: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/20684 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6f81db32a1818abc257957f6de942b5c9f36211a Gerrit-Change-Number: 20684 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
