Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23732 )
Change subject: IMPALA-14594: Scale grouping aggregator's input cardinality by the number of backends ...................................................................... IMPALA-14594: Scale grouping aggregator's input cardinality by the number of backends The grouping aggregator uses input cardinality in its decision about whether to continue expanding for a pre-aggregation. Currently, each fragment instance is using the full input cardinality for this calculation, but it will only be processing a fraction of the input. This makes pre-aggregations very aggressive about expanding, because they assume very high levels of future reduction. This fixes the input cardinality calculation to scale it down by the number of fragment instances for this fragment. Each fragment instance now assumes it is processing 1/Nth of the input cardinality (where N is the number of fragment instances). This will make the pre-aggregation's expansion calculation more accurate. Testing: - Ran a core job Change-Id: Iaf66709f12ea9b7328ca4e4c5cd45a05c9dc3032 Reviewed-on: http://gerrit.cloudera.org:8080/23732 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/aggregation-node-base.cc M be/src/exec/aggregation-node-base.h M be/src/runtime/coordinator-backend-state.cc M common/thrift/ImpalaInternalService.thrift 4 files changed, 34 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/23732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iaf66709f12ea9b7328ca4e4c5cd45a05c9dc3032 Gerrit-Change-Number: 23732 Gerrit-PatchSet: 5 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
