Hello Quanlong Huang, Csaba Ringhofer, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23732
to look at the new patch set (#3).
Change subject: IMPALA-14594: Scale grouping aggregator's input cardinality by
the number of backends
......................................................................
IMPALA-14594: Scale grouping aggregator's input cardinality by the number of
backends
The grouping aggregator uses input cardinality in its decision
about whether to continue expanding for a pre-aggregation.
Currently, each fragment instance is using the full input
cardinality for this calculation, but it will only be processing
a fraction of the input. This makes pre-aggregations very aggressive
about expanding, because they assume very high levels of future
reduction.
This fixes the input cardinality calculation to scale it down
by the number of fragment instances for this fragment. Each
fragment instance now assumes it is processing 1/Nth of the
input cardinality (where N is the number of fragment instances).
This will make the pre-aggregation's expansion calculation
more accurate.
Testing:
- Ran a core job
Change-Id: Iaf66709f12ea9b7328ca4e4c5cd45a05c9dc3032
---
M be/src/exec/aggregation-node-base.cc
M be/src/exec/aggregation-node-base.h
M be/src/runtime/coordinator-backend-state.cc
M common/thrift/ImpalaInternalService.thrift
4 files changed, 34 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/32/23732/3
--
To view, visit http://gerrit.cloudera.org:8080/23732
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iaf66709f12ea9b7328ca4e4c5cd45a05c9dc3032
Gerrit-Change-Number: 23732
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
