Joe McDonnell created IMPALA-15186:
--------------------------------------
Summary: Grouping pre-aggs have noticeable overhead due to
multiple partitions / BufferedTupleStreams
Key: IMPALA-15186
URL: https://issues.apache.org/jira/browse/IMPALA-15186
Project: IMPALA
Issue Type: Improvement
Components: Backend
Affects Versions: Impala 5.0.0
Reporter: Joe McDonnell
Grouping aggregators use 16 partitions, each with its own BufferedTupleStream.
This means that it can have memory overhead of 16 * buffer size. For large
aggregations, the buffer size is 2MB, so this can have an overhead of up to
32MB per thread. The separate partitions are necessary because the grouping
aggregator can spill individual partitions.
Grouping pre-aggs currently use the same code. Because they do not spill, the
partitioning is not necessary. Switching the pre-aggs to use a single partition
rather than 16 can reduce the memory overhead by 16x. For TPC-DS Q67, this is
fairly substantial for the big pre-agg:
{noformat}
With 16 partitioned:
07:AGGREGATE 3 12 1s396ms 1s469ms 11.71M
206.01M 323.91 MB 3.86 GB STREAMING
With 1 partition:
07:AGGREGATE 3 12 1s240ms 1s299ms 11.88M
206.01M 211.13 MB 3.86 GB STREAMING{noformat}
The downside is that rows coming out of the pre-agg are not partitioned
anymore. This means that the aggregator past the exchange is receiving rows
covering all 16 partitions randomly rather than a single partition at a time.
This increases its working set size. For large aggregations that are stressing
the CPU cache (particularly L3), this can push it over the edge and regress
performance dramatically. This comes up for TPC-H Q18 in some configurations.
(These scenarios would benefit from a more sophisticated aggregation
implementation to avoids stressing the CPU cache.)
Another approach would be to share the BufferedTupleStream between the 16
partitions for pre-aggs. The reduction in memory overhead should be basically
the same, but the output would remain partitioned like it is today.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
