Riza Suminto created IMPALA-12498:
-------------------------------------
Summary: MERGING-EXCHANGE can consume high memory if number of
sender is high
Key: IMPALA-12498
URL: https://issues.apache.org/jira/browse/IMPALA-12498
Project: IMPALA
Issue Type: Improvement
Components: Distributed Exec
Affects Versions: Impala 4.3.0
Reporter: Riza Suminto
MERGING-EXCHANGE dedicate one row batch queue per data stream sender with soft
limit of 10MB per queue. For large cluster with hundreds of backend executor,
this can lead MERGING-EXCHANGE to require hundreds of queues totaling hundreds
MBs of memory.
We need to improve the design to keep memory of coordinator fragment low.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
