Tim Armstrong created IMPALA-9416:
-------------------------------------
Summary: Reduce minimum reservation for shared broadcast join
builders
Key: IMPALA-9416
URL: https://issues.apache.org/jira/browse/IMPALA-9416
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Reporter: Tim Armstrong
This is an optimisation that I'm deferring from IMPALA-9156. The initial
version of IMPALA-9156 will be conservative in the amount of memory it reserves
for the broadcast join builder.
We should make it more precise to avoid blowing up the memory requirements.
Specifically:
* the reservation should depend on the number of consuming fragments scheduled,
not mt_dop
* we may want to look at how to reduce the amount of memory reserved per probe
side. E.g. use smaller buffer sizes for the probe streams, under the assumption
that a broadcast join is unlikely to spill.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
