Zoltán Borók-Nagy created IMPALA-15512:
------------------------------------------
Summary: Increase value of data_stream_sender_buffer_size
Key: IMPALA-15512
URL: https://issues.apache.org/jira/browse/IMPALA-15512
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Zoltán Borók-Nagy
Assignee: Zoltán Borók-Nagy
The current value (16KiB) of data_stream_sender_buffer_size originates from a
commit {*}14 years ago{*}:
[https://github.com/apache/impala/blame/bc08241ffbac0eec70b12ffc36929c9dbc97550c/be/src/exec/data-sink.cc#L29]
Today's hardware has significantly more RAM so it makes sense to re-evaluate a
proper value for it.
Impala 5 also provides a good opportunity to increase its value.
Increasing its value can significantly improve query performance, even 25% of
performance gain can be achieved on TPC-DS queries. (measured on a 5 node
cluster with
--data_stream_sender_buffer_size=128K, TPC-DS at scale 100). Setting it to 128K
might be a bold move, but at least 32K and 64K should be reasonable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
