[
https://issues.apache.org/jira/browse/FLINK-28377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingjie Cao updated FLINK-28377:
--------------------------------
Description: Currently, for sort blocking shuffle, the corresponding data
reader always allocate a fixed size of 8M buffers for shuffle data reading.
FLINK-28373 can increase buffer utilization, after which, we can reduce the
buffer size per request to reduce the time waiting for buffers. (This change is
guarded by TPC-DS test that there is no performance regression) (was:
Currently, for sort blocking shuffle, the corresponding data reader always
allocate a fixed size of buffers for shuffle data reading even when there is no
much data to read which will cause buffer waste. We can optimize this to
allocate less number of reading buffers if there is no enough data.)
> Decrease the memory size per request for sort-shuffle data read from 8M to 4M
> -----------------------------------------------------------------------------
>
> Key: FLINK-28377
> URL: https://issues.apache.org/jira/browse/FLINK-28377
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Yingjie Cao
> Priority: Major
> Fix For: 1.16.0
>
>
> Currently, for sort blocking shuffle, the corresponding data reader always
> allocate a fixed size of 8M buffers for shuffle data reading. FLINK-28373 can
> increase buffer utilization, after which, we can reduce the buffer size per
> request to reduce the time waiting for buffers. (This change is guarded by
> TPC-DS test that there is no performance regression)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
