[
https://issues.apache.org/jira/browse/FLINK-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yingjie Cao reassigned FLINK-25781:
-----------------------------------
Assignee: Yingjie Cao
> Adjust the maximum number of buffers can be used per result partition for
> shuffle data read
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-25781
> URL: https://issues.apache.org/jira/browse/FLINK-25781
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: Yingjie Cao
> Assignee: Yingjie Cao
> Priority: Major
> Fix For: 1.15.0
>
>
> In the current sort-shuffle implementation, the maximum number of buffers can
> be used per result partition for shuffle data read is 32M. However, for large
> parallelism jobs, 32M is not enough and for small parallelism jobs, 32M may
> waste buffers. This ticket aims to adjust the maximum number of buffers can
> be used per result partition to let it adjust according to parallelism and
> the selected value is an empirical one based on the TPC-DS test results.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
