This is an automated email from the ASF dual-hosted git repository.
gaoyunhaii pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new dfa672f [FLINK-25780][network] Reduce the maximum size of data output
buffer per result partition for sort-shuffle
dfa672f is described below
commit dfa672f48d365ec938e21d0e15816e8d8de6eca4
Author: kevin.cyj <[email protected]>
AuthorDate: Mon Jan 24 20:23:13 2022 +0800
[FLINK-25780][network] Reduce the maximum size of data output buffer per
result partition for sort-shuffle
The data output buffer of sort-shuffle is for better disk IO performance
and currently, the total data output buffer size is 16M which is pretty big.
However, blocking request too many buffers may influence performance. This
patch reduces the maximum size of data output buffer from 16M to 8M to reduce
the buffer request time. Note that the selected value is an empirical one based
on the TPC-DS benchmark results.
This closes #18471.
---
.../flink/runtime/io/network/partition/SortMergeResultPartition.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
b/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
index 36fde49..9456762 100644
---
a/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
+++
b/flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
@@ -68,9 +68,9 @@ public class SortMergeResultPartition extends ResultPartition
{
/**
* Number of expected buffer size to allocate for data writing. Currently,
it is an empirical
- * value (16M) which can not be configured.
+ * value (8M) which can not be configured.
*/
- private static final int NUM_WRITE_BUFFER_BYTES = 16 * 1024 * 1024;
+ private static final int NUM_WRITE_BUFFER_BYTES = 8 * 1024 * 1024;
private final Object lock = new Object();
