[
https://issues.apache.org/jira/browse/FLINK-22153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17322133#comment-17322133
]
Till Rohrmann commented on FLINK-22153:
---------------------------------------
My first observation is that the sort-merge blocking shuffle is not documented.
This makes the feature effectively non-existent because our users won't know
that it is there.
> Manually test the sort-merge blocking shuffle
> ---------------------------------------------
>
> Key: FLINK-22153
> URL: https://issues.apache.org/jira/browse/FLINK-22153
> Project: Flink
> Issue Type: Task
> Components: Runtime / Network
> Affects Versions: 1.13.0
> Reporter: Yingjie Cao
> Assignee: Till Rohrmann
> Priority: Blocker
> Fix For: 1.13.0
>
>
> In 1.12, we introduced sort-merge blocking shuffle to Flink and in 1.13, the
> feature was optimized which improves the usability (fix direct memory OOM
> issue) and performance (introduce IO scheduling and broadcast optimization).
> The sort-merge blocking shuffle can be tested following the bellow process:
> # Write a simple batch job using either sql/table or DataStream API; (Word
> count should be enough)
> # Enable sort-merge blocking shuffle by setting
> taskmanager.network.sort-shuffle.min-parallelism to 1 in the Flink
> configuration file;
> # Submit and run the batch job with different parallelism and data volume;
> # Tune the relevant config options
> (taskmanager.network.blocking-shuffle.compression.enabled,
> taskmanager.network.sort-shuffle.min-buffers,
> taskmanager.memory.framework.off-heap.batch-shuffle.size) and see the
> influence.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
