[
https://issues.apache.org/jira/browse/FLINK-27862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550819#comment-17550819
]
Weijie Guo commented on FLINK-27862:
------------------------------------
Hi [~aitozi], Thank you very much for your attention, welcome to participate in
it.
Let me share the current situation of this FLIP：
1、We already have a POC version in-house with some level of testing.
2、The implementation of this POC version is not exactly the same as the design
in FLIP-235. For example, the spill strategy adopts all data write to disk
strategy instead of selective spill strategy, etc.
3、In order to verify if there is a conflict in merging the code into the open
source Flink version, I pushed the code to a branch on my own ([github
repository|https://github.com/reswqa/flink/tree/hs-merge-from-vvr]). Since part
of the code is going to be discarded in the new design, it is not pick into the
test branch, so this branch cannot actually run. But it already contains the
core implementation of our POC version.
4、If you have any other questions, you are very welcome to communicate with me
offline.
> FLIP-235: Hybrid Shuffle Mode
> -----------------------------
>
> Key: FLINK-27862
> URL: https://issues.apache.org/jira/browse/FLINK-27862
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Network
> Reporter: Weijie Guo
> Assignee: Weijie Guo
> Priority: Major
> Labels: Umbrella
>
> Introduce a new shuffle mode can overcome some of the problems of Pipelined
> Shuffle and Blocking Shuffle in batch scenarios, it can make best use of
> available resources and minimize disk IO load.
> More details see
> [FLIP-235|https://cwiki.apache.org/confluence/display/FLINK/FLIP-235%3A+Hybrid+Shuffle+Mode]
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
