Yingjie Cao created FLINK-19297:
-----------------------------------
Summary: Make ResultPartitionWriter record-oriented
Key: FLINK-19297
URL: https://issues.apache.org/jira/browse/FLINK-19297
Project: Flink
Issue Type: Improvement
Components: Runtime / Network
Affects Versions: 1.12.0
Reporter: Yingjie Cao
Fix For: 1.12.0
Currently, ResultPartitionWriter is buffer-oriented, that is, RecordWriter will
add buffer of different channels to ResultPartitionWriter and the buffer
boundary serves as a nature boundary of data belonging to different channels.
However, this abstraction is not flexible enough to handle some cases where
records are appended a joint-structure shared by all channels and sorting is
used to cluster data belonging to different channels.
In this ticket, we propose to make ResultPartitionWriter record oriented which
offers more flexibility to the implementation of ResultPartitionWriter. And
based on the new record-oriented Interface, we will introduce the sort-merge
based blocking shuffle to Flink in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
