Yingjie Cao created FLINK-19297: ----------------------------------- Summary: Make ResultPartitionWriter record-oriented Key: FLINK-19297 URL: https://issues.apache.org/jira/browse/FLINK-19297 Project: Flink Issue Type: Improvement Components: Runtime / Network Affects Versions: 1.12.0 Reporter: Yingjie Cao Fix For: 1.12.0
Currently, ResultPartitionWriter is buffer-oriented, that is, RecordWriter will add buffer of different channels to ResultPartitionWriter and the buffer boundary serves as a nature boundary of data belonging to different channels. However, this abstraction is not flexible enough to handle some cases where records are appended a joint-structure shared by all channels and sorting is used to cluster data belonging to different channels. In this ticket, we propose to make ResultPartitionWriter record oriented which offers more flexibility to the implementation of ResultPartitionWriter. And based on the new record-oriented Interface, we will introduce the sort-merge based blocking shuffle to Flink in the future. -- This message was sent by Atlassian Jira (v8.3.4#803005)