[
https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16829038#comment-16829038
]
Stephan Ewen commented on FLINK-12070:
--------------------------------------
[~kevin.cyj] The pull request contains a newer version that works more in
memory: The file is immediately mapped and writes go to that memory as well,
lazily going to the file. That might improve the performance for smaller
results.
Concerning the Bug: As per the JavaDocs, closing the file should not affect the
validity of the memory mapping. If that is a problem after all (and the
JavaDocs are wrong or confusing there), then it should be easy to move the file
closing to where file deletion also happens (the unmapAndDeleteFile() method)
in the PR. That way, the file channel would stay open the full time.
> Make blocking result partitions consumable multiple times
> ---------------------------------------------------------
>
> Key: FLINK-12070
> URL: https://issues.apache.org/jira/browse/FLINK-12070
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Till Rohrmann
> Assignee: BoWang
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2019-04-18-17-38-24-949.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to avoid writing produced results multiple times for multiple
> consumers and in order to speed up batch recoveries, we should make the
> blocking result partitions to be consumable multiple times. At the moment a
> blocking result partition will be released once the consumers has processed
> all data. Instead the result partition should be released once the next
> blocking result has been produced and all consumers of a blocking result
> partition have terminated. Moreover, blocking results should not hold on slot
> resources like network buffers or memory as it is currently the case with
> {{SpillableSubpartitions}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
