[
https://issues.apache.org/jira/browse/FLINK-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16820414#comment-16820414
]
Stephan Ewen commented on FLINK-12070:
--------------------------------------
[~Ryantaocer] Can you explain this comment a bit more:
"multiple reads: the ByteBuffer object returned by the FileChannel.map() has an
internal position that corresponds to the sliding window of physical memory
within the range of mapping. In the sense, it cannot be shared among
simultaneous reads like speculative executions, which have differenct reading
positions resulting in frequent page-fault calling in OS. However, in other
cases, the ByteBuffer can be reset to the begin for subsequent reads."
What position are referring to here, could you share a pointer to the code?
My understanding was that which pages are in physical memory should be the
Kernel's decision, it should not be handled in the ByteBuffer.
> Make blocking result partitions consumable multiple times
> ---------------------------------------------------------
>
> Key: FLINK-12070
> URL: https://issues.apache.org/jira/browse/FLINK-12070
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Till Rohrmann
> Assignee: BoWang
> Priority: Major
>
> In order to avoid writing produced results multiple times for multiple
> consumers and in order to speed up batch recoveries, we should make the
> blocking result partitions to be consumable multiple times. At the moment a
> blocking result partition will be released once the consumers has processed
> all data. Instead the result partition should be released once the next
> blocking result has been produced and all consumers of a blocking result
> partition have terminated. Moreover, blocking results should not hold on slot
> resources like network buffers or memory as it is currently the case with
> {{SpillableSubpartitions}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
