[
https://issues.apache.org/jira/browse/FLINK-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15233507#comment-15233507
]
Ovidiu Marcu commented on FLINK-1404:
-------------------------------------
Thanks, I am trying to understand the engine including its runtime but I don't
'see' what kind of abstractions you have arrived to. Runtime intermediate
results are described in so many ways in Flink, but they rely on network
buffers also..
> Add support to cache intermediate results
> -----------------------------------------
>
> Key: FLINK-1404
> URL: https://issues.apache.org/jira/browse/FLINK-1404
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Runtime
> Reporter: Ufuk Celebi
> Assignee: Maximilian Michels
>
> With blocking intermediate results (FLINK-1350) and proper partition state
> management (FLINK-1359) it is necessary to allow the network buffer pool to
> request eviction of historic intermediate results when not enough buffers are
> available. With the currently available pipelined intermediate partitions
> this is not an issue, because buffer pools can be released as soon as a
> partition is consumed.
> We need to be able to trigger the recycling of buffers held by historic
> intermediate results when not enough buffers are available for new local
> pools.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
