[
https://issues.apache.org/jira/browse/FLINK-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhijiang updated FLINK-14498:
-----------------------------
Description:
In order to best-effort implement non-blocking output, we need to further
improve the interaction between LocalBufferPool and NetworkBufferPool in
non-blocking way as a supplementation of FLINK-14396.
In detail, we provide the NetworkBufferPool#isAvailable to indicate the global
pool state, then we could combine its state via LocalBufferPool#isAvailable`
method to avoid blocking in global request while task processing.
Meanwhile we would refactor the process when LocalBufferPool requests global
buffer. If there are no available buffers in NetworkBufferPool, the
LocalBufferPool should monitor the global's available future instead of waiting
2 seconds currently in every loop retry. So we can solve the wait delay and
cleanup the codes in a unified way.
was:
In order to best-effort implement non-blocking output, we need to further
improve the interaction between LocalBufferPool and NetworkBufferPool in
non-blocking way as a supplementation of FLINK-14396.
In detail, we provide the NetworkBufferPool#isAvailable to indicate the global
pool state, then we could combine its state via LocalBufferPool#isAvailable`
method to avoid blocking in global request while task processing.
Meanwhile we would refactor the process when LocalBufferPool requests global
buffer. If there are no available buffers in NetworkBufferPool, the
LocalBufferPool should monitor the global's available future instead of waiting
2 seconds currently in every loop retry. So we can solve the wait delay and
cleanup the codes in a unified way.
> Introduce NetworkBufferPool#isAvailable() for non-blocking output
> -----------------------------------------------------------------
>
> Key: FLINK-14498
> URL: https://issues.apache.org/jira/browse/FLINK-14498
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: zhijiang
> Assignee: Yingjie Cao
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to best-effort implement non-blocking output, we need to further
> improve the interaction between LocalBufferPool and NetworkBufferPool in
> non-blocking way as a supplementation of FLINK-14396.
> In detail, we provide the NetworkBufferPool#isAvailable to indicate the
> global pool state, then we could combine its state via
> LocalBufferPool#isAvailable` method to avoid blocking in global request while
> task processing.
> Meanwhile we would refactor the process when LocalBufferPool requests global
> buffer. If there are no available buffers in NetworkBufferPool, the
> LocalBufferPool should monitor the global's available future instead of
> waiting 2 seconds currently in every loop retry. So we can solve the wait
> delay and cleanup the codes in a unified way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
