[
https://issues.apache.org/jira/browse/FLINK-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhijiang updated FLINK-14498:
-----------------------------
Description:
In order to best-effort implement non-blocking output, we need to further
improve the interaction between LocalBufferPool and NetworkBufferPool in
non-blocking way as a supplementation of FLINK-14396.
In detail, we provide the NetworkBufferPool#isAvailable to indicate the global
pool state, then we could combine its state via LocalBufferPool#isAvailable`
method to avoid blocking in global request while task processing.
Meanwhile we would refactor the process when LocalBufferPool requests global
buffer. If there are no available buffers in NetworkBufferPool, the
LocalBufferPool should monitor the global's available future instead of waiting
2 seconds currently in every loop retry. So we can solve the wait delay and
cleanup the codes in a unified way.
was:
If the LocalBufferPool can not request available buffer from NetworkBufferPool,
it would wait for 2 seconds before trying to request again in a loop way.
Therefore it would bring some delays in practice.
To improve this interaction, we could introduce NetworkBufferPool#isAvailable
to return a future which would be monitored by LocalBufferPool. Then once there
are available buffers in NetworkBufferPool, it would complete this future to
notify LocalBufferPool immediately.
> Introduce NetworkBufferPool#isAvailable() for non-blocking output
> -----------------------------------------------------------------
>
> Key: FLINK-14498
> URL: https://issues.apache.org/jira/browse/FLINK-14498
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Reporter: zhijiang
> Assignee: Yingjie Cao
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to best-effort implement non-blocking output, we need to further
> improve the interaction between LocalBufferPool and NetworkBufferPool in
> non-blocking way as a supplementation of FLINK-14396.
> In detail, we provide the NetworkBufferPool#isAvailable to indicate the
> global pool state, then we could combine its state via
> LocalBufferPool#isAvailable` method to avoid blocking in global request while
> task processing.
>
> Meanwhile we would refactor the process when LocalBufferPool requests global
> buffer. If there are no available buffers in NetworkBufferPool, the
> LocalBufferPool should monitor the global's available future instead of
> waiting 2 seconds currently in every loop retry. So we can solve the wait
> delay and cleanup the codes in a unified way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
