[
https://issues.apache.org/jira/browse/FLINK-24035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17407129#comment-17407129
]
Piotr Nowojski commented on FLINK-24035:
----------------------------------------
Solution here is to always request at least a single buffer.
This is not an issue on the output side, because there are no
{{BufferListeners}} in that case. Also if buffer is requested on the output
side, it will be used and eventually flushed.
On the input on the other hand, with exclusive buffers > 0, we are already
requesting exclusive buffers in a blocking way with a timeout (FLINK-12852), so
we know that task will be able to make progress regardless if we notify about
more buffers or not. With exclusive buffers = 0, this solution requests a
single floating buffer, so we will also be able to make a progress. Once data
starts flowing/this single buffer will be recycled, listeners would be notified
and more buffers would be requested.
> Fix the deadlock issue caused by buffer listeners may not be notified
> ---------------------------------------------------------------------
>
> Key: FLINK-24035
> URL: https://issues.apache.org/jira/browse/FLINK-24035
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Network
> Affects Versions: 1.14.0
> Reporter: Yingjie Cao
> Assignee: Yingjie Cao
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> The buffer listeners are not notified when the the local buffer pool receives
> available notification from the global pool. This may cause potential
> deadlock issue:
> # A LocalBufferPool is created, but there is no available buffers in the
> global NetworkBufferPool.
> # The LocalBufferPool registers an available buffer listener to the global
> NetworkBufferPool.
> # The BufferManager requests buffers from the LocalBufferPool but no buffer
> is available. As a result, it registers an available buffer listener to the
> LocalBufferPool.
> # A buffer is recycled to the global pool and the local buffer pool is
> notified about the available buffer.
> # The local buffer pool requests the available buffer from the global pool
> but the registered available buffer listener of BufferManager is not notified
> and it can never get a chance to be notified so deadlock occurs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
