[
https://issues.apache.org/jira/browse/FLINK-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979037#comment-16979037
]
Yingjie Cao commented on FLINK-14872:
-------------------------------------
Only limit the number of Buffers can be allocated can only reduce the
possibility of deadlock and gives users a chance to increase the number of
network buffers by configuration. To totally solve the problem, we may also
need to request core buffers directly from global NetworkBufferPool like the
exclusive buffers for InputGate, we may add a method to local buffer pool like
reserveRequiredBuffers and call it at result partition setup. Currently, core
buffer request of the result partition is lazy, the buffer is not allocated
from global pool until emitting records. Like the exclusive buffer request for
InputGate, we can add a request time out and let the users know that the number
of network buffer should be increased.
Another possible option is letting the InputGate release Buffer by spilling to
disk, though the implementation is more complicated.
Letting the ResultPartition reserve core buffer at setup may lead to waste of
Buffer, but I think that is not a big problem, most of the task will emit
records sooner or latter.
> Potential deadlock for task reading from blocking ResultPartition.
> ------------------------------------------------------------------
>
> Key: FLINK-14872
> URL: https://issues.apache.org/jira/browse/FLINK-14872
> Project: Flink
> Issue Type: Bug
> Reporter: Yingjie Cao
> Priority: Major
> Fix For: 1.10.0
>
>
> Currently, the buffer pool size of InputGate reading from blocking
> ResultPartition is unbounded which have a potential of using too many buffers
> and may lead to ResultPartition of the same task can not acquire enough core
> buffers and finally lead to deadlock.
> Considers the following case:
> Core buffers are reserved for InputGate and ResultPartition -> InputGate
> consumes lots of Buffer (not including the buffer reserved for
> ResultPartition) -> Other tasks acquire exclusive buffer for InputGate and
> trigger redistribute of Buffers (Buffers taken by previous InputGate can not
> be released) -> The first task of which InputGate uses lots of buffers begin
> to emit records but can not acquire enough core Buffers (Some operators may
> not emit records out immediately or there is just nothing to emit) ->
> Deadlock.
>
> I think we can fix this problem by limit the number of Buffers can be
> allocated by a InputGate which reads from blocking ResultPartition.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
