[ 
https://issues.apache.org/jira/browse/FLINK-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346467#comment-16346467
 ] 

ASF GitHub Bot commented on FLINK-8523:
---------------------------------------

Github user zhijiangW commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5381#discussion_r164988217
  
    --- Diff: 
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java
 ---
    @@ -360,8 +360,9 @@ public boolean notifyBufferAvailable(Buffer buffer) {
     
                        // Important: double check the isReleased state inside 
synchronized block, so there is no
                        // race condition when notifyBufferAvailable and 
releaseAllResources running in parallel.
    -                   if (isReleased.get() || 
bufferQueue.getAvailableBufferSize() >= numRequiredBuffers) {
    +                   if (isReleased.get() || isBlocked() || 
bufferQueue.getAvailableBufferSize() >= numRequiredBuffers) {
    --- End diff --
    
    That is the exact issue I pointed above. 
    
    I think it will not cause credit leak currently.  But for further 
improvement, we can compare the current available credits with backlog after 
unblocking the channel. If not enough, we can trigger to request floating 
buffers at once. In current implementation, it has to wait next 
`onSenderBacklog` to trigger this process.


> Stop assigning floating buffers for blocked input channels in exactly-once 
> mode
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-8523
>                 URL: https://issues.apache.org/jira/browse/FLINK-8523
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>    Affects Versions: 1.5.0
>            Reporter: zhijiang
>            Assignee: zhijiang
>            Priority: Major
>
> In exactly-once mode, the input channel is set blocked state when reading 
> barrier from it. And the blocked state will be released after barrier 
> alignment or cancelled.
>  
> In credit-based network flow control, we should avoid assigning floating 
> buffers for blocked input channels because the buffers after barrier will not 
> be processed by operator until alignment.
> To do so, we can fully make use of floating buffers and speed up barrier 
> alignment in some extent.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to