[jira] [Commented] (FLINK-7406) Implement Netty receiver incoming pipeline for credit-based

[ASF GitHub Bot (JIRA)]

    [ 
https://issues.apache.org/jira/browse/FLINK-7406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264719#comment-16264719
 ] 

ASF GitHub Bot commented on FLINK-7406:
---------------------------------------

Github user NicoK commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4509#discussion_r152861029
  
    --- Diff: 
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java
 ---
    @@ -440,21 +476,25 @@ public void testFairDistributionFloatingBuffers() 
throws Exception {
        @Test
        public void testConcurrentOnSenderBacklogAndRelease() throws Exception {
                // Setup
    -           final NetworkBufferPool networkBufferPool = new 
NetworkBufferPool(256, 32, MemoryType.HEAP);
    +           final NetworkBufferPool networkBufferPool = new 
NetworkBufferPool(130, 32, MemoryType.HEAP);
    +           final int numExclusiveBuffers = 2;
    +           final int numFloatingBuffers = 128;
    +
                final ExecutorService executor = 
Executors.newFixedThreadPool(2);
    +
                final SingleInputGate inputGate = createSingleInputGate();
                final RemoteInputChannel inputChannel  = 
createRemoteInputChannel(inputGate);
                
inputGate.setInputChannel(inputChannel.partitionId.getPartitionId(), 
inputChannel);
                try {
    -                   final BufferPool bufferPool = 
networkBufferPool.createBufferPool(128, 128);
    +                   final BufferPool bufferPool = 
networkBufferPool.createBufferPool(numFloatingBuffers, numFloatingBuffers);
                        inputGate.setBufferPool(bufferPool);
    -                   inputGate.assignExclusiveSegments(networkBufferPool, 2);
    +                   inputGate.assignExclusiveSegments(networkBufferPool, 
numExclusiveBuffers);
     
    -                   final Callable<Void> requestBufferTask = new 
Callable<Void>() {
    +                   final Callable requestBufferTask = new Callable<Void>() 
{
    --- End diff --
    
    please keep `Callable<Void>` (or replace by a lambda)


> Implement Netty receiver incoming pipeline for credit-based
> -----------------------------------------------------------
>
>                 Key: FLINK-7406
>                 URL: https://issues.apache.org/jira/browse/FLINK-7406
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>            Reporter: zhijiang
>            Assignee: zhijiang
>             Fix For: 1.5.0
>
>
> This is a part of work for credit-based network flow control.
> Currently {{PartitionRequestClientHandler}} receives and reads 
> {{BufferResponse}} from producer. It will request buffer from {{BufferPool}} 
> for holding the message. If not got, the message is staged temporarily and 
> {{autoread}} for channel is set false.
> For credit-based mode, {{PartitionRequestClientHandler}} can always get 
> buffer from {{RemoteInputChannel}} for reading messages from producer.
> The related works are:
> * Add the backlog of producer in {{BufferResponse}} message structure
> * {{PartitionRequestClientHandler}} requests buffer from 
> {{RemoteInputChannel}} directly
> * {{PartitionRequestClientHandler}} updates backlog for 
> {{RemoteInputChannel}}, and it may trigger requests of floating buffers from 
> {{BufferPool}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)