ASF GitHub Bot commented on FLINK-7416:

GitHub user zhijiangW opened a pull request:


    [FLINK-7416][network] Implement Netty receiver outgoing pipeline for 

    ## What is the purpose of the change
    The receiver notifies the available buffers called unannounced credit here, 
then the sender controls sending data based on receiver's credit. 
    This PR realizes the process of sending credit via receiver's netty handler 
    This PR is based on #4509 whose commits are also included for passing 
travis, so review the fourth commit for this PR change.
    ## Brief change log
      - *Add new `AddCredit` message and add credit field in existing 
`PartitionRequest` message*
      - *Notify the initial credit via `PartitionRequest` message*
      - *Enqueue input channel in the pipeline for sending unannounced credit*
      - *Set method of `releaseAllResources` as public in `RemoteInputChannel 
for test call*
    ## Verifying this change
    This change added tests and can be verified as follows:
      - *Added test that verifies the netty message serialization for add 
credit and partition request*
      - *Added test to verify that the input channel is enqueued to trigger 
send credit by handler *
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (no)
      - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (yes)
      - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
    ## Documentation
      - Does this pull request introduce a new feature? (yes)
      - If yes, how is the feature documented? (docs)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zhijiangW/flink FLINK-7416

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4533
commit a64016633ef53e7116ce588a4110cff587acadf6
Author: Zhijiang <wangzhijiang...@aliyun.com>
Date:   2017-08-07T09:31:17Z

    [FLINK-7378][core]Implement the FixedBufferPool for floating buffers of 

commit 9b7ee9ff559a3bcfe7b8f08802cf55349bb72abc
Author: Zhijiang <wangzhijiang...@aliyun.com>
Date:   2017-08-09T05:43:56Z

    [FLINK-7394][core]Implement basic InputChannel for credit-based logic

commit 0e0cb51b2ef97c73ed4d17629b1928fb911c9c92
Author: Zhijiang <wangzhijiang...@aliyun.com>
Date:   2017-08-10T05:29:13Z

    [FLINK-7406][network]Implement Netty receiver incoming pipeline for 

commit 855f81b6184730408c5a13d69aebbd627dd53aea
Author: Zhijiang <wangzhijiang...@aliyun.com>
Date:   2017-08-12T14:13:25Z

    [FLINK-7416][network] Implement Netty receiver outgoing pipeline for 


> Implement Netty receiver outgoing pipeline for credit-based
> -----------------------------------------------------------
>                 Key: FLINK-7416
>                 URL: https://issues.apache.org/jira/browse/FLINK-7416
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>            Reporter: zhijiang
>            Assignee: zhijiang
>             Fix For: 1.4.0
> This is a part of work for credit-based network flow control.
> The related works are :
> * The {{InputChannel}} notifies the initial credit which equals to the number 
> of exclusive buffers per channel via {{PartitionRequest}} message.
> *  We define another message called {{AddCredit}} to notify the incremental 
> credit during data shuffle. 
> * Whenever an {{InputChannel}}’s unannounced credit goes up from zero, the 
> channel is enqueued in the pipeline.
> * Whenever the channel becomes writable, it takes the next {{InputChannel}} 
> and sends its unannounced credit. The credit is reset to zero after each sent.
> * That way, messages are sent as often as the network has capacity and 
> contain as much credit as available for the channel at that point in time. 
> Otherwise, it would only add latency to the announcements and not increase 
> throughput.

This message was sent by Atlassian JIRA

Reply via email to