[
https://issues.apache.org/jira/browse/FLINK-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372887#comment-16372887
]
ASF GitHub Bot commented on FLINK-8747:
---------------------------------------
GitHub user zhijiangW opened a pull request:
https://github.com/apache/flink/pull/5558
[FLINK-8747][bugfix] The tag of waiting for floating buffers in
RemoteInputChannel should be updated properly
## What is the purpose of the change
*In credit-based flow control mode, when the number of available buffers is
less than required buffers (backlog + initialCredit), the `RemoteInputChannel`
will request floating buffers from `BufferProvider`. If not get enough
available floating buffers, the `RemoteInputChannel` registers itself as
listener in `BufferProvider` and updates the tag `isWaitingForFloatingBuffers`
as true to avoid registration repeatedly.*
*When a floating buffer is recycled to `BufferProvider`, it will notify
the listener of available buffer. But the listener may not need floating
buffers currently if the available buffers is not less than required buffers,
then the floating buffers will be returned to BufferProvider directly. Most
importantly, the tag `isWaitingForFloatingBuffers` should also be updated as
false, otherwise the `RemoteInputChannel` will not request floating buffers any
more after the available buffers less than required buffers.*
*There are two scenarios for causing the above issue:*
- The recycled exclusive buffers increase the total available buffers which
is equal to or more than required buffers.
- The decreased sender's backlog resulting the available buffers equal to
required buffers.
## Brief change log
- *Updates the tag `isWaitingForFloatingBuffers` false on
`notifyBufferAvailable` if current available buffers is not less than required
buffers*
## Verifying this change
This change added tests and can be verified as follows:
- *Modify
`RemoteInputChannelTest#testAvailableBuffersLessThanRequiredBuffers to cover
this case*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zhijiangW/flink FLINK-8747
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5558.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5558
----
commit 10ad6db14d072a5208c45c4be08624ebd7e8ea13
Author: Zhijiang <wangzhijiang999@...>
Date: 2018-02-22T14:41:38Z
[FLINK-8747][bugfix] The tag of waiting for floating buffers in
RemoteInputChannel should be updated properly
----
> The tag of waiting for floating buffers in RemoteInputChannel should be
> updated properly
> ----------------------------------------------------------------------------------------
>
> Key: FLINK-8747
> URL: https://issues.apache.org/jira/browse/FLINK-8747
> Project: Flink
> Issue Type: Sub-task
> Components: Network
> Affects Versions: 1.5.0
> Reporter: zhijiang
> Assignee: zhijiang
> Priority: Minor
>
> In credit-based flow control mode, when the number of available buffers is
> less than required buffers (backlog + initialCredit), the
> {{RemoteInputChannel}} will request floating buffers from {{BufferProvider}}.
> If not get enough available floating buffers, the {{RemoteInputChannel}}
> registers itself as listener in {{BufferProvider}} and updates the tag
> {{isWaitingForFloatingBuffers}} as {{true}} to avoid registration repeatedly.
>
> When a floating buffer is recycled to {{BufferProvider}}, it will notify the
> listener of available buffer. But the listener may not need floating buffers
> currently if the available buffers is not less than required buffers, then
> the floating buffers will be returned to {{BufferProvider}} directly. Most
> importantly, the tag {{isWaitingForFloatingBuffers}} should also be updated
> as {{false, otherwise the RemoteInputChannel}} will not request floating
> buffers any more after the available buffers less than required buffers.
>
> There are two scenarios for causing the above issue:
> * The recycled exclusive buffers increase the total available buffers which
> is equal to or more than required buffers.
> * The decreased sender's backlog resulting the available buffers equal to
> required buffers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
