[
https://issues.apache.org/jira/browse/FLINK-9755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534518#comment-16534518
]
ASF GitHub Bot commented on FLINK-9755:
---------------------------------------
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/6272
[FLINK-9755][network] forward exceptions in
RemoteInputChannel#notifyBufferAvailable() to the responsible thread
## What is the purpose of the change
Exceptions in `RemoteInputChannel#notifyBufferAvailable()`, e.g. state
checks, were swallowed inside `LocalBufferPool#recycle()` and neither logged
nor otherwise processed and may have lead to stalling processes waiting for a
notification that never comes.
Please note that this PR builds upon #6271 which also touched the unit
tests which we change here. @tillrohrmann, @zhijiangW can you have a look at
this cleanup of `LocalBufferPool#recycle()`?
## Brief change log
- cleanup of `LocalBufferPool#recycle()` also clarifying the contract of
`BufferListener#notifyBufferAvailable()` which should recycle the given buffer
(one implementation already did that; `RemoteInputChannel` did not)
- forward exceptions in `RemoteInputChannel#notifyBufferAvailable()` to the
responsible channel and recycle the given buffer in that case
## Verifying this change
This change added tests and can be verified as follows:
- added `RemoteInputChannelTest#testFailureInNotifyBufferAvailable()`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no**
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: **no**
- The serializers: **no**
- The runtime per-record code paths (performance sensitive): **no**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
- The S3 file system connector: **no**
## Documentation
- Does this pull request introduce a new feature? **no**
- If yes, how is the feature documented? **JavaDocs**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-9755
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6272.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6272
----
commit 0b623b66399915d43f29245da148fed63bf940bf
Author: Nico Kruber <nico@...>
Date: 2018-07-05T13:49:15Z
[FLINK-9766][network][tests] fix cleanup in RemoteInputChannelTest
If an assertion in the test fails and as a result the cleanup fails, in most
tests the original assertion was swallowed making it hard to debug.
Furthermore, #testConcurrentRecycleAndRelease2() does even not clean up at
all
if successful.
commit 38a26a829b047ce1e50794470f979e708b0bb81f
Author: Nico Kruber <nico@...>
Date: 2018-07-04T22:48:33Z
[FLINK-9755][network] forward exceptions in
RemoteInputChannel#notifyBufferAvailable() to the responsible thread
This mainly involves state checks but previously these have only been
swallowed
without re-registration or any other logging/handling. This may have lead to
some thread stalling while waiting for the notification that never came.
----
> Exceptions in RemoteInputChannel#notifyBufferAvailable() are not propagated
> to the responsible thread
> -----------------------------------------------------------------------------------------------------
>
> Key: FLINK-9755
> URL: https://issues.apache.org/jira/browse/FLINK-9755
> Project: Flink
> Issue Type: Bug
> Components: Network
> Affects Versions: 1.5.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.5.2, 1.6.0
>
>
> The credit-based flow control implementation of
> RemoteInputChannel#notifyBufferAvailable() does not forward errors (like the
> {{IllegalStateException}}) to the thread that is being notified. The calling
> code at {{LocalBufferPool#recycle}}, however, relies on the callback
> forwarding errors and completely ignores any failures.
> Therefore, we could end up with a program waiting forever for the callback
> and not even a failure message in the logs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
