GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/6272
[FLINK-9755][network] forward exceptions in
RemoteInputChannel#notifyBufferAvailable() to the responsible thread
## What is the purpose of the change
Exceptions in `RemoteInputChannel#notifyBufferAvailable()`, e.g. state
checks, were swallowed inside `LocalBufferPool#recycle()` and neither logged
nor otherwise processed and may have lead to stalling processes waiting for a
notification that never comes.
Please note that this PR builds upon #6271 which also touched the unit
tests which we change here. @tillrohrmann, @zhijiangW can you have a look at
this cleanup of `LocalBufferPool#recycle()`?
## Brief change log
- cleanup of `LocalBufferPool#recycle()` also clarifying the contract of
`BufferListener#notifyBufferAvailable()` which should recycle the given buffer
(one implementation already did that; `RemoteInputChannel` did not)
- forward exceptions in `RemoteInputChannel#notifyBufferAvailable()` to the
responsible channel and recycle the given buffer in that case
## Verifying this change
This change added tests and can be verified as follows:
- added `RemoteInputChannelTest#testFailureInNotifyBufferAvailable()`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no**
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: **no**
- The serializers: **no**
- The runtime per-record code paths (performance sensitive): **no**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
- The S3 file system connector: **no**
## Documentation
- Does this pull request introduce a new feature? **no**
- If yes, how is the feature documented? **JavaDocs**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-9755
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6272.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6272
----
commit 0b623b66399915d43f29245da148fed63bf940bf
Author: Nico Kruber <nico@...>
Date: 2018-07-05T13:49:15Z
[FLINK-9766][network][tests] fix cleanup in RemoteInputChannelTest
If an assertion in the test fails and as a result the cleanup fails, in most
tests the original assertion was swallowed making it hard to debug.
Furthermore, #testConcurrentRecycleAndRelease2() does even not clean up at
all
if successful.
commit 38a26a829b047ce1e50794470f979e708b0bb81f
Author: Nico Kruber <nico@...>
Date: 2018-07-04T22:48:33Z
[FLINK-9755][network] forward exceptions in
RemoteInputChannel#notifyBufferAvailable() to the responsible thread
This mainly involves state checks but previously these have only been
swallowed
without re-registration or any other logging/handling. This may have lead to
some thread stalling while waiting for the notification that never came.
----
---
