GitHub user uce opened a pull request:
https://github.com/apache/flink/pull/624
[FLINK-1930] Separate output buffer pool and result partition life cycle
The problem: when a pipelined result is only consumed partially, the buffer
pool associated with the result partition will be destroyed too early. If there
is a pipelined producer online, which is still producing data for this
partition, it will run into an IllegalStateException.
The solution: by separating the life-cycle of the result partition and the
associated buffer pool this cannot happen anymore. The result buffer pool is
only destroyed after the producing task is finished, which is independent of
the state of the result partition.
Furthermore, this commit squashes the following commits:
- [FLINK-1930] [tests] Add test for FLINK-1930
- [tests] Move iterative tests to correct package
Thanks to @vasia for reporting the original issue and the program and data
to reproduce the problem.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/uce/incubator-flink tooearly1930
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/624.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #624
----
commit 6b7f14471ef34a18d284a42bfe5fc53bbf756585
Author: Ufuk Celebi <[email protected]>
Date: 2015-04-24T14:58:27Z
[FLINK-1930] Separate output buffer pool and result partition life cycle
The problem: when a pipelined result is only consumed partially, the buffer
pool
associated with the result partition will be destroyed too early. If there
is a
pipelined producer online, which is still producing data for this
partition, it
will run into an IllegalStateException.
The solution: by separating the life-cycle of the result partition and the
associated buffer pool this cannot happen anymore. The result buffer pool is
only destroyed after the producing task is finished, which is independent of
the state of the result partition.
Furthermore, this commit squashes the following commits:
- [FLINK-1930] [tests] Add test for FLINK-1930
- [tests] Move iterative tests to correct package
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
