[
https://issues.apache.org/jira/browse/FLINK-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061970#comment-14061970
]
ASF GitHub Bot commented on FLINK-909:
--------------------------------------
GitHub user markus-h opened a pull request:
https://github.com/apache/incubator-flink/pull/69
[FLINK-909] Additional superstep barrier
This change introduces a new additional barrier at the end of a superstep
in iterations. It prevents the execution of an extra iteration at termination
and therby fixes[FLINK-909].
This pull request depends on [FLINK-951] and uses the old package names,
because [FLINK-951] was not merged yet.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/markus-h/incubator-flink
aggregatorsReworkToAccumulatorsRebase4_sync2-pr
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/69.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #69
----
commit 9da8aa6442861be17c7658c1dd2c2d9a6943d507
Author: Markus Holzemer <[email protected]>
Date: 2014-06-16T12:56:36Z
Iteration superstep synchronization through RPC and JobManager
Unification of Accumulators and Aggregators (removal of former Aggregators)
Adjusted Testcases accordingly
commit f41bf163203621b7dddd5a3073e654f40cb6143b
Author: Markus Holzemer <[email protected]>
Date: 2014-07-14T13:59:50Z
Additional superstep barrier for iterations to prevent execution of an
extra superstep at termination
----
> Pitfall due to additional superstep after the iteration has stopped
> -------------------------------------------------------------------
>
> Key: FLINK-909
> URL: https://issues.apache.org/jira/browse/FLINK-909
> Project: Flink
> Issue Type: Bug
> Reporter: GitHub Import
> Assignee: Markus Holzemer
> Labels: github-import
> Fix For: pre-apache
>
>
> Currently, after an iteration has exceeded the maximum number of iterations,
> all tasks are started again for an additional superstep during which they are
> stopped. This works if a tasks only waits for dynamic input. However, in the
> case where one has a task, e.g. a coGroup operation, which gets dynamic and
> static input the execution is not blocked. This can then lead to erroneous
> behaviour which the user is not aware of.
> I had this problem implementing ALS. Here one has a loop which gets as
> dynamic input matrix columns and as static input matrix entries. The columns
> and the entries are used to construct a matrix which represents a system of
> linear equations. If the set of columns are empty, then the matrix is
> singular and thus not solvable. During the additional superstep the task
> won't receive any columns but would still try to solve the now singular
> matrix.
> It would be good to finish the iteration without initiating this additional
> superstep.
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/909
> Created by: [tillrohrmann|https://github.com/tillrohrmann]
> Labels:
> Created at: Thu Jun 05 17:50:17 CEST 2014
> State: open
--
This message was sent by Atlassian JIRA
(v6.2#6252)
