[
https://issues.apache.org/jira/browse/FLINK-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14396350#comment-14396350
]
ASF GitHub Bot commented on FLINK-951:
--------------------------------------
Github user uce commented on the pull request:
https://github.com/apache/flink/pull/570#issuecomment-89823498
Nice to see that you picked this up again. :-) I know from experience that
it can be tricky to port the old RPC calls to Akka msgs, so kudos. ;-)
- I actually agree with your reformatings, but we had a discussion recently
(http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Issues-with-heterogeneity-of-the-code-td4292.html#a4443)
to refrain from reformattings. I think it makes it harder to review your
changes. We haven't really written this down somewhere (this is actually a good
reminder that we need to do this), but you should keep it in mind.
- I only had a quick look at the hack you pointed out. I think we should
*not* merge it in the current state (in any case I would vote to postpone
merging this until *after* the upcoming milestone release). Can you provide
more information about what kind of Exception is thrown? Waiting for 10 ms is
not robust against different timings on different machines.
I think the blocking Await is OK in this case. We should add tests for the
basic JM <-> IterationHeadTask interaction though.
> Reworking of Iteration Synchronization, Accumulators and Aggregators
> --------------------------------------------------------------------
>
> Key: FLINK-951
> URL: https://issues.apache.org/jira/browse/FLINK-951
> Project: Flink
> Issue Type: Improvement
> Components: Iterations, Optimizer
> Affects Versions: 0.9
> Reporter: Markus Holzemer
> Assignee: Markus Holzemer
> Labels: refactoring
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> I just realized that there is no real Jira issue for the task I am currently
> working on.
> I am currently reworking a few things regarding Iteration Synchronization,
> Accumulators and Aggregators. Currently the synchronization at the end of one
> superstep is done through channel events. That makes it hard to track the
> current status of iterations. That is why I am changing this synchronization
> to use RPC calls with the JobManager, so that the JobManager manages the
> current status of all iterations.
> Currently we use Accumulators outside of iterations and Aggregators inside of
> iterations. Both have a similiar function, but a bit different interfaces and
> handling. I want to unify these two concepts. I propose that we stick in the
> future to Accumulators only. Aggregators therefore are removed and
> Accumulators are extended to cover the usecases Aggregators were used fore
> before. The switch to RPC for iterations makes it possible to also send the
> current Accumulator values at the end of each superstep, so that the
> JobManager (and thereby the webinterface) will be able to print intermediate
> accumulation results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
