Markus Holzemer created FLINK-951:
-------------------------------------
Summary: Reworking of Iteration Synchronization, Accumulators and
Aggregators
Key: FLINK-951
URL: https://issues.apache.org/jira/browse/FLINK-951
Project: Flink
Issue Type: Improvement
Components: Compiler/Optimizer
Affects Versions: 0.6-incubating
Reporter: Markus Holzemer
Assignee: Markus Holzemer
I just realized that there is no real Jira issue for the task I am currently
working on.
I am currently reworking a few things regarding Iteration Synchronization,
Accumulators and Aggregators. Currently the synchronization at the end of one
superstep is done through channel events. That makes it hard to track the
current status of iterations. That is why I am changing this synchronization to
use RPC calls with the JobManager, so that the JobManager manages the current
status of all iterations.
Currently we use Accumulators outside of iterations and Aggregators inside of
iterations. Both have a similiar function, but a bit different interfaces and
handling. I want to unify these two concepts. I propose that we stick in the
future to Accumulators only. Aggregators therefore are removed and Accumulators
are extended to cover the usecases Aggregators were used fore before. The
switch to RPC for iterations makes it possible to also send the current
Accumulator values at the end of each superstep, so that the JobManager (and
thereby the webinterface) will be able to print intermediate accumulation
results.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
