Stephan Ewen created FLINK-1675:
-----------------------------------
Summary: Rework Accumulators
Key: FLINK-1675
URL: https://issues.apache.org/jira/browse/FLINK-1675
Project: Flink
Issue Type: Bug
Components: JobManager, TaskManager
Affects Versions: 0.9
Reporter: Stephan Ewen
Fix For: 0.9
The accumulators need an overhaul to address various issues:
1. User defined Accumulator classes crash the client, because it is not using
the user code classloader to decode the received message.
2. They should be attached to the ExecutionGraph, not the dedicated
AccumulatorManager. That makes them accessible also for archived execution
graphs.
3. Accumulators should be sent periodically, as part of the heart beat that
sends metrics. This allows them to be updated in real time
4. Accumulators should be stored fine grained (per executionvertex, or per
execution) and the final value should be on computed by merging all involved
ones. This allows users to access the per-subtask accumulators, which is often
interesting.
5. Accumulators should subsume the aggregators by allowing to be "versioned"
with a superstep. The versioned ones should be redistributed to the cluster
after each superstep.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
