GitHub user mxm opened a pull request:
https://github.com/apache/flink/pull/896
[FLINK-2292][FLINK-1573] add live per-task accumulators
This refactors the accumulators to accumulate per task execution. The
accumulators are reported from the task managers periodically to the job
manager via the Heartbeat message. If the execution contains chained
tasks, the accumulators are chained as well. The final accumulator
results are reported via the UpdateTaskExecutionState message.
The accumulators are now saved in the Execution within the
ExecutionGraph. This makes the AccumulatorManager obsolete. It has been
removed for now. In the future, we might introduce some caching for the
web frontend visualization.
Two types of accumulators are available:
- external (user-defined via the RuntimeContext)
- internal (flink metrics defined in the invocables)
The internal (built-in) metrics are targeted at users who want to
monitor their programs, e.g. through the job manager's web frontend.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mxm/flink live-accumulators
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/896.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #896
----
commit 7cec1236f087e72b40022bf02a6dbb12d74acbac
Author: Maximilian Michels <[email protected]>
Date: 2015-07-08T07:23:42Z
[FLINK-2292][FLINK-1573] add live per-task accumulators
This refactors the accumulators to accumulate per task execution. The
accumulators are reported from the task managers periodically to the job
manager via the Heartbeat message. If the execution contains chained
tasks, the accumulators are chained as well. The final accumulator
results are reported via the UpdateTaskExecutionState message.
The accumulators are now saved in the Execution within the
ExecutionGraph. This makes the AccumulatorManager obsolete. It has been
removed for now. In the future, we might introduce some caching for the
web frontend visualization.
Two types of accumulators are available:
- external (user-defined via the RuntimeContext)
- internal (flink metrics defined in the invocables)
The internal (built-in) metrics are targeted at users who want to
monitor their programs, e.g. through the job manager's web frontend.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
