[
https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621992#comment-14621992
]
ASF GitHub Bot commented on FLINK-2292:
---------------------------------------
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/896#issuecomment-120302065
> Looks like this change breaks the YARN integration. The YARN WordCount no
longer works.
Should be working again now.
> It would be good if the accumulator update interval was configurable.
> Edit: Is that the same value as the heartbeats?
Yes, that was a design rationale to keep the message count low. We could
only send the accumulators in every Nth heartbeat and let it be configurable.
> The is a potential modification conflict: Drawing a snapshot for
serialization and registering a new accumulator can lead to a
ConcurrentModificationException in the drawing of the snapshot.
I conducted tests with concurrent insertions and deletions and found that
only concurrent removals cause ConcurrentModificationExceptions. Removals are
not allowed for accumulators. Anyways, we could switch to a synchronized or
copy on write hash map. If we do I would opt for the latter.
> The naming of the accumulators refers sometimes to "flink vs.
user-defined", and sometimes to "internal vs. external". Can we make this
consistent? I actually like the "flink vs. user-defined" naming better.
Then let's stick to the "flink vs. user-defined" naming scheme.
> I think the code would be simpler is the registry simply always had a
created map for internal and external accumulators. Also, a "reporter" object
would help.
I agree that would be a nicer way of dealing with the API.
> Report accumulators periodically while job is running
> -----------------------------------------------------
>
> Key: FLINK-2292
> URL: https://issues.apache.org/jira/browse/FLINK-2292
> Project: Flink
> Issue Type: Sub-task
> Components: JobManager, TaskManager
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Fix For: 0.10
>
>
> Accumulators should be sent periodically, as part of the heartbeat that sends
> metrics. This allows them to be updated in real time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
