[
https://issues.apache.org/jira/browse/HADOOP-2208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12554316
]
Owen O'Malley commented on HADOOP-2208:
---------------------------------------
I'm pretty worried about the approach of this patch. It takes it from always
sending the current values for the counters to just sending the ones that
changed. That doesn't seem like an optimization that is likely to be important.
Have you run large jobs that show this is important? My concern is that sending
the deltas makes the system very vulnerable to losing or duplicating a message.
My preference would be to have a boolean in the TaskStatus whether it should be
sending the counters or not, but always send the current values of all counters.
I'd also recommend against the current sendCounters and doSendCounters. I think
your original names were better: {get,set}SendCounters. Maybe they should be
something like: {get,set}IncludeCounters...
> Reduce frequency of Counter updates in the task tracker status
> --------------------------------------------------------------
>
> Key: HADOOP-2208
> URL: https://issues.apache.org/jira/browse/HADOOP-2208
> Project: Hadoop
> Issue Type: Improvement
> Reporter: Amareshwari Sri Ramadasu
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.16.0
>
> Attachments: patch-2208.txt, patch-2208.txt, patch-2208.txt,
> patch-2208.txt
>
>
> Currently, We have counter updates from task tracker to job tracker on every
> heartbeat. Both counter name and the values are updated for every heartbeat.
> This can be improved by sending names and values for the first time and only
> the values after that.
> The frequency can be reduced by doing update only when the counters got
> changed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
