[
https://issues.apache.org/jira/browse/MAPREDUCE-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839114#action_12839114
]
Aaron Kimball commented on MAPREDUCE-1304:
------------------------------------------
You're right that if a task throws an exception, I think that it will probably
not set the counter. I could put the increment in a finally block and that
would fix that issue.
Based on a quick look at the code in Counters and Counter, I think that there
wouldn't be major issues with thread safety or performance (every operation on
a Counter is already synchronized). An extra thread is pretty heavy-weight,
though. I think something like this should actually go into the TaskReporter
itself; it could just increment the gc counter itself right before sending a
status every 3 seconds.
I ran a quick benchmark test which got a handle to the GarbageCollectorMXBean
list and polled them; I put all of this in a loop and ran it a million times in
about a half-second, so I don't think this would negatively impact performance.
> Add counters for task time spent in GC
> --------------------------------------
>
> Key: MAPREDUCE-1304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1304
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: task
> Reporter: Todd Lipcon
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1304.patch
>
>
> It's easy to grab the number of millis spent in GC (see JvmMetrics for
> example). Exposing these as task counters would be handy - occasionally I've
> seen user jobs where long GC pauses cause big "unexplainable" performance
> problems, and a large counter would make it obvious to the user what's going
> on.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
