[
https://issues.apache.org/jira/browse/MAPREDUCE-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920442#action_12920442
]
luoli commented on MAPREDUCE-2026:
----------------------------------
hi Scott and Joydeep, what the patch does is really what we have thought,
moreover, the getCounters method will still consume some cpu time and it will
still synchronized the JIP, what if we add a call time interval of
getCounters? For example, call of getCounters return the result that got like
ten seconds ago, and if the call time from the last call exceed ten seconds we
recompute the counters? Or what if we launch a separate thread to do counters
compute? These are XiaoKang and I have discussed, what do you think?
> JobTracker.getJobCounters() should not hold JobTracker lock while calling
> JobInProgress.getCounters()
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2026
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2026
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Scott Chen
> Assignee: Joydeep Sen Sarma
> Fix For: 0.22.0
>
> Attachments: 2026.1.patch, MAPREDUCE-2026.txt
>
>
> JobTracker.getJobCounter() will lock JobTracker and call
> JobInProgress.getCounters().
> JobInProgress.getCounters() can be very expensive because it aggregates all
> the task counters.
> We found that from the JobTracker jstacks that this method is one of the
> bottleneck of the JobTracker performance.
> JobInProgress.getCounters() should be able to be called out side the
> JobTracker lock because it already has JobInProgress lock.
> For example, it is used by jobdetails.jsp without a JobTracker lock.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
