[
https://issues.apache.org/jira/browse/MAPREDUCE-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880017#action_12880017
]
Scott Chen commented on MAPREDUCE-220:
--------------------------------------
The failed contrib test is TestSimulatorDeterministicReplay.testMain.
It is a know issue in MAPREDUCE-1834.
In the patch we put task cumulative CPU time, current physical memory and
current virtual memory in task counters.
So it will be aggregated in JobInProgress.getJobCounter(). We will get the
total CPU time and current total memory usage.
They will go to both web ui and history as part of the counters.
We can access the task counters to obtain these information in place like task
scheduler too.
@Vinod: I didn't do the refactoring of the ProcTree. Because
LinuxResourceCalculatorPlugin is now called by the task (updateCounters is in
Task.java). It is in a different process than TaskTracker so we can not reuse
the ProcTree. But directory /proc/ is in memory, this may not be so bad in
terms of performance. What do you think?
> Collecting cpu and memory usage for MapReduce tasks
> ---------------------------------------------------
>
> Key: MAPREDUCE-220
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-220
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: task, tasktracker
> Reporter: Hong Tang
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-220-20100616.txt, MAPREDUCE-220-v1.txt,
> MAPREDUCE-220.txt
>
>
> It would be nice for TaskTracker to collect cpu and memory usage for
> individual Map or Reduce tasks over time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
