[
https://issues.apache.org/jira/browse/MAPREDUCE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799040#action_12799040
]
Scott Chen commented on MAPREDUCE-1218:
---------------------------------------
Thank you again for the review.
{quote}
In that case, should we instead set it to the jiffyLengthInMillis as that is
the smallest unit of measurement for cpu-time?
{quote}
If we use the jiffyLengthInMillis, in this period of time, the CPU time can
only move 1 jiffy or not for each CPU. So for each CPU, the CPU usage will be
0% or 100%. It is not very accurate. But using jiffyLengthInMillis as the unit
is definitely a good idea. I have changed the MINIMUM_UPDATE_INTERVAL to 10 *
jiffyLengthInMillis.
{quote}
To handle the deprecation of mapreduce.tasktracker.memorycalculatorplugin in
TaskTracker, for memory calculations we should first try to use the class
denoted by this configuration if present, otherwise only we should fall back to
the new resource-calculator. To facilitate this we will also retain a
deprecated TTConfig.TT_MEMORY_CALCULATOR_PLUGIN constant.
{quote}
I have adapted the psuedo code you've provided in TaskTracker.java. I have
found that there is a mistake in the previous patch. Previously I made
MemoryCalculatorPlugin extends ResourceCalculatorPlugin and
LinuxMemoryCalculatorPlugin extends LinuxResourceCalculatorPlugin. But
LinuxResourceCalculatorPlugin does not extend MemoryCalculatorPlugin. So
MemoryCalculatorPlugin.getMemoryCalculatorPlugin will fail because it will
actually returns LinuxMemoryCalculatorPlugin which does not extends
MemoryCalculatorPlugin. I have fixed the problem in this patch. The same
problem for DummyMemoryCalculatorPlugin as well. I have also fixed that. I have
run the deleted test TestTTMemoryReporting to be sure that they work correctly.
> Collecting cpu and memory usage for TaskTrackers
> ------------------------------------------------
>
> Key: MAPREDUCE-1218
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1218
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Environment: linux
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1218-rename.sh, MAPREDUCE-1218-v2.patch,
> MAPREDUCE-1218-v3.patch, MAPREDUCE-1218-v4.patch, MAPREDUCE-1218-v5.patch,
> MAPREDUCE-1218-v6.patch, MAPREDUCE-1218.patch
>
>
> The information can be used for resource aware scheduling.
> Note that this is related to MAPREDUCE-220. There the per task resource
> information is collected.
> This one collects the per machine information.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
