[
https://issues.apache.org/jira/browse/MAPREDUCE-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788606#action_12788606
]
Vinod K V commented on MAPREDUCE-1218:
--------------------------------------
Quickly looked at the patch. Some comments:
- Your renaming script will not work while committing too, because other
classes still refer to them using the old names. You should generate a patch
after renaming the classes yourselves. Classes that need to be renamed to
reflect the word 'Resource' are: MemoryCalculator and all its sub-classes and
TTMemoryReporting
- Further, MemoryCalculatorPlugin.java and the its subclasses are all public.
So, you still should retain the old classes, deprecate them and redirect all
the old functionality to the new Resource* classes.
- TaskTracker.java: Creation of the plugin should be taken out of
initializeMemoryManagement() (+3471 through +3491) into initialize() method.
- TTConfig.java: TT_MEMORY_CALCULATOR_PLUGIN needs to be renamed too.
- TaskTrackerStatus.java: Why isn't cpu-usage serialized and de-serialized
directly? Calculating it at the time of de-serialization from cumulativeCpuTime
will yield wrong results, I think.
- LinuxMemoryCalculatorPlugin:
-- JIFFY_LENGTH calculation code is duplicated from MAPREDUCE-1201. Should
that be a blocker for this one?
-- In the main method, I didn't understand why we should sleep for 500ms.
What is the reason for doing this?
-- This class has a lot of parsing and calculation code. Though I could
verify them, it will be more helpful if we can write tests validating this. For
this, we can write dummy /proc files ourselves and call the calculator class's
methods similar to some of the {{TestProcfsBasedProcessTree}} tests, for e.g.,
testMemoryForProcessTree(). Can you add some similar tests here too, verifying
each of the APIs?
> Collecting cpu and memory usage for TaskTrackers
> ------------------------------------------------
>
> Key: MAPREDUCE-1218
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1218
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Affects Versions: 0.22.0
> Environment: linux
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1218-rename.sh, MAPREDUCE-1218-v2.patch,
> MAPREDUCE-1218.patch
>
>
> The information can be used for resource aware scheduling.
> Note that this is related to MAPREDUCE-220. There the per task resource
> information is collected.
> This one collects the per machine information.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
