[
https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16117264#comment-16117264
]
Haibo Chen commented on YARN-6668:
----------------------------------
Thanks [[email protected]] for the update! I have a few follow-up
knits:
1) The change in ResourceCalculatorProcessTree.java is no longer needed.
2) Can we add @Override annotation to CGroupsHandlerImpl.getControllerPath()?
3) The CGroupsResoureCalculator now takes precedence over user-configuration.
4) There's an unused import java.io.FileNotFoundException; in
CGroupsResourceCalculator
5) ContainersMonitorImpl is missing an import of YarnException, which leads to
compilation failure
6) performance is misspelled as "perforance" in
ContainersMonitorImpl.getResourceCalculatorProcessTree()
7) Now that we separate getCgroupHandlers from initCgroupHandlers in
ResourceHandlerModule, and calls getCgroupHandlers() in
CGroupsResourceCalculator to determine whether it is available. I think we have
an implicit assumption here that CGroup is initialized before
CGroupsResourceCalculator.isAvailable() is called. Can we add a comment in
isAvailable to make it more obvious?
8) In CGroupsResourceCalculator constructor, should we initialize jiffyLengthMs
first and then cpuTimeTracker to avoid the cases where cpuTimeTracker get -1 as
the jiffyLength whereas jiffyLengthMs is set to 10?
9) We call setCGroupFilePaths() in constructor which reads file. Can we move
that out into an init() method instead?
10) the lock around firstError is unnecessary if we make firstError volatile.
11) TestCGroupsResourceCalculator.testNoPid() is misnamed in that the
constructor actually takes a pid '1234'. Likewise, testNoCGgroup is testing
calculator.getRssMemorySize(0) returns UNAVAILABLE, right?
12) How about we reference '/tmp/' + this.getClass().getName() as a static
field profcBase? Optionally, we could then move all deletions of procfs into a
teardown method.
13) testNoMemoryCGgroupMount can use a ExpectedException to replace the
AssertTrue in the catch block.
> Use cgroup to get container resource utilization
> ------------------------------------------------
>
> Key: YARN-6668
> URL: https://issues.apache.org/jira/browse/YARN-6668
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Affects Versions: 3.0.0-alpha3
> Reporter: Haibo Chen
> Assignee: Miklos Szegedi
> Attachments: YARN-6668.000.patch, YARN-6668.001.patch,
> YARN-6668.002.patch
>
>
> Container Monitor relies on proc file system to get container resource
> utilization, which is not as efficient as reading cgroup accounting. We
> should in NM, when cgroup is enabled, read cgroup stats instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
