[
https://issues.apache.org/jira/browse/YARN-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109801#comment-14109801
]
Jason Lowe commented on YARN-2440:
----------------------------------
Thanks for updating the patch, Varun.
I don't see why we need both a containers-cpu-cores and
containers-cpu-percentage, and I think it leads to confusion when both exist.
At first I did not realize that one overrode the other. Instead I assumed that
if you set cpu-cores to X and cpu-percentage to Y then you were requesting Y%
of X cores. Then there's the additional question of whether container usage is
pinned to those cores, etc. Only having cpu-percentage is a simpler model that
still allows the user to specify cores indirectly (e.g.: 25% of an 8 core
system is 2 cores). Maybe I'm missing the use case where we really need
containers-cpu-cores and the confusing (to me at least) override behavior
between the two properties.
Other comments on the patch:
- I'm not thrilled about the name template "containers-cpu-*" since it could
easily be misinterpreted as a per-container thing as well, but I'm currently at
a loss for a better prefix. Suggestions welcome.
- Does getOverallLimits need to check for a quotaUS that's too low as well?
- I think minimally we need to log a warning if we're going to ignore setting
up cgroups to limit CPU usage across all containers if the user specified to do
so.
- Related to the previous comment, I think it would be nice if we didn't try to
setup any limits if none were specified. That way if there's some issue with
correctly determining the number of cores on a particular system it can still
work in the default, "use everything" scenario.
- NodeManagerHardwareUtils.getContainerCores should be getContainersCores (the
per-container vs. all-containers confusion again)
> Cgroups should allow YARN containers to be limited to allocated cores
> ---------------------------------------------------------------------
>
> Key: YARN-2440
> URL: https://issues.apache.org/jira/browse/YARN-2440
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Varun Vasudev
> Assignee: Varun Vasudev
> Attachments: apache-yarn-2440.0.patch, apache-yarn-2440.1.patch,
> apache-yarn-2440.2.patch, screenshot-current-implementation.jpg
>
>
> The current cgroups implementation does not limit YARN containers to the
> cores allocated in yarn-site.xml.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
