[
https://issues.apache.org/jira/browse/YARN-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288029#comment-14288029
]
Varun Vasudev commented on YARN-160:
------------------------------------
bq. HADOOP_HEAPSIZE_MAX in trunk. HADOOP_HEAPSIZE was deprecated.
Thanks for pointing this out Allen. I'll provide a patch for trunk and one for
branch-2 when I address Vinod's comments.
bq. yarn.nodemanager.count-logical-processors-as-cores: Not sure of the use for
this. On Linux, shouldn't we simply use the the returned numCores if they are
valid? And fall-back to numProcessors?
Some people prefer to count hyperthreads as a CPU and some don't. This lets
users choose.
bq. yarn.nodemanager.enable-hardware-capability-detection: I think specifying
the capabilities to be -1 is already a way to trigger this automatic detection,
let's simply drop the flag and assume it to be true all the time?
Junping felt we should add it to cover upgrade scenarios. What do you think?
bq. We already have resource.percentage-physical-cpu-limit for CPUs -
YARN-2440. How about simply adding a resource.percentage-pmem-limit instead
making it a magic number in the code? Of course, we can have a default reserved
percentage.
I think resource.percentage-pmem-limit should be analogous to
resource.percentage-physical-cpu-limit in that it sets the limit as a
percentage of total memory. What about something like
"yarn.nodemanager.default-percentage-pmem-limit"?
> nodemanagers should obtain cpu/memory values from underlying OS
> ---------------------------------------------------------------
>
> Key: YARN-160
> URL: https://issues.apache.org/jira/browse/YARN-160
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.0.3-alpha
> Reporter: Alejandro Abdelnur
> Assignee: Varun Vasudev
> Fix For: 2.7.0
>
> Attachments: apache-yarn-160.0.patch, apache-yarn-160.1.patch,
> apache-yarn-160.2.patch, apache-yarn-160.3.patch
>
>
> As mentioned in YARN-2
> *NM memory and CPU configs*
> Currently these values are coming from the config of the NM, we should be
> able to obtain those values from the OS (ie, in the case of Linux from
> /proc/meminfo & /proc/cpuinfo). As this is highly OS dependent we should have
> an interface that obtains this information. In addition implementations of
> this interface should be able to specify a mem/cpu offset (amount of mem/cpu
> not to be avail as YARN resource), this would allow to reserve mem/cpu for
> the OS and other services outside of YARN containers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
