[
https://issues.apache.org/jira/browse/YARN-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819132#comment-15819132
]
Devaraj K commented on YARN-5764:
---------------------------------
bq. Do you have any benchmarks results that would illustrate the kind of
performance gains that could potentially be realised with this patch?
Thanks [~raviprak] for going through this. I will share the performance results
here.
Thanks [~sunilg] for the comments.
bq. if NM is taking the decision based on cores (NUMA cpus), it ll be more
container specific. Could we apply it more of application specific where few
apps containers only will be NUMA aware.
bq. Also I think such NUMA aware nodes could be controlled within a specific
nodelabel, I think it may yield better use cases for NUMA. So during NM init,
such awareness info could be passed to RM and it can be made as node attribute.
Such nodes could then be labelled together as well.
If we want to run an application only on NUMA aware nodes, we can group NUMA
aware nodes into a node-label and specify this node-label for the application.
I am wondering why do some applications don't want to run in NUMA if the NM
supports and getting some perf gain for making this as applications specific.
We can also include this as an attribute once the constraint node
labels(YARN-3409) feature gets in.
> NUMA awareness support for launching containers
> -----------------------------------------------
>
> Key: YARN-5764
> URL: https://issues.apache.org/jira/browse/YARN-5764
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, yarn
> Reporter: Olasoji
> Assignee: Devaraj K
> Attachments: NUMA Awareness for YARN Containers.pdf,
> YARN-5764-v0.patch, YARN-5764-v1.patch
>
>
> The purpose of this feature is to improve Hadoop performance by minimizing
> costly remote memory accesses on non SMP systems. Yarn containers, on launch,
> will be pinned to a specific NUMA node and all subsequent memory allocations
> will be served by the same node, reducing remote memory accesses. The current
> default behavior is to spread memory across all NUMA nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
