[
https://issues.apache.org/jira/browse/YARN-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393910#comment-16393910
]
Wangda Tan commented on YARN-5764:
----------------------------------
[~devaraj.k]
Just took a very quick look at overall integration to other NM components, some
comments:
1) The latest patch doesn't persistent assigned NUMA resources, you can take a
look at GpuResourceAllocator as an example:
{code}
// Update state store.
nmContext.getNMStateStore().storeAssignedResources(container, GPU_URI,
new ArrayList<>(assignedGpus));
{code}
2) Is it better to move all NUMA related works to
{{org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.numa}}?
Which is consistent to other plugins such as GPU/FPGA. The scheduler package
is for new container allocation. (Just like scheduler module in RM).
> NUMA awareness support for launching containers
> -----------------------------------------------
>
> Key: YARN-5764
> URL: https://issues.apache.org/jira/browse/YARN-5764
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, yarn
> Reporter: Olasoji
> Assignee: Devaraj K
> Priority: Major
> Attachments: NUMA Awareness for YARN Containers.pdf, NUMA Performance
> Results.pdf, YARN-5764-v0.patch, YARN-5764-v1.patch, YARN-5764-v2.patch,
> YARN-5764-v3.patch, YARN-5764-v4.patch, YARN-5764-v5.patch,
> YARN-5764-v6.patch, YARN-5764-v7.patch, YARN-5764-v8.patch, YARN-5764-v9.patch
>
>
> The purpose of this feature is to improve Hadoop performance by minimizing
> costly remote memory accesses on non SMP systems. Yarn containers, on launch,
> will be pinned to a specific NUMA node and all subsequent memory allocations
> will be served by the same node, reducing remote memory accesses. The current
> default behavior is to spread memory across all NUMA nodes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
