[
https://issues.apache.org/jira/browse/YARN-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387021#comment-16387021
]
Miklos Szegedi commented on YARN-5764:
--------------------------------------
Thank you, [~devaraj.k] for the updated patch.
{code:java}
3599 public static final String NM_NUMA_AWARENESS_NODE_MEMORY = NM_PREFIX
3600 + "numa-awareness.<NODE_ID>.memory";
3601 public static final String NM_NUMA_AWARENESS_NODE_CPUS = NM_PREFIX
3602 + "numa-awareness.<NODE_ID>.cpus";{code}
These two lines are no-op, they can probably be omitted.
{code:java}
yarn.nodemanager.numa-awareness.1.memory
{code}
Optional: Is there an example of a NUMA architecture of assymetric
architecture. It might make sense in the future to define nodes once and
specify a multiplier, so that we can make the configuration easier.
{code:java}
145 String[] args = new String[] {"numactl", "--hardware"};{code}
This should be {{/usr/bin/numactl}} for security reasons. In fact should not it
use the configured numactl path?
I think {{recoverCpus}} and {{recoverMemory}} can be eliminated. You could just
create a Resource object and use assignResources.
{code}
213 NumaResourceAllocation numaNode = allocate(containerId, resource);
{code}
This is a little bit misleading. Allocate may return multiple allocations on
multiple nodes not just a single numaNode.
I have a question. {{recoverNumaResource}} reallocates the resources based on
the registered values. Where are those resources released? It looks like
testRecoverNumaResource() does not test a container allocation, release and
then relaunch cycle but the opposite direction. What is the reason for that?
> NUMA awareness support for launching containers
> -----------------------------------------------
>
> Key: YARN-5764
> URL: https://issues.apache.org/jira/browse/YARN-5764
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, yarn
> Reporter: Olasoji
> Assignee: Devaraj K
> Priority: Major
> Attachments: NUMA Awareness for YARN Containers.pdf, NUMA Performance
> Results.pdf, YARN-5764-v0.patch, YARN-5764-v1.patch, YARN-5764-v2.patch,
> YARN-5764-v3.patch, YARN-5764-v4.patch, YARN-5764-v5.patch,
> YARN-5764-v6.patch, YARN-5764-v7.patch
>
>
> The purpose of this feature is to improve Hadoop performance by minimizing
> costly remote memory accesses on non SMP systems. Yarn containers, on launch,
> will be pinned to a specific NUMA node and all subsequent memory allocations
> will be served by the same node, reducing remote memory accesses. The current
> default behavior is to spread memory across all NUMA nodes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
