[
https://issues.apache.org/jira/browse/YARN-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962766#comment-15962766
]
Rajesh Balamohan commented on YARN-5764:
----------------------------------------
[~devaraj.k] Thank you for sharing the patch and the results. Recent JVM
versions have {{-XX:useNUMA}} (java -XX:+PrintFlagsFinal | grep useNUMA).
Enabling this would instruct JVM to be NUMA aware and GC can take advantage of
this fact.
Was this flag ({{-XX:useNUMA}}) enabled in the tasks when running the
benchmark?
Hive on MR is outdated, network intensive and slow. It would be great, if BB
benchmark can be run with Hive on Tez which optimizes queries to a great
extent. It has much better resource utilization and also elimiates a lot of IO
barriers and would be a lot efficient than MR codebase.
> NUMA awareness support for launching containers
> -----------------------------------------------
>
> Key: YARN-5764
> URL: https://issues.apache.org/jira/browse/YARN-5764
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, yarn
> Reporter: Olasoji
> Assignee: Devaraj K
> Attachments: NUMA Awareness for YARN Containers.pdf, NUMA Performance
> Results.pdf, YARN-5764-v0.patch, YARN-5764-v1.patch, YARN-5764-v2.patch
>
>
> The purpose of this feature is to improve Hadoop performance by minimizing
> costly remote memory accesses on non SMP systems. Yarn containers, on launch,
> will be pinned to a specific NUMA node and all subsequent memory allocations
> will be served by the same node, reducing remote memory accesses. The current
> default behavior is to spread memory across all NUMA nodes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
