[
https://issues.apache.org/jira/browse/PIG-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546642#comment-14546642
]
Remi Catherinot commented on PIG-4555:
--------------------------------------
note all computers have NUMA architectures and also -XX-like options are not
always the same for all JVM, version of JVM. I think adding -XX:+UseNUMA is
very specific to one users environnement and won't be supported and may even
crash lots of other people evironements where hadoop/tez is runned. This kind
of settings are more likely to be configurable and configured by the user to
fit its specific needs/environnement. I'm pretty sure there is already some
configuration variables one can set in its tez-site.xml file to set this option
so no need to have pig force this setting by code.
Can you explain exactly where and how you wanted to set/force the -XX:+UseNUMA ?
besides, UseNUMA should not correct OOM issues, it's more related to
memory-performance than to memory-availability so strange activating it
corrects an OOM issue. Seems to be accepted by oracle JVM starting from java
6u2.
> Add -XX:+UseNUMA for Tez jobs
> -----------------------------
>
> Key: PIG-4555
> URL: https://issues.apache.org/jira/browse/PIG-4555
> Project: Pig
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.15.0
>
>
> For very big Tez jobs (~50K tasks), AM quickly goes OOM without
> -XX:+UseNUMA. tez.am.launch.cmd-opts default setting has that, but since pig
> gives preference to yarn.app.mapreduce.am.command-opts if present (which
> usually it is), -XX:+UseNUMA is not there. Need to add -XX:+UseNUMA if we
> are picking up mapreduce setting.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
