[
https://issues.apache.org/jira/browse/MAPREDUCE-7152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667258#comment-16667258
]
Peter Bacsko commented on MAPREDUCE-7152:
-----------------------------------------
[~jlowe] by looking at MRJobConfig.java, the default value of LD_LIBRARY_PATH
comes from DEFAULT_MAPRED_ADMIN_USER_ENV where it's already cross-platformified:
{noformat}
public static final String DEFAULT_MAPRED_ADMIN_USER_ENV =
Shell.WINDOWS ?
"PATH=%PATH%;%HADOOP_COMMON_HOME%\\bin" :
"LD_LIBRARY_PATH=" + Apps.crossPlatformify("HADOOP_COMMON_HOME") +
"/lib/native";{noformat}
I mistakenly thought that the expansion occurs inside the AM. Looks like it's
already working properly. Will chat with other devs and probably close this as
Won't Fix.
> LD_LIBRARY_PATH is always passed from MR AM to tasks
> ----------------------------------------------------
>
> Key: MAPREDUCE-7152
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7152
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
> Attachments: MAPREDUCE-7152-NMAdminEnvPOC_POC01.patch,
> MAPREDUCE-7152-lazyEval_POC01.patch
>
>
> {{LD_LIBRARY_PATH}} is set to {{$HADOOP_COMMON_HOME/lib/native}} by default
> in Hadoop (as part of {{mapreduce.admin.user.env}} and
> {{yarn.app.mapreduce.am.user.env}}), and passed as an environment variable
> from AM container to task containers in the container launch context.
> In cases where {{HADOOP_COMMON_HOME}} is different in AM node and task node,
> tasks will fail to load native library. A reliable way to fix this is to add
> {{LD_LIBRARY_PATH}} in {{yarn.nodemanager.admin-env}} instead.
> Another approach is to perform a lazy evaluation of {{LD_LIBRARY_PATH}} on
> the NM side.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
