[
https://issues.apache.org/jira/browse/YARN-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728957#comment-13728957
]
Chris Nauroth commented on YARN-1025:
-------------------------------------
As a workaround, you can set the PATH environment variable to point to the
directory containing hadoop.dll before launching NodeManager. The environment
variable will propagate to child container processes, so this works.
I have a question on how this is expected to work on Linux. I see code in the
{{yarn}} shell script and {{LinuxContainerExecutor}} that propagates
java.library.path to children, but I also see checks in
{{YARNRunner#warnForJavaLibPath}} that warn if a caller is attempting to
control java.library.path and state that the caller should set LD_LIBRARY_PATH
instead. What is the expected way for this to work on Linux?
If the expectation is for the caller to set LD_LIBRARY_PATH, then this issue is
likely a Won't Fix. Setting PATH on Windows is roughly equivalent to setting
LD_LIBRARY_PATH on Linux. I'm already using this technique successfully as a
workaround.
If the expectation is that the shell scripts set up java.library.path, and
NodeManager propagates that to its children, then we do have a bug on Windows
and need to make some code changes. The first part is changing {{yarn.cmd}} to
set java.library.path. That's a small easy change that gets NodeManager to
load hadoop.dll. To cover the child processes, we'll need additional code
changes, probably in {{DefaultContainerExecutor}}.
> NodeManager does not propagate java.library.path to launched child containers
> on Windows
> ----------------------------------------------------------------------------------------
>
> Key: YARN-1025
> URL: https://issues.apache.org/jira/browse/YARN-1025
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 3.0.0, 2.1.1-beta
> Reporter: Chris Nauroth
>
> Neither the NodeManager process itself nor the child container processes that
> it launches have the correct setting for java.library.path on Windows. This
> prevents the processes from loading native code from hadoop.dll. The native
> code is required for correct functioning on Windows (not optional), so this
> ultimately can cause failures in MapReduce jobs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
