[
https://issues.apache.org/jira/browse/YARN-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated YARN-1025:
--------------------------------
Component/s: resourcemanager
Description: ResourceManager and NodeManager do not have the correct
setting for java.library.path when launched on Windows. This prevents the
processes from loading native code from hadoop.dll. The native code is
required for correct functioning on Windows (not optional), so this ultimately
can cause failures. (was: Neither the NodeManager process itself nor the child
container processes that it launches have the correct setting for
java.library.path on Windows. This prevents the processes from loading native
code from hadoop.dll. The native code is required for correct functioning on
Windows (not optional), so this ultimately can cause failures in MapReduce
jobs.)
Summary: ResourceManager and NodeManager do not load native libraries
on Windows. (was: NodeManager does not propagate java.library.path to launched
child containers on Windows)
Kihwal and Chuan, thanks for the comments here and in MAPREDUCE-5451. I think
the conclusion then is:
# YARN applications, including MR, are responsible for managing the library
path of their launched containers via configuration of LD_LIBRARY_PATH on Linux
or PATH on Windows.
# However, there is still a need for the YARN daemons themselves to launch with
a correct library path for finding libhadoop.so or hadoop.dll. The other
Hadoop daemons automatically find the library and set java.library.path, so
ResourceManager and NodeManager need to do the same. They already do this on
Linux in the yarn shell script, so we just need to update the cmd scripts with
equivalent logic.
I'm leaving this issue open for just the cmd script changes, but we won't make
changes for NodeManager to propagate java.library.path to its launched child
containers.
I've updated the issue summary and description accordingly.
> ResourceManager and NodeManager do not load native libraries on Windows.
> ------------------------------------------------------------------------
>
> Key: YARN-1025
> URL: https://issues.apache.org/jira/browse/YARN-1025
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager, resourcemanager
> Affects Versions: 3.0.0, 2.1.1-beta
> Reporter: Chris Nauroth
>
> ResourceManager and NodeManager do not have the correct setting for
> java.library.path when launched on Windows. This prevents the processes from
> loading native code from hadoop.dll. The native code is required for correct
> functioning on Windows (not optional), so this ultimately can cause failures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
