mapreduce's MapReduceChildJVM no longer proxies value of java.library.path
---------------------------------------------------------------------------
Key: MAPREDUCE-3693
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3693
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Roman Shaposhnik
Fix For: 0.23.1
I have noticed that org.apache.hadoop.mapred.MapReduceChildJVM doesn't forward
the value of -Djava.library.path= from the parent JVM to the child JVM. Thus if
one wants to use native libraries for compression the only option seems to be
to manually include relevant java.library.path settings into the
mapred-site.xml (as mapred.[map|reduce].child.java.opts).
This seems to be a change in behavior compared to MR1 where TaskRunner.java
used to do that:
{noformat}
String libraryPath = System.getProperty("java.library.path");
if (libraryPath == null) {
libraryPath = workDir.getAbsolutePath();
} else {
libraryPath += SYSTEM_PATH_SEPARATOR + workDir;
}
boolean hasUserLDPath = false;
for(int i=0; i<javaOptsSplit.length ;i++) {
if(javaOptsSplit[i].startsWith("-Djava.library.path=")) {
javaOptsSplit[i] += SYSTEM_PATH_SEPARATOR + libraryPath;
hasUserLDPath = true;
break;
}
}
if(!hasUserLDPath) {
vargs.add("-Djava.library.path=" + libraryPath);
}
for (int i = 0; i < javaOptsSplit.length; i++) {
vargs.add(javaOptsSplit[i]);
}
{noformat}
Is this a regression or a deliberate choice?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
