[
https://issues.apache.org/jira/browse/MAPREDUCE-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189224#comment-13189224
]
Todd Lipcon commented on MAPREDUCE-3693:
----------------------------------------
Hey Arun. I agree with you in principle here... but the use case is still valid
and needs to be addressed -- users shouldn't have to set up their individual
configurations to point to wherever the compression libraries are on the
server. We need some way of setting up the environment here in a sensible way -
same as how users "automatically" get HDFS on their classpath via the NM, for
example.
I haven't looked into the confs, etc, yet, so apologies if I'm missing the
obvious right answer.
> mapreduce's MapReduceChildJVM no longer proxies value of java.library.path
> ---------------------------------------------------------------------------
>
> Key: MAPREDUCE-3693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Roman Shaposhnik
> Fix For: 0.23.1
>
>
> I have noticed that org.apache.hadoop.mapred.MapReduceChildJVM doesn't
> forward the value of -Djava.library.path= from the parent JVM to the child
> JVM. Thus if one wants to use native libraries for compression the only
> option seems to be to manually include relevant java.library.path settings
> into the mapred-site.xml (as mapred.[map|reduce].child.java.opts).
> This seems to be a change in behavior compared to MR1 where TaskRunner.java
> used to do that:
> {noformat}
> String libraryPath = System.getProperty("java.library.path");
> if (libraryPath == null) {
> libraryPath = workDir.getAbsolutePath();
> } else {
> libraryPath += SYSTEM_PATH_SEPARATOR + workDir;
> }
> boolean hasUserLDPath = false;
> for(int i=0; i<javaOptsSplit.length ;i++) {
> if(javaOptsSplit[i].startsWith("-Djava.library.path=")) {
> javaOptsSplit[i] += SYSTEM_PATH_SEPARATOR + libraryPath;
> hasUserLDPath = true;
> break;
> }
> }
> if(!hasUserLDPath) {
> vargs.add("-Djava.library.path=" + libraryPath);
> }
> for (int i = 0; i < javaOptsSplit.length; i++) {
> vargs.add(javaOptsSplit[i]);
> }
> {noformat}
> Is this a regression or a deliberate choice?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
