[
https://issues.apache.org/jira/browse/MAPREDUCE-5655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Pados updated MAPREDUCE-5655:
------------------------------------
Attachment: YARNRunner.patch
MRApps.patch
These 2 patches intend to fix different jobLaunch/jobExecution OS problems with
MapReduce. It also reqires to add the following property to
mapred-site.xml (or mapred-default.xml):
<property>
<name>mapred.remote.os</name>
<value>Linux</value>
<description>Remote MapReduce framework's OS, can be either Linux or
Windows</description>
</property
This is used when job launched from windows, executed in Linux.
> Remote job submit from windows to a linux hadoop cluster fails due to wrong
> classpath
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5655
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5655
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, job submission
> Affects Versions: 2.2.0
> Environment: Client machine is a Windows 7 box, with Eclipse
> Remote: there is a multi node hadoop cluster, installed on Ubuntu boxes (any
> linux)
> Reporter: Attila Pados
> Attachments: MRApps.patch, YARNRunner.patch
>
>
> I was trying to run a java class on my client, windows 7 developer
> environment, which submits a job to the remote Hadoop cluster, initiates a
> mapreduce there, and then downloads the results back to the local machine.
> General use case is to use hadoop services from a web application installed
> on a non-cluster computer, or as part of a developer environment.
> The problem was, that the ApplicationMaster's startup shell script
> (launch_container.sh) was generated with wrong CLASSPATH entry. Together with
> the java process call on the bottom of the file, these entries were generated
> in windows style, using % as shell variable marker and ; as the CLASSPATH
> delimiter.
> I tracked down the root cause, and found that the MrApps.java, and the
> YarnRunner.java classes create these entries, and is passed forward to the
> ApplicationMaster, assuming that the OS that runs these classes will match
> the one running the ApplicationMaster. But it's not the case, these are in 2
> different jvm, and also the OS can be different, the strings are generated
> based on the client/submitter side's OS.
> I made some workaround changes to these 2 files, so i could launch my job,
> however there may be more problems ahead.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
