[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493913#comment-15493913
]
Bibin A Chundatt commented on MAPREDUCE-6779:
---------------------------------------------
[~sjlee0]
>From ClassLoader file is removed as per new implementation.Since the jar is
>not available in applcation class path all map execution will fail.
{noformat}
- classPath.add(file.toURI().toURL());
{noformat}
When {{JobConf#setJarByClass}} is set tries to get the jar name from classloader
{noformat}
public void setJarByClass(Class cls) {
String jar = ClassUtil.findContainingJar(cls);
if (jar != null) {
setJar(jar);
}
}
{noformat}
{{ClassUtil#findContainingJar}}
{noformat}
for(final Enumeration<URL> itr = loader.getResources(classFile);
itr.hasMoreElements();) {
final URL url = itr.nextElement();
if ("jar".equals(url.getProtocol())) {
String toReturn = url.getPath();
if (toReturn.startsWith("file:")) {
toReturn = toReturn.substring("file:".length());
}
toReturn = URLDecoder.decode(toReturn, "UTF-8");
return toReturn.replaceAll("!.*$", "");
}
}
{noformat}
which returns null. The job jar will not get loaded to staging dir in
{{JobResourceUploader#uploadResources}} since jobjar is {{null}}.
{noformat}
private void uploadJobJar(Job job, String jobJar, Path submitJobDir,
short submitReplication) throws IOException {
if (jobJar != null) {
{noformat}
> Mapreduce job failure on submission
> -----------------------------------
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Bibin A Chundatt
> Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_000001_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
