[
https://issues.apache.org/jira/browse/HADOOP-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mateusz Berezecki updated HADOOP-4020:
--------------------------------------
Description:
There's a patch provided below.
It was created on the 0.17.0 version,
but with a little tweaking can be applied
to the 0.18.0 as well. It fixes a timing bug,
whereas a custom inputformat was loaded
from the jar file, but the inputformat was being
processed much earlier in time, resulting
in a class not found exception.
was:
There's a patch provided below.
It was created on the 0.17.0 version,
but with a little tweaking can be applied
to the 0.18.0 as well. It fixes a timing bug,
whereas a custom inputformat was loaded
from the jar file, but the inputformat was being
processed much earlier in time, resulting
in a class not found exception.
--- src/java/org/apache/hadoop/mapred/pipes/Submitter.java.orig 2008-05-15
09:20:16.000000000 +0200
+++ src/java/org/apache/hadoop/mapred/pipes/Submitter.java 2008-07-23
13:01:58.000000000 +0200
@@ -364,6 +364,14 @@
}
if (results.hasOption("-jar")) {
conf.setJar((String) results.getValue("-jar"));
+ // if they gave us a jar file, include it into the class path
+ String jarFile = conf.getJar();
+ if (jarFile != null) {
+ ClassLoader loader =
+ new URLClassLoader(new URL[]{ FileSystem.getLocal(conf).
+ pathToFile(new
Path(jarFile)).toURL()});
+ conf.setClassLoader(loader);
+ }
}
if (results.hasOption("-inputformat")) {
setIsJavaRecordReader(conf, true);
@@ -406,14 +414,6 @@
conf.set(keyValSplit[0], keyValSplit[1]);
}
}
- // if they gave us a jar file, include it into the class path
- String jarFile = conf.getJar();
- if (jarFile != null) {
- ClassLoader loader =
- new URLClassLoader(new URL[]{ FileSystem.getLocal(conf).
- pathToFile(new
Path(jarFile)).toURL()});
- conf.setClassLoader(loader);
- }
submitJob(conf);
} catch (OptionException oe) {
cli.printUsage();
> Hadoop Pipes do not load custom InputFormats at appropriate time, rendering
> them useless in certain scenarios
> -------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4020
> URL: https://issues.apache.org/jira/browse/HADOOP-4020
> Project: Hadoop Core
> Issue Type: Improvement
> Components: pipes
> Affects Versions: 0.16.4, 0.17.0, 0.17.1, 0.17.2
> Reporter: Mateusz Berezecki
>
> There's a patch provided below.
> It was created on the 0.17.0 version,
> but with a little tweaking can be applied
> to the 0.18.0 as well. It fixes a timing bug,
> whereas a custom inputformat was loaded
> from the jar file, but the inputformat was being
> processed much earlier in time, resulting
> in a class not found exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
