JobClient looking for classes for submitted job in the wrong place
------------------------------------------------------------------
Key: HADOOP-303
URL: http://issues.apache.org/jira/browse/HADOOP-303
Project: Hadoop
Type: Bug
Components: mapred
Versions: 0.3.2
Reporter: Benjamin Reed
Priority: Minor
JobClient does some checking of the job being submitted when it submits a jar
file along with the job. The problem is that the JobClient pulls classes from
the classpath rather than the submitted jar file. Because the jar file may
contain newer (or older) versions of classes on the classpath this behavior
leads to confusing errors when the job is run. It is also a pain to ensure that
the jar file being submitted is on the classpath. Further, if the JobClient
uses the submitted jar file rather than the classpath, missing classes from the
jar file can be detected earlier.
This patch will cause the JobClient to load the classes for a job from the jar
file rather than the classpath. Because of the class loading precedence rules
in Java, if the class is on the system class path, it will be loaded from there
rather than the submitted jar file, but now users need not (and should not) put
job classfiles on the system classpath.
This patch also allows config files to be put in a configuration directory
rather than on the classpath, which also eliminates some confusing behavior
when there are duplicate instances of config files in different parts of the
classpath.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
