[
https://issues.apache.org/jira/browse/HADOOP-4975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12723481#action_12723481
]
Amareshwari Sriramadasu commented on HADOOP-4975:
-------------------------------------------------
bq. I made the change to the Parser class, but I think the question is still
open about whether the change should go into the Configuration(Configuration)
constructor.
Yes. I figured out that the solution is to move the changes to
Configuration(Configuration), when I'm trying to change join package to use new
api. Raised HADOOP-6103 for the Configuration changes.
> CompositeRecordReader: ClassLoader set in JobConf is not passed onto
> WrappedRecordReaders
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-4975
> URL: https://issues.apache.org/jira/browse/HADOOP-4975
> Project: Hadoop Common
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Jingkei Ly
> Assignee: Jingkei Ly
> Fix For: 0.21.0
>
> Attachments: break-wrapped-rr-test.patch, HADOOP-4975-1.patch,
> HADOOP-4975-2.patch
>
>
> I am using a custom ClassLoader which I set in my JobConf via
> setClassLoader(). The ClassLoader is loaded key and value classes which are
> required to read records from SequenceFiles that were written out in a
> previous MapReduce job.
> However, I am getting a ClassNotFoundException when using the
> CompositeInputFormat to create a RecordReader to read these SequenceFiles
> from HDFS. It occurs when the SequenceFile.Reader tries to create an instance
> of the Key/Value classes, presumably because the class loader
> SequenceFile.Reader is using is not the one I set with
> JobConf.setClassLoader. Below is an example of the stack trace I get:
> {code}
> Caused by: java.io.IOException: WritableName can't load class
> at org.apache.hadoop.io.WritableName.getClass(WritableName.java:73)
> at
> org.apache.hadoop.io.SequenceFile$Reader.getKeyClass(SequenceFile.java:1596)
> ... 33 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.MyWritableClass
> at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673)
> at org.apache.hadoop.io.WritableName.getClass(WritableName.java:71)
> ... 34 more
> {code}
> I'll attach a unit test that can demonstrate this more clearly....
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
