Im using a custom Writable in a proxy server feeding records to hadoop.mapred.RecordReaders. The proxy talks hadoop rpc with the record reader.
This works fine when I run with mapred.job.tracker="local", but I get a ClassNotFoundException when i run distributed: 282106-2006-08-17 15:02:34,609 INFO org.apache.hadoop.ipc.Client: Client connection to 127.0.0.1:19093 caught: java.lang.RuntimeException: readObject can't find class 282266-java.lang.RuntimeException: readObject can't find class 282322- at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:178) 282398- at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:60) 282473- at org.apache.hadoop.ipc.Client$Connection.run(Client.java:170) 282538:Caused by: java.lang.ClassNotFoundException: dk.statsbiblioteket.sbandex.mapred.WritableRecord 282633- at java.net.URLClassLoader$1.run(URLClassLoader.java:200) 282692- at java.security.AccessController.doPrivileged(Native Method) 282755- at java.net.URLClassLoader.findClass(URLClassLoader.java:188) 282818- at java.lang.ClassLoader.loadClass(ClassLoader.java:306) 282876- at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) 282942- at java.lang.ClassLoader.loadClass(ClassLoader.java:251) 283000- at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) 283066- at java.lang.Class.forName0(Native Method) 283110- at java.lang.Class.forName(Class.java:242) 283154- at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:174) 283230- ... 2 more Do I have to add the module containing dk.statsbiblioteket.sbandex.mapred.WritableRecord to Hadoops classpath? I would assume that "hadoop jar myPackage.jar" would import everything from myPackage.jar... Cheers, Mikkel
