Are you constructing the JobConf with a Class? And is your class file in a directory on the classpath (rather than the Jar file)? If it is, your jar file will NOT be submitted with the job. (I opened a bug on this a while ago.)
The easiest way to make sure your jar gets sent is to use the setJar() method on JobConf. ben Mikkel Kamstrup Erlandsen wrote: > Im using a custom Writable in a proxy server feeding records to > hadoop.mapred.RecordReaders. The proxy talks hadoop rpc with the record > reader. > > This works fine when I run with mapred.job.tracker="local", but I get a > ClassNotFoundException when i run distributed: > > 282106-2006-08-17 15:02:34,609 INFO org.apache.hadoop.ipc.Client: Client > connection to 127.0.0.1:19093 caught: java.lang.RuntimeException: > readObject can't find class > 282266-java.lang.RuntimeException: readObject can't find class > 282322- at > org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:178) > 282398- at > org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:60) > 282473- at org.apache.hadoop.ipc.Client$Connection.run(Client.java:170) > 282538:Caused by: java.lang.ClassNotFoundException: > dk.statsbiblioteket.sbandex.mapred.WritableRecord > 282633- at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > 282692- at java.security.AccessController.doPrivileged(Native Method) > 282755- at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > 282818- at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > 282876- at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) > 282942- at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > 283000- at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) > 283066- at java.lang.Class.forName0(Native Method) > 283110- at java.lang.Class.forName(Class.java:242) > 283154- at > org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:174) > 283230- ... 2 more > > > Do I have to add the module containing > dk.statsbiblioteket.sbandex.mapred.WritableRecord to Hadoops classpath? > I would assume that "hadoop jar myPackage.jar" would import everything > from myPackage.jar... > > Cheers, > Mikkel > >
