Hey, folks, I've written my own custom sequence file loader, and I'm wondering how to use it in a hadoop job.
According to this: http://www.mail-archive.com/[email protected]/msg01762.html I should just be able to put it in the mahout-0.4/lib directory, right? I did this and modified bin/mahout to print the classpath it runs mahout with to confirm that the JAR I added was on the classpath (it is). I'm still getting a CNFE exception when trying to run it as a hadoop job. Running locally is fine, as setting CLASSPATH=urlclustering.jar when running bin/mahout works like a charm. 10/11/08 21:09:39 WARN driver.MahoutDriver: Unable to add class: com.mspitz.urlclustering.BagsToSequenceFile java.lang.ClassNotFoundException: com.mspitz.urlclustering.BagsToSequenceFile at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:169) at org.apache.mahout.driver.MahoutDriver.addClass(MahoutDriver.java:207) at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:124) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:186) Any thoughts? Thanks, Matt
