When I run mean shift as a MR job, I find this:
java.lang.IllegalStateException: java.lang.ClassNotFoundException:
at
org.apache.mahout.clustering.DistanceMeasureCluster.readFields(DistanceMeasureCluster.java:59)
at
org.apache.mahout.clustering.kmeans.Cluster.readFields(Cluster.java:70) at
org.apache.mahout.clustering.meanshift.MeanShiftCanopy.readFields(MeanShiftCanopy.java:121)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at
org.apache.hadoop.io.SequenceFile$Reader.deserializeValue(SequenceFile.java:1832)
at
org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1816)
at
org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:74)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:455)
at
org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67) at
org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.ClassNotFoundException:
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.mahout.clustering.DistanceMeasureCluster.readFields(DistanceMeasureCluster.java:53)
As you can see, the exception is less than forthcoming with any diagnostic
information. The line in question is in this code in DistanceMeasureCluster:
@Override
public void readFields(DataInput in) throws IOException {
String dm = in.readUTF();
try {
ClassLoader ccl = Thread.currentThread().getContextClassLoader();
this.measure =
ccl.loadClass(dm).asSubclass(DistanceMeasure.class).newInstance();
} catch (InstantiationException e) {
throw new IllegalStateException(e);
} catch (IllegalAccessException e) {
throw new IllegalStateException(e);
} catch (ClassNotFoundException e) {
throw new IllegalStateException(e);
}
super.readFields(in);
}
I am using CosineDistance, and mahout-core and mahout-math are in my classpath
(along with mahout-collections and mahout-utils). Can you tell me why this
might occur?
Thanks.
