RowSimilarityJob error
-----------------------
Key: MAHOUT-450
URL: https://issues.apache.org/jira/browse/MAHOUT-450
Project: Mahout
Issue Type: Test
Components: Collaborative Filtering
Affects Versions: 0.4
Reporter: Hui Wen Han
Fix For: 0.4
when ran RowSimilarityJob job as following:
hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar
org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input
/steer/item/tap/temp/maybePruneItemUserMatrixPath --output
/steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
got following error:
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:31 INFO mapred.JobClient: Task Id :
attempt_201007261131_0026_r_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:44 INFO mapred.JobClient: Task Id :
attempt_201007261131_0026_r_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
It seemed that here error:
public static class SimilarityReducer
extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey,
MatrixEntryWritable> {
private DistributedVectorSimilarity similarity;
private int numberOfColumns;
@Override
protected void setup(Context ctx) throws IOException, InterruptedException {
super.setup(ctx);
similarity =
instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME));
//this line,seems can not load the class.
numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
if (numberOfColumns < 1) {
throw new IllegalStateException("Number of columns was not correctly
set!");
}
}
I guess that maybe be wrong here:
in SimilarityType:
SIMILARITY_COOCCURRENCE(DistributedCooccurrenceVectorSimilarity.class), //just
the class name is needed,has no the ".class"
SIMILARITY_EUCLIDEAN_DISTANCE(DistributedEuclideanDistanceVectorSimilarity.class),
SIMILARITY_LOGLIKELIHOOD(DistributedLoglikelihoodVectorSimilarity.class),
SIMILARITY_PEARSON_CORRELATION(DistributedPearsonCorrelationVectorSimilarity.class),
SIMILARITY_TANIMOTO_COEFFICIENT(DistributedTanimotoCoefficientVectorSimilarity.class),
SIMILARITY_UNCENTERED_COSINE(DistributedUncenteredCosineVectorSimilarity.class),
SIMILARITY_UNCENTERED_ZERO_ASSUMING_COSINE(DistributedUncenteredZeroAssumingCosineVectorSimilarity.class);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
