Space: Apache Mahout (https://cwiki.apache.org/confluence/display/MAHOUT)
Page: Why I can't pass the test of ALS-WR with the moivelens dataset
(https://cwiki.apache.org/confluence/display/MAHOUT/2012/09/03/Why+I+can%27t+pass+the+test+of+ALS-WR+with+the+moivelens+dataset)
Added by Yuanshuai Sun:
---------------------------------------------------------------------
I deploy the mahout code in my machine according to the quickstart. But it
throws NullPointerException, just as follows
dm@cloud:~/workspace/mahout/examples/bin$ sh factorize-movielens-1M.sh
path/to/ratings.dat
creating work directory at /tmp/mahout-work-dm
Converting ratings...
Warning: $HADOOP_HOME is deprecated.
Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB:
/home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.
Exception in thread "main" java.lang.NullPointerException
at org.apache.mahout.common.AbstractJob.getOption(AbstractJob.java:385)
at
org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter.run(DatasetSplitter.java:75)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at
org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter.main(DatasetSplitter.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Warning: $HADOOP_HOME is deprecated.
Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB:
/home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.
12/09/03 14:45:21 INFO common.AbstractJob: Command line arguments:
{--alpha=[40], --endPhase=[2147483647], --implicitFeedback=[false],
--input=[/tmp/mahout-work-dm/dataset/trainingSet/], --lambda=[0.065],
--numFeatures=[20], --numIterations=[10],
--output=[/tmp/mahout-work-dm/als/out], --startPhase=[0],
--tempDir=[/tmp/mahout-work-dm/als/tmp]}
****/tmp/mahout-work-dm/dataset/trainingSet
12/09/03 14:45:22 INFO mapred.JobClient: Cleaning up the staging area
hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/dm/.staging/job_201209030824_0013
12/09/03 14:45:22 ERROR security.UserGroupInformation:
PriviledgedActionException as:dm
cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path
does not exist: /tmp/mahout-work-dm/dataset/trainingSet
Exception in thread "main"
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does
not exist: /tmp/mahout-work-dm/dataset/trainingSet
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at
org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob.run(ParallelALSFactorizationJob.java:137)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at
org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob.main(ParallelALSFactorizationJob.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Warning: $HADOOP_HOME is deprecated.
Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB:
/home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.
12/09/03 14:45:24 INFO common.AbstractJob: Command line arguments:
{--endPhase=[2147483647], --input=[/tmp/mahout-work-dm/dataset/probeSet/],
--itemFeatures=[/tmp/mahout-work-dm/als/out/M/],
--output=[/tmp/mahout-work-dm/als/rmse/], --startPhase=[0],
--tempDir=[/tmp/mahout-work-dm/als/tmp],
--userFeatures=[/tmp/mahout-work-dm/als/out/U/]}
****/tmp/mahout-work-dm/dataset/probeSet
12/09/03 14:45:25 INFO mapred.JobClient: Cleaning up the staging area
hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/dm/.staging/job_201209030824_0014
12/09/03 14:45:25 ERROR security.UserGroupInformation:
PriviledgedActionException as:dm
cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path
does not exist: /tmp/mahout-work-dm/dataset/probeSet
Exception in thread "main"
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does
not exist: /tmp/mahout-work-dm/dataset/probeSet
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at
org.apache.mahout.cf.taste.hadoop.als.FactorizationEvaluator.run(FactorizationEvaluator.java:92)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at
org.apache.mahout.cf.taste.hadoop.als.FactorizationEvaluator.main(FactorizationEvaluator.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Warning: $HADOOP_HOME is deprecated.
Running on hadoop, using /usr/local/hadoop//bin/hadoop and HADOOP_CONF_DIR=
MAHOUT-JOB:
/home/dm/workspace/mahout/examples/target/mahout-examples-0.8-SNAPSHOT-job.jar
Warning: $HADOOP_HOME is deprecated.
12/09/03 14:45:27 INFO common.AbstractJob: Command line arguments:
{--endPhase=[2147483647], --input=[/tmp/mahout-work-dm/als/out/userRatings/],
--itemFeatures=[/tmp/mahout-work-dm/als/out/M/], --maxRating=[5],
--numRecommendations=[6], --output=[/tmp/mahout-work-dm/recommendations/],
--startPhase=[0], --tempDir=[temp],
--userFeatures=[/tmp/mahout-work-dm/als/out/U/]}
****/tmp/mahout-work-dm/als/out/userRatings
12/09/03 14:45:28 INFO mapred.JobClient: Cleaning up the staging area
hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/dm/.staging/job_201209030824_0015
12/09/03 14:45:28 ERROR security.UserGroupInformation:
PriviledgedActionException as:dm
cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path
does not exist: /tmp/mahout-work-dm/als/out/userRatings
Exception in thread "main"
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does
not exist: /tmp/mahout-work-dm/als/out/userRatings
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)
at
org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:55)
at
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at
org.apache.mahout.cf.taste.hadoop.als.RecommenderJob.run(RecommenderJob.java:95)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at
org.apache.mahout.cf.taste.hadoop.als.RecommenderJob.main(RecommenderJob.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
-e
RMSE is:
cat: /tmp/mahout-work-dm/als/rmse/rmse.txt: 没有那个文件或目录
-e
-e
Sample recommendations:
shuf: /tmp/mahout-work-dm/recommendations/part-m-00000: 没有那个文件或目录
-e
Change your notification preferences:
https://cwiki.apache.org/confluence/users/viewnotifications.action
