Hello all I am a master student who is willing to implement a scalable recommender system using Mahout, hadoop and spark as my dissertation. Furthermore, I configured Hadoop in a pseudo-distributed mode on my Macbook Pro and install eclipse with m2eclipse plugin to assist me in implementing mahout based projects.
I’ve added Mahout0.9 and Hadoop 2.4.0 dependencies to my maven project in eclipse and things were OK when I tried to execute it on my machine as a single node (in non-distributed mode). However, as I tried to execute it on hadoop server using: Bin/hadoop jar itemBasedRecommender.jar org.apache.mahut.cf.taste.hadoop.itemRecommenderJob –input input/links- converted.txt –output output –n 15 –s SIMILARITY_LOGLIKELIHOOD – booleanData TRUE I got an error message saying: Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.mahout.common.HadoopUtil.getCustomJobName (HadoopUtil.java:174) at org.apache.mahout.common.AbstractJob.prepareJob(AbstractJob.java:614) at org.apache.mahout.cf.taste.hadoop.preparation.PreparePreferenceMatrixJob.ru n(PreparePreferenceMatrixJob.java:73) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run (RecommenderJob.java:164) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main (RecommenderJob.java:322) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) I search the web for a solution and I got to know that there is a compatibility problem with mahout 0.9 and hadoop 2.4.0. Some were saying that I have to download patches 1329, apply them and then re-compile mahout. I tried that but patches failed as well! Khadijas-MacBook-Pro:mahout-mahout-0.9 khadijaalmohsen$ patch -i 1329- 3.patch patching file pom.xml Hunk #1 FAILED at 213. Hunk #2 FAILED at 227. 2 out of 2 hunks FAILED -- saving rejects to file pom.xml.rej patching file pom.xml Hunk #1 FAILED at 35. Hunk #2 FAILED at 144. Hunk #3 FAILED at 195. 3 out of 3 hunks FAILED -- saving rejects to file pom.xml.rej patching file pom.xml Khadijas-MacBook-Pro:mahout-mahout-0.9 khadijaalmohsen$ patch -i 1329-3- additional.patch patching file pom.xml Hunk #1 succeeded at 676 with fuzz 1 (offset 460 lines). Hunk #2 succeeded at 751 with fuzz 1 (offset 521 lines). patching file pom.xml Hunk #2 FAILED at 677. Hunk #3 FAILED at 752. 2 out of 6 hunks FAILED -- saving rejects to file pom.xml.rej patching file pom.xml Hunk #1 FAILED at 201. 1 out of 1 hunk FAILED -- saving rejects to file pom.xml.rej Some resources were advising to simply re-compile mahout. Thus, I tried that using several variation of mvn command: mvn clean install –U mvn clean install –Dhadoop2 –Dhadoop2.version=2.4.0 but again I got an error : ………………… testPrime(org.apache.mahout.math.random.MultinomialTest) Time elapsed: 0.041 sec <<< FAILURE! java.lang.AssertionError: expected:<16> but was:<17> at __randomizedtesting.SeedInfo.seed([113DA6AF561FD9C7:2689253B3FD58807]:0) at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.mahout.math.random.MultinomialTest.testPrime (MultinomialTest.java:143) ……………………… [INFO] -------------------------------------------------------------------- ---- [INFO] Reactor Summary: [INFO] [INFO] Mahout Build Tools ................................ SUCCESS [ 1.898 s] [INFO] Apache Mahout ..................................... SUCCESS [ 0.516 s] [INFO] Mahout Math ....................................... FAILURE [01:41 min] [INFO] Mahout Core ....................................... SKIPPED [INFO] Mahout Integration ................................ SKIPPED [INFO] Mahout Examples ................................... SKIPPED [INFO] Mahout Release Package ............................ SKIPPED [INFO] Mahout Math/Scala wrappers ........................ SKIPPED [INFO] -------------------------------------------------------------------- ---- [INFO] BUILD FAILURE [INFO] -------------------------------------------------------------------- ---- [INFO] Total time: 01:44 min [INFO] Finished at: 2014-09-03T08:25:35+03:00 [INFO] Final Memory: 28M/191M [INFO] -------------------------------------------------------------------- ---- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire- plugin:2.16:test (default-test) on project mahout-math: There are test failures. [ERROR] And now I do not know what to do and how to proceed further. Your cooperation is highly appreciated since I have only few months to finish my dissertation and I am struggling. Thank you in advance Regards, kalmohsen
