Mahout 0.8 is compatible with Hadoop 1.x (and not Hadoop 2.x). Hence the issue s u r seeing.
Sent from my iPhone On Jul 24, 2013, at 2:44 AM, Carlos Mundi <[email protected]> wrote: > Is anyone running Mahout 0.8 against Hadoop 2.0.5-alpha? > > I think I have a hadoop configuration issue. I've seen similar discussions > but have not found (or recognized) a solution yet. Rather than tell you > everything I've tried, here's a simple example: > > First, I grab the 0.8 tag and build it: > > cd /home/hduser/Mahout > svn co http://svn.apache.org/repos/asf/mahout/tags/mahout-0.8/ > export $MAHOUT_HOME=cd /home/hduser/Mahout/mahout-0.8 > mvn -DskipTests clean install > > The maven build succeeds. > I start a single-cluster instance of hadoop and verify it is up. > > Now I attempt a canopy clustering, using the helpfully provided script: > > hduser@ubuntu:~/Mahout/mahout-0.8/examples/bin$ > ./cluster-syntheticcontrol.sh > Please select a number to choose the corresponding clustering algorithm > 1. canopy clustering > 2. kmeans clustering > 3. fuzzykmeans clustering > 4. dirichlet clustering > 5. meanshift clustering > Enter your choice : 1 > ok. You chose 1 and we'll use canopy Clustering > creating work directory at /tmp/mahout-work-hduser > Downloading Synthetic control data > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 100 281k 100 281k 0 0 598k 0 --:--:-- --:--:-- --:--:-- > 707k > Checking the health of DFS... > 13/07/23 23:01:24 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > Found 2 items > drwxr-xr-x - hduser supergroup 0 2013-07-20 08:24 guten > drwxr-xr-x - hduser supergroup 0 2013-07-23 22:54 testdata > DFS is healthy... > Uploading Synthetic control data to HDFS > rmr: DEPRECATED: Please use 'rm -r' instead. > 13/07/23 23:01:26 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > Deleted testdata > 13/07/23 23:01:28 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 13/07/23 23:01:30 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > Successfully Uploaded Synthetic control data to HDFS > MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath. > Running on hadoop, using /usr/local/hadoop/bin/hadoop and > HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop > MAHOUT-JOB: > /home/hduser/Mahout/mahout-0.8/examples/target/mahout-examples-0.8-job.jar > 13/07/23 23:01:51 WARN driver.MahoutDriver: No > org.apache.mahout.clustering.syntheticcontrol.canopy.Job.props found on > classpath, will use command-line arguments only > Exception in thread "main" java.lang.NoSuchMethodError: > org.apache.hadoop.util.ProgramDriver.driver([Ljava/lang/String;)V > at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:194) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > > > Hmmm... looking at the pom.xml, I see that Mahout 0.8 calls for hadoop > 1.1.2. I also see that there is no POM for hadoop 2.x.y at > http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-core/* > > Consulting both of > > http://hadoop.apache.org/docs/r1.1.2/api/index.html > http://hadoop.apache.org/docs/r2.0.5-alpha/api/index.html > > shows that org.apache.hadoop.util has been seriously reworked for 2.x > > Perhaps it is as simple as rewriting the examples to use the new hadoop > API? I've already been working around some trivial API name changes in > hadoop 2.0, so maybe ProgramDriver just got refactored? > > I need the federation in hadoop 2.x so I am hoping someone has been down > this path! > > So... Is anyone running Mahout 0.8 against Hadoop 2.x.y ? > > I am learning the ecosystem, and I appreciate links to things I may have > missed. > > Thanks!
