The version of Spark being used for Mahout is now 1.0.x It was just changed in the repo
On Aug 13, 2014, at 7:48 AM, Andrea Abelli <[email protected]> wrote: Hello again I did some additional fiddling with ./bin/mahout : vagrant@vagrant-ubuntu-trusty-64:~/tl/mahout$ git diff diff --git a/bin/mahout b/bin/mahout index 5f54181..0174b31 100755 --- a/bin/mahout +++ b/bin/mahout @@ -161,7 +161,7 @@ then fi # add scala dev target - for f in $MAHOUT_HOME/math-scala/target/mahout-math-scala-*.jar ; do + for f in $MAHOUT_HOME/math-scala/target/mahout-math-scala_*.jar ; do CLASSPATH=${CLASSPATH}:$f; done @@ -173,11 +173,11 @@ then CLASSPATH=${CLASSPATH}:$f; done - for f in $MAHOUT_HOME/spark/target/mahout-spark-*.jar ; do + for f in $MAHOUT_HOME/spark/target/mahout-spark_*.jar ; do CLASSPATH=${CLASSPATH}:$f; done - for f in $MAHOUT_HOME/spark-shell/target/mahout-spark-shell-*.jar ; do + for f in $MAHOUT_HOME/spark-shell/target/mahout-spark-shell_*.jar ; do CLASSPATH=${CLASSPATH}:$f; done and got this error when running ./bin/mahout spark-shell Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.HttpServer.<init>(Ljava/io/File;)V at org.apache.spark.repl.SparkIMain.<init>(SparkIMain.scala:100) at org.apache.spark.repl.SparkILoop$SparkILoopInterpreter.<init>(SparkILoop.scala:172) at org.apache.spark.repl.SparkILoop.createInterpreter(SparkILoop.scala:191) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:883) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:881) at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:881) at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:881) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:973) at org.apache.mahout.sparkbindings.shell.Main$.main(Main.scala:31) at org.apache.mahout.sparkbindings.shell.Main.main(Main.scala) I then changed SPARK_HOME from 0.9.1 to spark-1.0.2 and now it seems to work fine. Regards Andrea On Wed, Aug 13, 2014 at 2:19 PM, Andrea Abelli <[email protected]> wrote: > Hi > > I hope you are well. > While following this tutorial > https://mahout.apache.org/users/sparkbindings/play-with-shell.html > I ran into some problems. > At point 4. of "Starting Mahout's Spark shell", executing `bin/mahout > spark-shell` returns > Error: Could not find or load main class > org.apache.mahout.sparkbindings.shell.Main > so I had a look at classes folder's tree and ./bin/mahout's source code. > > vagrant@vagrant-ubuntu-trusty-64:~/tl/mahout$ ls -l > $MAHOUT_HOME/spark-shell/target/ > total 40 > drwxrwxr-x 3 vagrant vagrant 4096 Aug 13 11:18 classes > -rw-rw-r-- 1 vagrant vagrant 1 Aug 13 11:18 classes.timestamp > -rw-rw-r-- 1 vagrant vagrant 3014 Aug 13 11:18 > mahout-spark-shell_2.10-1.0-SNAPSHOT-sources.jar > -rw-rw-r-- 1 vagrant vagrant 3132 Aug 13 11:18 > mahout-spark-shell_2.10-1.0-SNAPSHOT-tests.jar > -rw-rw-r-- 1 vagrant vagrant 14136 Aug 13 11:18 > mahout-spark-shell_2.10-1.0-SNAPSHOT.jar > drwxrwxr-x 2 vagrant vagrant 4096 Aug 13 11:18 maven-archiver > drwxrwxr-x 2 vagrant vagrant 4096 Aug 13 11:18 test-classes > > while line 180 in ./bin/mahout reads > for f in $MAHOUT_HOME/spark-shell/target/mahout-spark-shell-*.jar ; do > > Now, by applying the following diff > > diff --git a/bin/mahout b/bin/mahout > index 5f54181..a6f4ba8 100755 > --- a/bin/mahout > +++ b/bin/mahout > @@ -177,7 +177,7 @@ then > CLASSPATH=${CLASSPATH}:$f; > done > > - for f in $MAHOUT_HOME/spark-shell/target/mahout-spark-shell-*.jar ; do > + for f in $MAHOUT_HOME/spark-shell/target/mahout-spark-shell_*.jar ; do > CLASSPATH=${CLASSPATH}:$f; > done > > I'm now able to get to mahout's shell after running `./bin/mahout > spark-shell`, but I get the following errors > > Using Scala version 2.10.3 (OpenJDK 64-Bit Server VM, Java 1.7.0_55) > Type in expressions to have them evaluated. > Type :help for more information. > <console>:9: error: object drm is not a member of package > org.apache.mahout.math > @transient implicit val sdc: > org.apache.mahout.math.drm.DistributedContext = > ^ > <console>:10: error: type SparkDistributedContext is not a member of > package org.apache.mahout.sparkbindings > new > org.apache.mahout.sparkbindings.SparkDistributedContext( > ^ > Mahout distributed context is available as "implicit val sdc". > <console>:13: error: not found: value scalabindings > import scalabindings._ > ^ > <console>:13: error: not found: value RLikeOps > import RLikeOps._ > ^ > <console>:13: error: not found: value drm > import drm._ > ^ > <console>:13: error: not found: value RLikeDrmOps > import RLikeDrmOps._ > ^ > > Has anyone any idea of what's going on/wrong? Any hints on what I'm doing > wrong or how I could fix this? > > Thanks in advance, and thanks for the awesome project. > Looking forward to participate. > > Regards > Andrea > -- Andrea Abelli | TERALYTICS *analytics scientist* Teralytics AG | Zollstrasse 62 | 8005 Zurich | Switzerland phone: +353 83 442 44 88 email: [email protected] www.teralytics.net Company registration number: CH-020.3.037.709-7 | Trade register Canton Zurich Board of directors: Georg Polzer, Mark Schmitz, Dr. Angelica Kohlmann Küpper Data Privacy Supervisor: Prof. Dr. Donald Alan Kossmann This e-mail message contains confidential information which is for the sole attention and use of the intended recipient. Please notify us at once if you think that it may not be intended for you and delete it immediately.
