I have given this a try in a spark-shell and I still get many Allocation
Failures
On Thursday, July 3, 2014 9:51 AM, Xiangrui Meng men...@gmail.com wrote:
The SparkKMeans is just an example code showing a barebone
implementation of k-means. To run k-means on big datasets, please use
the
I want to make some minor modifications in the SparkMeans.scala so running the
basic example won't do.
I have also packed my code under a jar file with sbt. It completes
successfully but when I try to run it : java -jar myjar.jar I get the same
error:
Exception in thread main
Got it ! Ran the jar with spark-submit. Thanks !
On Wednesday, July 2, 2014 9:16 AM, Wanda Hawk wanda_haw...@yahoo.com wrote:
I want to make some minor modifications in the SparkMeans.scala so running the
basic example won't do.
I have also packed my code under a jar file with sbt. It
The scripts that Xiangrui mentions set up the classpath...Can you run
./run-example for the provided example sucessfully?
What you can try is set SPARK_PRINT_LAUNCH_COMMAND=1 and then call
run-example -- that will show you the exact java command used to run
the example at the start of execution.
I can run it now with the suggested method. However, I have encountered a new
problem that I have not faced before (sent another email with that one but here
it goes again ...)
I ran SparkKMeans with a big file (~ 7 GB of data) for one iteration with
spark-0.8.0 with this line in bash.rc
You can use either bin/run-example or bin/spark-summit to run example
code. scalac -d classes/ SparkKMeans.scala doesn't recognize Spark
classpath. There are examples in the official doc:
http://spark.apache.org/docs/latest/quick-start.html#where-to-go-from-here
-Xiangrui
On Tue, Jul 1, 2014 at
