Yeah, I tested that, I had my SPARK_HOME point to a very old location, after I fixed that, everything goes well
Thank you so much for pointing this out Best, -- Nan Zhu On Friday, March 14, 2014 at 6:41 PM, Michael Armbrust wrote: > Sorry to revive an old thread, but I just ran into this issue myself. It > is likely that you do not have the assembly jar built, or that you have > SPARK_HOME set incorrectly (it does not need to be set). > > Michael > > > On Thu, Feb 27, 2014 at 8:13 AM, Nan Zhu <zhunanmcg...@gmail.com > (mailto:zhunanmcg...@gmail.com)> wrote: > > > Hi, all > > > > Actually this problem exists for months in my side, when I run the test > > cases, it will stop (actually pause?) at the ReplSuite > > > > [info] ReplSuite: > > 2014-02-27 10:57:37.220 java[3911:1303] Unable to load realm info from > > SCDynamicStore > > [info] - propagation of local properties (7 seconds, 646 milliseconds) > > [info] - simple foreach with accumulator (6 seconds, 204 milliseconds) > > [info] - external vars (4 seconds, 271 milliseconds) > > [info] - external classes (3 seconds, 186 milliseconds) > > [info] - external functions (4 seconds, 843 milliseconds) > > [info] - external functions that access vars (3 seconds, 503 milliseconds) > > [info] - broadcast vars (4 seconds, 313 milliseconds) > > [info] - interacting with files (2 seconds, 492 milliseconds) > > > > > > > > The next test case should be > > > > test("local-cluster mode") { > > val output = runInterpreter("local-cluster[1,1,512]", > > """ > > |var v = 7 > > |def getV() = v > > |sc.parallelize(1 to 10).map(x => getV()).collect.reduceLeft(_+_) > > |v = 10 > > |sc.parallelize(1 to 10).map(x => getV()).collect.reduceLeft(_+_) > > |var array = new Array[Int](5) > > |val broadcastArray = sc.broadcast(array) > > |sc.parallelize(0 to 4).map(x => broadcastArray.value(x)).collect > > |array(0) = 5 > > |sc.parallelize(0 to 4).map(x => broadcastArray.value(x)).collect > > """.stripMargin) > > assertDoesNotContain("error:", output) > > assertDoesNotContain("Exception", output) > > assertContains("res0: Int = 70", output) > > assertContains("res1: Int = 100", output) > > assertContains("res2: Array[Int] = Array(0, 0, 0, 0, 0)", output) > > assertContains("res4: Array[Int] = Array(0, 0, 0, 0, 0)", output) > > } > > > > > > > > I didn't see any reason for it spending so much time on it.... > > > > Any idea? I'm using mbp, OS X 10.9.1, Intel Core i7 2.9 GHz, Memory 8GB > > 1600 MHz DDR3 > > > > Best, > > > > -- > > Nan Zhu > > > > >
