Re: spark 0.8.0 fails on larger data set (Failed to run reduce at GradientDescent.scala:144)

2013-12-16 Thread Taka Shinagawa
c 12, 2013 at 9:38 PM, Patrick Wendell wrote: > >> See if there are any logs on the slaves that suggest why the tasks are >> failing. Right now the master log is just saying "some stuff is failing" >> but it's not clear why. >> >> >> On Thu, Dec 1

Re: spark 0.8.0 fails on larger data set (Failed to run reduce at GradientDescent.scala:144)

2013-12-12 Thread Taka Shinagawa
How big is your data set? Did you set SPARK_MEM and SPARK_WORKER_MEMORY environmental variables? On Thu, Dec 12, 2013 at 9:07 AM, Walrus theCat wrote: > Hi all, > > I've had smashing success with Spark 0.7.x with this code, and this same > code on Spark 0.8.0 using a smaller data set. However