Re: Why Spark require this object to be serializerable?

2014-04-29 Thread Earthson
/Why-Spark-require-this-object-to-be-serializerable-tp5009p5036.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Why Spark require this object to be serializerable?

2014-04-28 Thread Earthson
-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009p5024.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Why Spark require this object to be serializerable?

2014-04-28 Thread DB Tsai
; runtinfo.value, it.toSeq, innerRound).toIterator).cache > 99 println("failed?") //DEBUG > 100 step = (step + 1) % saveStep > 101 if (step == 0 && i+1 != round) > mwz.saveAsTextFile(s"hdfs://ns1/nlp/lda/solution.round.${i}") > 102

Re: Why Spark require this object to be serializerable?

2014-04-28 Thread Earthson
} 105 } -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009p5020.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Why Spark require this object to be serializerable?

2014-04-28 Thread DB Tsai
c(ForkJoinTask.java:260) > at > > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > 14/04/29 09:48:42 INFO yarn.ApplicationMaster: AppMaster received a signal. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Why Spark require this object to be serializerable?

2014-04-28 Thread Earthson
The RDD hold "this" in its closure? How to fix such a problem? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009p5015.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Why Spark require this object to be serializerable?

2014-04-28 Thread Earthson
Or what is the action that make the rdd run. I don't what to save it as file, and I've tried cache(), it seems to be some kind of lazy too. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009

Why Spark require this object to be serializerable?

2014-04-28 Thread Earthson
ForkJoinWorkerThread.java:107) 14/04/29 09:48:42 INFO yarn.ApplicationMaster: AppMaster received a signal. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009.html Sent from the Apache Spark User List mailing list archive at Nabble.com.