/Why-Spark-require-this-object-to-be-serializerable-tp5009p5036.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009p5024.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
; runtinfo.value, it.toSeq, innerRound).toIterator).cache
> 99 println("failed?") //DEBUG
> 100 step = (step + 1) % saveStep
> 101 if (step == 0 && i+1 != round)
> mwz.saveAsTextFile(s"hdfs://ns1/nlp/lda/solution.round.${i}")
> 102
}
105 }
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009p5020.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
c(ForkJoinTask.java:260)
> at
>
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
>
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> 14/04/29 09:48:42 INFO yarn.ApplicationMaster: AppMaster received a signal.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
The RDD hold "this" in its closure? How to fix such a problem?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009p5015.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Or what is the action that make the rdd run. I don't what to save it as file,
and I've tried cache(), it seems to be some kind of lazy too.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009
ForkJoinWorkerThread.java:107)
14/04/29 09:48:42 INFO yarn.ApplicationMaster: AppMaster received a signal.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Why-Spark-require-this-object-to-be-serializerable-tp5009.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.