If you run the JVM with -Dsun.io.serialization.extendedDebugInfo=true
it will dump the 'path' to the unserializable object when an error
occurs.
On Wed, Nov 5, 2014 at 11:56 PM, ankits <ankitso...@gmail.com> wrote:
> In my spark job, I have a loop something like this:
>
> bla.forEachRdd(rdd => {
> //init some vars
> rdd.forEachPartition(partiton => {
> //init some vars
> partition.foreach(kv => {
> ...
>
> I am seeing serialization errors (unread block data), because I think spark
> is trying to serialize the whole containing class. But I have been careful
> not to reference instance vars in the block.
>
> Is there a way to see exactly what class is failing serialization, and maybe
> how spark decided it needs to be serialized?
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-trace-debug-serialization-tp18230.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
