Can you paste the complete code? Thanks Best Regards
On Sat, Mar 7, 2015 at 2:25 AM, Ulanov, Alexander <alexander.ula...@hp.com> wrote: > Hi, > > I've implemented class MyClass in MLlib that does some operation on > LabeledPoint. MyClass extends serializable, so I can map this operation on > data of RDD[LabeledPoints], such as data.map(lp => MyClass.operate(lp)). I > write this class in file with ObjectOutputStream.writeObject. Then I stop > and restart Spark. I load this class from file with > ObjectInputStream.readObject.asInstanceOf[MyClass]. When I try to map the > same operation of this class to RDD, Spark throws not serializable > exception: > org.apache.spark.SparkException: Task not serializable > at > org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:166) > at > org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:158) > at org.apache.spark.SparkContext.clean(SparkContext.scala:1453) > at org.apache.spark.rdd.RDD.map(RDD.scala:273) > > Could you suggest why it throws this exception while MyClass is > serializable by definition? > > Best regards, Alexander >
