Hi, I have a problem when i want to use spark kryoserializer by extending a class Kryoregistarar to register custom classes inorder to create objects.I am getting following exception When I run following program..Please let me know what could be the problem... ] (run-main) org.apache.spark.SparkException: Job failed: java.io.NotSerializableException: main.scala.Utilities
Registering classes objects: package main.scala import com.esotericsoftware.kryo import org.apache.spark.serializer.KryoRegistrator import com.esotericsoftware.kryo._ class MykryoRegistrar extends KryoRegistrator { override def registerClasses(kryo: Kryo) { kryo.register(classOf[main.scala.Meter_data]) kryo.register(classOf[main.scala.Utilities]) } } MeterData_PerDay:Main class object MeterData_PerDay { def main(args: Array[String]) { System.setProperty("spark.serializer", "org.apache.spark.serializer.KryoSerializer") System.setProperty("spark.kryo.registrator", "main.scala.MykryoRegistrar") var utilclass:Utilities = new Utilities() val sc = new SparkContext("local", "Simple App", utilclass.spark_home, List("target/scala-2.9.3/simple-project_2.9.3-1.0.jar")) val file = sc.textFile(utilclass.data_home) }} -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Problem-with-KryoSerializer-tp4295.html Sent from the Apache Spark User List mailing list archive at Nabble.com.