Thanks for the notification! For now, I'll use the Kryo serializer without registering classes until the bug fix has been merged into the next version of Spark (I guess that will be 1.3, right?).
arun On Sun, Feb 1, 2015 at 10:58 PM, Shixiong Zhu <zsxw...@gmail.com> wrote: > It's a bug that has been fixed in > https://github.com/apache/spark/pull/4258 but not yet been merged. > > Best Regards, > Shixiong Zhu > > 2015-02-02 10:08 GMT+08:00 Arun Lists <lists.a...@gmail.com>: > >> Here is the relevant snippet of code in my main program: >> >> =========================================== >> >> sparkConf.set("spark.serializer", >> "org.apache.spark.serializer.KryoSerializer") >> sparkConf.set("spark.kryo.registrationRequired", "true") >> val summaryDataClass = classOf[SummaryData] >> val summaryViewClass = classOf[SummaryView] >> sparkConf.registerKryoClasses(Array( >> >> summaryDataClass, summaryViewClass)) >> >> =========================================== >> >> I get the following error: >> >> Exception in thread "main" java.lang.reflect.InvocationTargetException >> ... >> >> Caused by: org.apache.spark.SparkException: Failed to load class to >> register with Kryo >> ... >> >> Caused by: java.lang.ClassNotFoundException: >> com.dtex.analysis.transform.SummaryData >> >> >> Note that the class in question SummaryData is in the same package as the >> main program and hence in the same jar. >> >> What do I need to do to make this work? >> >> Thanks, >> arun >> >> >> >
