For the exception w.r.t. ManifestFactory , there is SPARK-6497 which is
Open.
FYI
On Fri, Aug 28, 2015 at 8:25 AM, donhoff_h <165612...@qq.com> wrote:
> Hi, all
>
> I wrote a spark program which uses the Kryo serialization. When I count a
> rdd which type is RDD[(String,String)], it reported an Exception like the
> following :
> * Class is not registered: org.apache.spark.util.collection.CompactBuffer[]
> * Note: To register this class use:
> kryo.register(org.apache.spark.util.collection.CompactBuffer[].class);
>
> I found that CompactBuffer is a private class that I could not use the API
> sparkConf.registerKryoClasses to register it. Although I found a solution
> from Andras Barjak that is using the statement :
> kryo.register(ClassTag(Class.forName("org.apache.spark.util.collection.CompactBuffer")).wrap.runtimeClass),
> I tried it with sparkConfg.registerKryoClasses, and only got another
> Exception :
> * Class is not registered: scala.reflect.ManifestFactory$$anon$1
>
> Could anybody tell me :
> 1.Why should I register CompactBuffer[] ? My program never used it.
> 2.How to solve this problem?
>
> Many Thanks!
>
>
