This shows how to serialize user classes. I wanted Spark to serialize all shuffle files and object files using Kryo. How can I specify that? Or would that be done by default if I just set spark.serializer to kryo?
On Mon, Nov 25, 2013 at 7:42 PM, Matei Zaharia <[email protected]>wrote: > Did you look through > http://spark.incubator.apache.org/docs/latest/tuning.html#data-serialization?It > shows an example of how to register classes with Kryo. In particular, in > your Registrator, you can use kryo.register(yourClass, new YourSerializer) > to pass a custom serializer too. > > Matei > > On Nov 25, 2013, at 4:25 PM, Mayuresh Kunjir <[email protected]> > wrote: > > Hi Spark users, > > This has probably been answered before, but I could not locate it. I > understand from the tuning guide that using Kryo serialization for shuffles > improves the performance. I would like to know how to register the Kryo > serializer. Apart from the shuffles, my standalone application needs to > store and retrieve a few object files as well. I would really appreciate > any pointers on registering Kryo serializer for both these serialization > tasks. > > Thanks and regards, > ~Mayuresh > > > >
