Hi all, how can i tell if my kryos serializer is actually working. I have a
class which extends Serializable and i have included the following imports:
import com.esotericsoftware.kryo.Kryo
import org.apache.spark.serializer.KryoRegistrator
i also have class included
MyRegistrator extends KryoRegistrator {
override def registerClasses(kryo: Kryo) {
kryo.register(classOf[Person])
}
}
my configuration includes
spark.serializer org.apache.spark.serializer.KryoSerializer
but not sure if its working (code runs fine though, although parts using
larger data sets pretty slow). I see in the documentation i should use
conf.set("spark.kryo.registrator", "mypackage.MyRegistrator") but as im
using the scala repl not sure where this should go or what format should be
in which leads me to wondering if im using the default java serialization.
Is there a way i can tell, and how to I go about including the myRegistrator
bit above.
Thanks in advance for any help
D
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/kryos-serializer-tp16454.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
