Please take a look at the source code -- it's relatively friendly, and very useful for digging into Spark internals! (KryoSerializer<https://github.com/apache/incubator-spark/blob/master/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala> )
As you can see, a Kryo instance is available via ser.newKryo(). You can also use Spark's SerializerInstance interface (which features serialize() and deserialize() methods) by simply calling ser.newInstance(). On Mon, Jan 6, 2014 at 5:20 PM, Aureliano Buendia <[email protected]>wrote: > In a map closure, I could use: > > val ser = SparkEnv.get.serializer.asInstanceOf[KryoSerializer] > > But how to get the instance of Kryo that spark uses from ser? > > > On Tue, Jan 7, 2014 at 1:04 AM, Aaron Davidson <[email protected]> wrote: > >> I believe SparkEnv.get.serializer would return the serializer created >> from the "spark.serializer" property. >> >> You can also obtain a Kryo serializer directly via it's no-arg >> constructor (it still invokes your spark.kryo.registrator): >> val serializer = new KryoSerializer() >> but this could have some overhead, and so should probably not be done for >> every element you process. >> >> >> On Mon, Jan 6, 2014 at 4:36 PM, Aureliano Buendia >> <[email protected]>wrote: >> >>> Hi, >>> >>> Is there a way to access the global kryo instance created by spark? I'm >>> referring to the one which is passed to registerClasses() in a >>> KryoRegistrator sub class. >>> >>> I'd like to access this kryo instance inside a map closure, so it should >>> be accessible from thw workers side too. >>> >> >> >
