in the comments on SparkContext.objectFile it says:
"It will also be pretty slow if you use the default serializer (Java
serialization)"

this suggests the spark.serializer is used, which means i can switch to the
much faster kryo serializer. however when i look at the code it uses
Utils.deserialize, which is always using Java serialization.

did i get that right? and is this desired?
it seems straightforward to switch objectFile to use the serializer as
specified by spark.serializer (although it might being in new classloader
issues).

Reply via email to