It it deserialized in a streaming manner as the iterator moves over the partition. This is a functionality of core Spark, and Spark Streaming just uses it as is. What do you want to customize it to?
On Tue, Nov 4, 2014 at 9:22 AM, Mohit Jaggi <mohitja...@gmail.com> wrote: > Folks, > If I have an RDD persisted in MEMORY_ONLY_SER mode and then it is needed > for a transformation/action later, is the whole partition of the RDD > deserialized into Java objects first before my transform/action code works > on it? Or is it deserialized in a streaming manner as the iterator moves > over the partition? Is this behavior customizable? I generally use the Kryo > serializer. > > Mohit. >
