Problem with flatmap.

[Archit Thakur]

Hi,

I am facing a general problem with flatmap operation on rdd.

I am doing

MyRdd.flatmap(func(_))
MyRdd.saveAsTextFile(..)

func(Tuple2[Key, Value]): List[Tuple2[MyCustomKey, MyCustomValue]] = {

//

println(list)
list
}

now if I check the list from the logs at worker and check the textfile it
has created, it differs.

Only the no. of records are same, but the actual records in the file
differs from one in the logs.

Does Spark modifies keys/values in between? What other operations does it
perform with Key or Value?

Thanks and Regards,
Archit Thakur.