Thanks for coming back to the list with response!
pt., 27 lut 2015, 3:16 PM Himanish Kushary użytkownik himan...@gmail.com
napisał:
Hi,
I was able to solve the issue. Putting down the settings that worked for
me.
1) It was happening due to the large number of partitions.I *coalesce*'d
Hi,
I was able to solve the issue. Putting down the settings that worked for me.
1) It was happening due to the large number of partitions.I *coalesce*'d
the RDD as early as possible in my code into lot less partitions ( used .
coalesce(1) to bring down from 500K to 10k)
2) Increased the
