Re: Question regarding doing aggregation over custom partitions

Thanks, that was what I was missing! arun arun *__* *Arun Swami* +1 408-338-0906 On Fri, May 2, 2014 at 4:28 AM, Mayur Rustagi mayur.rust...@gmail.comwrote: You need to first partition the data by the key Use mappartition instead of map. Mayur Rustagi Ph: +1 (760) 203 3257

Question regarding doing aggregation over custom partitions

individually in each partition. In a separate step, I want to recover the global counts across all partitions while keeping the partition based counts. How can I do this in Spark? Thanks! arun *__* *Arun Swami*