Well the issue was because I was using some non thread-safe functions for generating the key.
Regards, Sushrut Ikhar [image: https://]about.me/sushrutikhar <https://about.me/sushrutikhar?promo=email_sig> On Tue, Dec 15, 2015 at 2:27 PM, Paweł Szulc <[email protected]> wrote: > Hard to imagine. Can you share a code sample? > > On Tue, Dec 15, 2015 at 8:06 AM, Sushrut Ikhar <[email protected]> > wrote: > >> Hi, >> I am finding it difficult to understand the following problem : >> I count the number of records before and after applying the mapValues >> transformation for a JavaPairRDD. As expected the number of records were >> same before and after. >> >> Now, I counted number of distinct keys before and after applying the >> mapValues transformation for the same JavaPairRDD. However, I get less >> count after applying the transformation. I expected mapValues will not >> change the keys. Then why am I getting lesser distinct keys? Note that - >> the total records are the same only distinct keys have dropped. >> >> using spark-1.4.1. >> >> Thanks in advance. >> >> Regards, >> >> Sushrut Ikhar >> [image: https://]about.me/sushrutikhar >> <https://about.me/sushrutikhar?promo=email_sig> >> >> > > > > -- > Regards, > Paul Szulc > > twitter: @rabbitonweb > blog: www.rabbitonweb.com >
