Re: How does HashPartitioner distribute data in Spark?

2017-06-25 Thread Russell Spitzer
a is distributed uniformaly. >>>> I do have following questions on the basis of above observation: >>>> >>>> 1. In case of rdd1, hash partitioning should calculate hashcode of key >>>> (i.e. "aa" in this case), so all records should go to single partition >>>> instead of uniform distribution? >>>> 2. In case of rdd2, there is no key value pair so how hash partitoning >>>> going to work i.e. what is the key to calculate hashcode? >>>> >>>> I have followed @zero323 answer but not getting answer of these. >>>> >>>> https://stackoverflow.com/questions/31424396/how-does-hashpartitioner-work >>>> >>>> >>>> >>>> >>>> - >>>> >>>> __Vikash Pareek >>>> -- >>>> View this message in context: >>>> http://apache-spark-user-list.1001560.n3.nabble.com/How-does-HashPartitioner-distribute-data-in-Spark-tp28785.html >>>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>>> >>>> - >>>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>>> >>>> >>> >>

Re: How does HashPartitioner distribute data in Spark?

2017-06-24 Thread Russell Spitzer
questions on the basis of above observation: >>> >>> 1. In case of rdd1, hash partitioning should calculate hashcode of key >>> (i.e. "aa" in this case), so all records should go to single partition >>> instead of uniform distribution? >>> 2. In

Re: How does HashPartitioner distribute data in Spark?

2017-06-24 Thread Vikash Pareek
; (i.e. "aa" in this case), so all records should go to single partition >> instead of uniform distribution? >> 2. In case of rdd2, there is no key value pair so how hash partitoning >> going to work i.e. what is the key to calculate hashcode? >> >> I have followe

Re: How does HashPartitioner distribute data in Spark?

2017-06-23 Thread Vadim Semenov
> instead of uniform distribution? > 2. In case of rdd2, there is no key value pair so how hash partitoning > going to work i.e. what is the key to calculate hashcode? > > I have followed @zero323 answer but not getting answer of these. > https://stackoverflow.com/questions/3

How does HashPartitioner distribute data in Spark?

2017-06-23 Thread Vikash Pareek
late hashcode? I have followed @zero323 answer but not getting answer of these. https://stackoverflow.com/questions/31424396/how-does-hashpartitioner-work ----- __Vikash Pareek -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-does-HashPartitioner-d