Change RDDs using map()

[Deep Pradhan]

Hi,
I want to make the following changes in the RDD (create new RDD from the
existing to reflect some transformation):
In an RDD of key-value pair, I want to get the keys for which the values
are 1.
How to do this using map()?
Thank You

Re: Change RDDs using map()

[Mark Hamstra]

You don't.  That's what filter or the partial function version of collect
are for:

val transformedRDD = yourRDD.collect { case (k, v) if k == 1 = v }

On Wed, Sep 17, 2014 at 3:24 AM, Deep Pradhan pradhandeep1...@gmail.com
wrote:

 Hi,
 I want to make the following changes in the RDD (create new RDD from the
 existing to reflect some transformation):
 In an RDD of key-value pair, I want to get the keys for which the values
 are 1.
 How to do this using map()?
 Thank You

RE: Change RDDs using map()

[qihong]

if you want the result as RDD of (key, 1)

  new_rdd = rdd.filter(x = x._2 == 1)

if you want result as RDD of keys (since you know the values are 1), then

  new_rdd = rdd.filter(x = x._2 == 1).map(x = x._1)

x._1 and x._2 are the way of scala to access the key and value from
key/value pair.



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Change-RDDs-using-map-tp14436p14481.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org