I think that you want the lookup() method in PairRDDFunctions? http://spark.incubator.apache.org/docs/latest/api/core/index.html#org.apache.spark.rdd.PairRDDFunctions
It is supposed to be more efficient than filter... Shankari On Thu, Dec 12, 2013 at 7:30 PM, Yadid <[email protected]> wrote: > I have a pairRDD and I would like to access a specific key-value. > The first thing that comes to mind is filtering using the specified key, > but that seems very inefficient as that would iterate over the entire RDD. > And even more so if I need to access several keys. > > Is there any other way to perform this ? this seems like a really useful > feature. Im guessing that in order to implement this, I would need a > mapping of keys to partitions, and a method to access data from a specific > partition. > > Yadid > >
