PipelinedRDD is an RDD generated by Python mapper/reducer, such as rdd.map(func) will be PipelinedRDD.
PipelinedRDD is an subclass of RDD, so it should have all the APIs which RDD has. >>> sc.parallelize(range(10)).map(lambda x: (x, str(x))).sortByKey().count() 10 I'm wondering that how can you trigger this error? Davies On Tue, Sep 16, 2014 at 10:03 PM, edmond_huo <huoxiang5...@gmail.com> wrote: > Hi, > > I am a freshman about spark. I tried to run a job like wordcount example in > python. But when I tried to get the top 10 popular words in the file, I got > the message:AttributeError: 'PipelinedRDD' object has no attribute > 'sortByKey'. > > So my question is what is the difference between PipelinedRDD and RDD? and > if I want to sort the data in PipelinedRDD, how can I do it? > > Thanks > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/The-difference-between-pyspark-rdd-PipelinedRDD-and-pyspark-rdd-RDD-tp14421.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
