On Wed, Sep 17, 2014 at 7:37 AM, edmond_huo <[email protected]> wrote: > Hi Davis, > > When I run your code in pyspark, I still get the same error: > >>>> sc.parallelize(range(10)).map(lambda x: (x, str(x))).sortByKey().count() > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > AttributeError: 'PipelinedRDD' object has no attribute 'sortByKey' > > Is it the matter with spark version? I am using spark-0.7.3.
This is the problem, this version is so old, could you upgrade to 1.1? There are bunch of improvements out of the box, it will save your life :-) Davies > Thanks. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/The-difference-between-pyspark-rdd-PipelinedRDD-and-pyspark-rdd-RDD-tp14421p14449.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
