Thanks Matthew. According to documentation, foreachRDD is used for writing right?
I did with like this and it worked: address_by_destination = address.updateStateByKey(updateFunction).transform(lambda rdd: rdd.sortBy(lambda x: x[1], False)) Thanks anyways :) On Thu, Nov 12, 2015 at 4:53 PM, Young, Matthew T <matthew.t.yo...@intel.com > wrote: > You can use foreachRDD > <http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations-on-dstreams> > to get access to the batch API in streaming jobs. > > > > > > *From:* Amir Rahnama [mailto:amirrahn...@gmail.com] > *Sent:* Thursday, November 12, 2015 12:11 AM > *To:* ayan guha <guha.a...@gmail.com> > *Cc:* user <user@spark.apache.org> > *Subject:* Re: How can you sort wordcounts by counts in > stateful_network_wordcount.py example > > > > Hi Ayan, > > > > Thanks for help, > > > > Your example is not the streaming example. There we don't have sortByKey. > > > > > > > > On Wed, Nov 11, 2015 at 11:35 PM, ayan guha <guha.a...@gmail.com> wrote: > > how about this? > > > > sorted = running_counts.map(lambda t: t[1],t[0]).sortByKey() > > > > Basically swap key and value of the RDD and then sort? > > > > On Thu, Nov 12, 2015 at 8:53 AM, Amir Rahnama <amirrahn...@gmail.com> > wrote: > > Hey, > > > > Anybody knows how can one sort the result in the stateful example? > > > > Python would be prefered. > > > > > https://github.com/apache/spark/blob/859dff56eb0f8c63c86e7e900a12340c199e6247/examples/src/main/python/streaming/stateful_network_wordcount.py > > -- > > Thanks and Regards, > > > > Amir Hossein Rahnama > > > > *Tel: +46 (0) 761 681 102* > > Website: www.ambodi.com > > Twitter: @_ambodi <https://twitter.com/_ambodi> > > > > > > -- > > Best Regards, > Ayan Guha > > > > > > -- > > Thanks and Regards, > > > > Amir Hossein Rahnama > > > > *Tel: +46 (0) 761 681 102* > > Website: www.ambodi.com > > Twitter: @_ambodi <https://twitter.com/_ambodi> > -- Thanks and Regards, Amir Hossein Rahnama *Tel: +46 (0) 761 681 102* Website: www.ambodi.com Twitter: @_ambodi <https://twitter.com/_ambodi>
