Nevermind, I've just run the code in the REPL. Indeed if we do not sort, then the order is totally random. Which actually makes sens if you think about it
On Thu, Oct 16, 2014 at 9:58 PM, Paweł Szulc <[email protected]> wrote: > Just to have this clear, can you answer with quick yes or no: > > Does it mean that when I create RDD from a file and I simply iterate > through it like this: > > sc.textFile("some_text_file.txt").foreach(line => println(line)) > > then the actual lines might come in different order then they are in the > file? > > On Thu, Oct 16, 2014 at 9:13 PM, Sean Owen <[email protected]> wrote: > >> Since you're concerned with the particular ordering, you will need to >> sort your RDD to ensure the ordering you have in mind. Simply reverse >> the Ordering with Ordering.reverse() and sort by that instead, and >> then use toLocalIterator() I suppose. >> >> Depending on what you're really trying to achieve, there may be a better >> way. >> >> On Thu, Oct 16, 2014 at 2:49 PM, ll <[email protected]> wrote: >> > hello... what is the best way to iterate through an rdd backward (last >> > element first, first element last)? thanks! >> > >> > >> > >> > -- >> > View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/reverse-an-rdd-tp16602.html >> > Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] >> > For additional commands, e-mail: [email protected] >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> >
