That's because a generic Iterator doesn't necessarily have a definite size (cf. Iterator.hasDefiniteSize), and it is not possible to know where the end of the Iterator is or how to take the last k elements from the Iterator. Indeed, a Iterator could be over an infinite stream and have no end or last k elements.
On Thu, Oct 24, 2013 at 3:59 PM, Stoney Vintson <[email protected]> wrote: > I don't see common iterable trait methods such as takeRight() or last() in > the spark scala api documentation. There are sampling and sorting methods. > sample, sortByKey > > spark scala api > > http://spark.incubator.apache.org/docs/latest/scala-programming-guide.html#rdd-operations > > org.apache.spark.rdd > > http://spark.incubator.apache.org/docs/latest/api/core/index.html#org.apache.spark.rdd.RDD > > spark-0.8.0-incubating/core/src/main/scala/org/apache/spark/rdd > > Also, remember that in scala tail() will return all of the elements except > for head. > > > > On Thu, Oct 24, 2013 at 3:23 PM, Matt Cheah <[email protected]> wrote: > >> Hi everyone, >> >> I see there is a take() function for RDDs, getting the first n >> elements. Is there a way to get the last n elements? >> >> Thanks, >> >> -Matt Cheah >> > > >
