I type T i already have Object ... I have RDD<Object> and then I am calling ZipWithIndex on this RDD and getting RDD<Object,Long> on this I am running MapToPair and converting into RDD<Long,Object> so that i can use it later for other operation like lookup and join.
On 16 April 2015 at 23:42, Ted Yu <yuzhih...@gmail.com> wrote: > The Long in RDD[(T, Long)] is type parameter. You can create RDD with > Integer as the first type parameter. > > Cheers > > On Thu, Apr 16, 2015 at 11:07 AM, Jeetendra Gangele <gangele...@gmail.com> > wrote: > >> Hi Ted. >> This works for me. But since Long takes here 8 bytes. Can I reduce it to >> 4 bytes. its just a index and I feel 4 bytes was more than enough.is >> there any method which takes Integer or similar for Index? >> >> >> On 13 April 2015 at 01:59, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> bq. will return something like JavaPairRDD<Object, long> >>> >>> The long component of the pair fits your description of index. What >>> other requirement does ZipWithIndex not provide you ? >>> >>> Cheers >>> >>> On Sun, Apr 12, 2015 at 1:16 PM, Jeetendra Gangele <gangele...@gmail.com >>> > wrote: >>> >>>> Hi All I have an RDD JavaRDD<Object> and I want to convert it to >>>> JavaPairRDD<Index,Object>.. Index should be unique and it should maintain >>>> the order. For first object It should have 1 and then for second 2 like >>>> that. >>>> >>>> I tried using ZipWithIndex but it will return something like >>>> JavaPairRDD<Object, long> >>>> I wanted to use this RDD for lookup and join operation later in my >>>> workflow so ordering is important. >>>> >>>> >>>> Regards >>>> jeet >>>> >>> >>> >> >> >> >
