It sounds like mapPartitionsWithIndex will give you the information you
want over flatMap.
On Thursday, October 13, 2016, Shushant Arora
wrote:
> Hi
>
> I have a transformation on a pair rdd using flatmap function.
>
> 1.Can I detect in flatmap whether the current record is last record of
> part
Hi
I have a transformation on a pair rdd using flatmap function.
1.Can I detect in flatmap whether the current record is last record of
partition being processed and
2. what is the partition index of this partition.
public Iterable> call(Tuple2 t)
throws Exception {
//whether element is last ele