You can do this with the 'cartesian' product method on RDD. For example:
val rdd1 = ...
val rdd2 = ...
val combinations = rdd1.cartesian(rdd2).filter{ case (a,b) => a < b }
Reza
On Sat, Mar 21, 2015 at 10:37 PM, Xi Shen <davidshe...@gmail.com> wrote:
> Hi,
>
> I have two big RDD, and I need to do some math against each pair of them.
> Traditionally, it is like a nested for-loop. But for RDD, it cause a nested
> RDD which is prohibited.
>
> Currently, I am collecting one of them, then do a nested for-loop, so to
> avoid nested RDD. But would like to know if there's spark-way to do this.
>
>
> Thanks,
> David
>
>
