Check roll up and cube functions in spark sql.
On Wed, 23 Jan 2019 at 10:47 PM, Pierremalliard <
[email protected]> wrote:
> Hi,
>
> I am trying to generate a dataframe of all combinations that have a same
> key
> using Pyspark.
>
> example:
>
> (a,1)
> (a,2)
> (a,3)
> (b,1)
> (b,2)
>
> should return:
>
> (a, 1 , 2)
> (a, 1 , 3)
> (a, 2, 3)
> (b, 1 ,2)
>
>
> i want to do something like df.groupBy('key').combinations().apply(...)
>
> any suggestions are welcome !!!
>
> Thanks
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: [email protected]
>
>
