Re: Optimisation Tips

2017-04-12 Thread Pushkar.Gujar
; > I have implemented repartitions on dataframe to 1. It helped the > performance but not to a great extent. I am also looking for answers from > the experts. > > Thanks, > Asmath > > On Wed, Apr 12, 2017 at 9:45 AM, Steve Robinson < > steve.robin...@aquilainsight.com> wrote: &

Re: Optimisation Tips

2017-04-12 Thread KhajaAsmath Mohammed
; > Does anyone have any optimisation tips or could propose an alternative way > to perform the below: > > > val groupedUserItems1 = userItems1.groupByKey{_.customer_id} > val groupedUserItems2 = userItems2.groupByKey{_.customer_id} > groupedUserItems1.cogroup(groupedUserItems2){

Optimisation Tips

2017-04-12 Thread Steve Robinson
Hi, Does anyone have any optimisation tips or could propose an alternative way to perform the below: val groupedUserItems1 = userItems1.groupByKey{_.customer_id} val groupedUserItems2 = userItems2.groupByKey{_.customer_id} groupedUserItems1.cogroup(groupedUserItems2){ case (_, userItems1