Re: coalesce at DataFrame missing argument for shuffle.

2015-12-11 Thread Reynold Xin
I am not sure if we need it. The RDD API has way too many methods and parameters. As you said, it is simply "repartition". On Fri, Dec 11, 2015 at 2:56 PM, Hyukjin Kwon wrote: > Hi all, > > I accidentally met coalesce() function and found this taking arguments > different

coalesce at DataFrame missing argument for shuffle.

2015-12-10 Thread Hyukjin Kwon
Hi all, I accidentally met coalesce() function and found this taking arguments different for RDD and DataFrame. It looks shuffle option is missing for DataFrame. I understand repartition() exactly works as coalesce() with shuffling but it looks a bit weird that the same functions take different