Same concept applies to Dataframe as it is with RDD with respect to transformations. Both are distributed data set.
Thanks On Thu, Feb 7, 2019 at 8:51 AM Faiz Chachiya <[email protected]> wrote: > Hello Team, > > With RDDs it is pretty clear which operations would result in wide > transformations and there are also options available to find out parent > dependencies > > I have been struggling to do the same with DataFrame/DataSet, I need your > helping in finding out which operations may lead to wide transformations > like (OrderBy) and if there is way to find out the parent dependencies. > > There is one way to find out parent dependencies by converting the DF/DS > to RDD and invoke the dependencies. > > I hope my question is clear and would request your help with it. > > Thanks, > Faiz >
