this has been very slow On Tue, Jun 22, 2021 at 1:15 PM Sachit Murarka <connectsac...@gmail.com> wrote:
> Hi Chetan, > > You can substract the data frame or use except operation. > First DF contains full rows. > Second DF contains unique rows (post remove duplicates) > Subtract first and second DF . > > hope this helps > > Thanks > Sachit > > On Tue, Jun 22, 2021, 22:23 Chetan Khatri <chetan.opensou...@gmail.com> > wrote: > >> Hi Spark Users, >> >> I want to use DropDuplicate, but those records which I discard. I >> would like to log to the instrumental table. >> >> What would be the best approach to do that? >> >> Thanks >> >
