Re: Hard Delete

2020-07-17 Thread Sivaprakash 
Hi Balaji,


   - I   want to retain my commits for a long time - normally I'll never
   delete as I want to back track and query history for a longer period of
   time
   - In this scenario, is it still cleaner to help me to remove particular
   records or other options?




On Fri, Jul 17, 2020 at 5:07 PM Balaji Varadarajan
 wrote:

>  Hi Sivaprakash,
> You can configure cleaner to clean the older file versions which contain
> those records to be deleted. You can take a look at
> https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-WhatdoestheHudicleanerdo
>  for
> more details.
>
> Balaji.V
> On Friday, July 17, 2020, 07:47:55 AM PDT, Sivaprakash <
> sivaprakashshanmu...@gmail.com> wrote:
>
>  Hello
>
> Do we have any option to delete a record from every partition? Which mean I
> want to completely wipe out particular record from complete data set (first
> commit, all the changes,  delta commit etc)
>
> Currently, when I delete it affects only the last commit but if I do an
> incremental query on the history it still has it - I want to remove them as
> well. Possible?
>
> Thank you !!
>



-- 
- Prakash.

Re: Hard Delete

2020-07-17 Thread Balaji Varadarajan 
 Hi Sivaprakash,
You can configure cleaner to clean the older file versions which contain those 
records to be deleted. You can take a look at 
https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-WhatdoestheHudicleanerdo
 for more details.

Balaji.V
On Friday, July 17, 2020, 07:47:55 AM PDT, Sivaprakash 
 wrote:  
 
 Hello

Do we have any option to delete a record from every partition? Which mean I
want to completely wipe out particular record from complete data set (first
commit, all the changes,  delta commit etc)

Currently, when I delete it affects only the last commit but if I do an
incremental query on the history it still has it - I want to remove them as
well. Possible?

Thank you !!

Hard Delete

2020-07-17 Thread Sivaprakash 
Hello

Do we have any option to delete a record from every partition? Which mean I
want to completely wipe out particular record from complete data set (first
commit, all the changes,  delta commit etc)

Currently, when I delete it affects only the last commit but if I do an
incremental query on the history it still has it - I want to remove them as
well. Possible?

Thank you !!