Hi Gimantha, To add to this question, if we have enabled data purging, does it mean we shouldn't use INSERT OVERWRITE? Doing INSERT OVEWRITES to a table which has scheduled data purging will also result in data loss right?
On Mon, Nov 13, 2017 at 4:57 PM, Gimantha Bandara <[email protected]> wrote: > Hi, > > I don't think we have a valid use case of using OVERWRITE with incremental > processing unless we want to keep the last processed data set in a separate > table. > > On Mon, Nov 13, 2017 at 4:30 PM, Sajith Ravindra <[email protected]> wrote: > >> Hi all, >> >> Can there be a valid use case where we do an INSERT OVERWRITE to a table >> and then do incremental table commit? >> >> AFAIU, it leads to loss of data as incremental processing processes only >> unprocessed data and OVERWRITE statement drops previously processed data. >> >> Please advice. >> >> Thanks >> *,Sajith Ravindra* >> Associate Technical Lead >> WSO2 Inc.; http://wso2.com >> lean.enterprise.middleware >> >> mobile: +94 77 2273550 >> blog: http://sajithr.blogspot.com/ >> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >> > > > > -- > Gimantha Bandara > Software Engineer > WSO2. Inc : http://wso2.com > Mobile : +94714961919 <+94%2071%20496%201919> > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Thanks & Regards, *Fazlan Nazeem* Senior Software Engineer WSO2 Inc Mobile : +94772338839 <%2B94%20%280%29%20773%20451194> [email protected]
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
