<http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab>
On Mon, Nov 13, 2017 at 5:43 PM, Gihan Anuruddha <[email protected]> wrote: > Keeping a last processed data set in a separate table might be a valid > use case for some scenario. Users can run an one job per day and gather > last process data and could do some operations. This avoids the overhead to > go through the entire data set. > @Gihan, Gimanatha: But, is this a direct or intended use case of using these two construts together. Isn't it more of a by product? If some user wants to process only subset of data wouldn't the user use a "WHERE" clause? > > On Mon, Nov 13, 2017 at 4:57 PM, Gimantha Bandara <[email protected]> > wrote: > >> Hi, >> >> I don't think we have a valid use case of using OVERWRITE with >> incremental processing unless we want to keep the last processed data set >> in a separate table. >> >> On Mon, Nov 13, 2017 at 4:30 PM, Sajith Ravindra <[email protected]> >> wrote: >> >>> Hi all, >>> >>> Can there be a valid use case where we do an INSERT OVERWRITE to a table >>> and then do incremental table commit? >>> >>> AFAIU, it leads to loss of data as incremental processing processes only >>> unprocessed data and OVERWRITE statement drops previously processed data. >>> >>> Please advice. >>> >>> Thanks >>> *,Sajith Ravindra* >>> Associate Technical Lead >>> WSO2 Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> mobile: +94 77 2273550 >>> blog: http://sajithr.blogspot.com/ >>> <http://lk.linkedin.com/pub/shani-ranasinghe/34/111/ab> >>> >> >> >> >> -- >> Gimantha Bandara >> Software Engineer >> WSO2. Inc : http://wso2.com >> Mobile : +94714961919 <071%20496%201919> >> > > > > -- > W.G. Gihan Anuruddha > Associate Technical Lead | WSO2, Inc. > M: +94772272595 >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
