Vihang Karajgaonkar created IMPALA-10949:
--------------------------------------------

             Summary: Improve batching logic of events
                 Key: IMPALA-10949
                 URL: https://issues.apache.org/jira/browse/IMPALA-10949
             Project: IMPALA
          Issue Type: Improvement
            Reporter: Vihang Karajgaonkar


This is a followup based on the review comment 
https://gerrit.cloudera.org/#/c/17848/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1641

Current approach of batching batches together the events from a single 
operation so that self-event check is done per-batch. However, it looks like 
there is a considerable scope of improving the batching logic by clubbing 
together accross the various sources of the events on a table when IMPALA-10926 
is merged. After IMPALA-10926 each table will track the last_synced_event and 
then the events processor can simply ignore a event which <= the 
last_synced_event. This simplification of self-events logic will enable easier 
batching for all the events of a type on a table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to