Vihang Karajgaonkar created IMPALA-9857:
-------------------------------------------
Summary: Batch ALTER_PARTITION events
Key: IMPALA-9857
URL: https://issues.apache.org/jira/browse/IMPALA-9857
Project: IMPALA
Issue Type: Sub-task
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
When Hive inserts data into partitioned tables, it generates a lot of
ALTER_PARTITION (and possibly INSERT_EVENT) in quick succession. Currently,
such events are processed one by one by EventsProcessor which is can be slow
and can cause EventsProcessor to lag behind. This JIRA proposes to use batching
for such ALTER_PARTITION events such that all the successive ALTER_PARTITION
events for the same table are batched together into one ALTER_PARTITIONS event
and then are processed together to refresh all the partitions from the events.
This can significantly speed up the event processing in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
