[
https://issues.apache.org/jira/browse/IMPALA-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078919#comment-17078919
]
ASF subversion and git services commented on IMPALA-8632:
---------------------------------------------------------
Commit e0ed7d321c1dca85a0f1482842f8db6db517c909 in impala's branch
refs/heads/master from xiaomeng
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e0ed7d3 ]
IMPALA-8632: Add support for self-event detection for insert events
In case of INSERT_EVENTS if Impala inserts into a table it causes a
refresh to the underlying table/partition. This could be unnecessary
when there is only one Impala cluster in the system.
We can detect a self-event in such cases when the HMS API to fire a
listener event returns the event id. This is used by EventProcessor
to ignore the event when it is fetched later in the next polling cycle.
Testing:
Add testInsertFromImpala() in MetastoreEventsProcessorTest.java to test
insert event self-event detection when insert into table and partition.
Change-Id: I7873fbb2c159343690f93b9d120f6b425b983dcf
Reviewed-on: http://gerrit.cloudera.org:8080/15648
Reviewed-by: Vihang Karajgaonkar <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add support for self-event detection for insert events
> ------------------------------------------------------
>
> Key: IMPALA-8632
> URL: https://issues.apache.org/jira/browse/IMPALA-8632
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Vihang Karajgaonkar
> Assignee: Xiaomeng Zhang
> Priority: Critical
>
> In case of {{INSERT_EVENTS}} if Impala inserts into a table it causes a
> refresh to the underlying table/partition. This could be unnecessary when
> there is only one Impala cluster in the system. The existing self-event
> detection framework cannot identify such events because they are not sending
> HMS objects like tables and partitions to the HMS. Instead in case of
> {{INSERT_EVENT}} HMS API only asks for a table name or partition value to
> fire a insert event on it.
> We can detect a self-event in such cases if the HMS API to fire a listener
> event is improved to return the event id. This would be used by
> EventProcessor to ignore the event when it is fetched later in the next
> polling cycle. In order to support this, we will need to make a change to
> Hive as well so that the enhanced API can be used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
