Vihang Karajgaonkar has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15648 )
Change subject: IMPALA-8632: Add support for self-event detection for insert events ...................................................................... IMPALA-8632: Add support for self-event detection for insert events In case of INSERT_EVENTS if Impala inserts into a table it causes a refresh to the underlying table/partition. This could be unnecessary when there is only one Impala cluster in the system. We can detect a self-event in such cases when the HMS API to fire a listener event returns the event id. This is used by EventProcessor to ignore the event when it is fetched later in the next polling cycle. Testing: Add testInsertFromImpala() in MetastoreEventsProcessorTest.java to test insert event self-event detection when insert into table and partition. Change-Id: I7873fbb2c159343690f93b9d120f6b425b983dcf Reviewed-on: http://gerrit.cloudera.org:8080/15648 Reviewed-by: Vihang Karajgaonkar <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/events/InFlightEvents.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/main/java/org/apache/impala/catalog/events/SelfEventContext.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M tests/custom_cluster/test_event_processing.py 17 files changed, 504 insertions(+), 171 deletions(-) Approvals: Vihang Karajgaonkar: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15648 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7873fbb2c159343690f93b9d120f6b425b983dcf Gerrit-Change-Number: 15648 Gerrit-PatchSet: 10 Gerrit-Owner: Xiaomeng Zhang <[email protected]> Gerrit-Reviewer: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Xiaomeng Zhang <[email protected]>
