Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20427 )
Change subject: IMPALA-12399: Add filter to skip OPEN_TXN events from HMS ...................................................................... IMPALA-12399: Add filter to skip OPEN_TXN events from HMS Notification events like OPEN_TXN are ignored on catalogd MetastoreEventsProcessor. So, we can pass eventTypeSkipList with OPEN_TXN in NotificationEventRequest while invoking get_next_notification() to avoid reading such notification messages from metastore and then ignoring on catalogd. OPEN_TXN event being more frequent(received even upon describe table operation from beeline), we can significantly reduce unwanted processing on both hive metastore and catalogd. Catalogd reads events in batches of EVENTS_BATCH_SIZE_PER_RPC, skipping such unnecessary events can help catchup the events faster. Testing: - Manually tested in cluster and added a testcase Change-Id: Id2f2e1040abce0b00a8a07bfe8b46afcd98290bf Reviewed-on: http://gerrit.cloudera.org:8080/20427 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Sai Hemanth Gantasala <[email protected]> Reviewed-by: Quanlong Huang <[email protected]> --- M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java 3 files changed, 23 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Verified Sai Hemanth Gantasala: Looks good to me, but someone else must approve Quanlong Huang: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/20427 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id2f2e1040abce0b00a8a07bfe8b46afcd98290bf Gerrit-Change-Number: 20427 Gerrit-PatchSet: 7 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
