Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21045 )
Change subject: IMPALA-12771: Impala catalogd events-skipped may mark the wrong number ...................................................................... Patch Set 13: (1 comment) http://gerrit.cloudera.org:8080/#/c/21045/11/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java File fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java: http://gerrit.cloudera.org:8080/#/c/21045/11/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@1995 PS11, Line 1995: response.getEvents_skipped() >= numEventsSkippedBefore + 2); > we can see function https://github.com/apache/impala/blob/master/fe/src/mai I ran the test locally using the following command and check the logs in logs/fe_tests/FeSupport.INFO (pushd fe && mvn test -Dtest=MetastoreEventsProcessorTest#testEventProcessorMetrics) Based on the logs, 3 events are skipped: I0716 15:14:52.353045 17699 MetastoreEvents.java:302] Total number of events received: 5 Total number of events filtered out: 0 I0716 15:14:52.356055 17699 MetastoreEvents.java:825] EventId: 9102918 EventType: CREATE_DATABASE Successfully added database events_test_db I0716 15:14:52.356364 17699 MetastoreEvents.java:825] EventId: 9102919 EventType: CREATE_TABLE Found table level flag impala.disableHmsSync is set to true for table events_test_db.tbl_should_skipped I0716 15:14:52.356426 17699 MetastoreEvents.java:825] EventId: 9102919 EventType: CREATE_TABLE Skipping this event because of flag evaluation I0716 15:14:52.366181 17699 MetastoreEvents.java:825] EventId: 9102920 EventType: CREATE_TABLE Successfully added table events_test_db.testeventprocessormetrics I0716 15:14:52.366394 17699 MetastoreEvents.java:825] EventId: 9102921 EventType: ADD_PARTITION Found table level flag impala.disableHmsSync is set to true for table events_test_db.tbl_should_skipped I0716 15:14:52.366451 17699 MetastoreEvents.java:825] EventId: 9102921 EventType: ADD_PARTITION Skipping this event because of flag evaluation I0716 15:14:52.367728 17699 CatalogOpExecutor.java:4715] EventId: 9102922 Table events_test_db.testeventprocessormetrics is not loaded. Skipping add partitions The CREATE_DATABASE is not skipped. The first CREATE_TABLE event is skipped due to table property impala.disableHmsSync=true. The second CREATE_TABLE event is not skipped. The first ADD_PARTITION event is skipped due to table property impala.disableHmsSync=true. The second ADD_PARTITION is skipped due to the table is not loaded. Let's modify the test to be assertEquals("3 events should be skipped", response.getEvents_skipped(), numEventsSkippedBefore + 3); And update the comment to explain this. -- To view, visit http://gerrit.cloudera.org:8080/21045 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7aeb04e999b82187eb138c0b643ead259da22f1a Gerrit-Change-Number: 21045 Gerrit-PatchSet: 13 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Tue, 16 Jul 2024 07:27:22 +0000 Gerrit-HasComments: Yes
