[email protected] has posted comments on this change. ( http://gerrit.cloudera.org:8080/21045 )
Change subject: IMPALA-12771: Impala catalogd events-skipped may mark the wrong number ...................................................................... Patch Set 9: (9 comments) http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1764 PS8, Line 1764: // Only bump the skipped metric if the old table is not removed and the new table : // is not added. Not doing this in other cases since we need to either remove the : // old table or add the new table, which is processing the event. : if (!oldTblRemoved.getRef() && ! > nit: might be more clear if reword this to Done http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1811 PS8, Line 1811: infoLog("Not processing the alter table event {} as it is an older event", > Should we also bump the skipped metric in this case and L1819 for isSelfEve No, the function has bumped the skipped metric before the function returned true for MetastoreTableEvent see :https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java#L1272 .And the same processing logic also occurs in the isSelfEvent function. See : https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java#L918 http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1842 PS8, Line 1842: metrics_.getCounter(MetastoreEventsProcessor.EVENTS_SKIPPED_METRIC).inc(); > Shouldn't this be !notSkipped? yes, you are right,I made a mistake. http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@241 PS8, Line 241: be > nit: remove period "." and use lowercase "b". This should be the same sente Done http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4879 PS8, Line 4879: return 0; > Could you add a comment about this? Why should we use -1 here? sorry, this should be 0, which is a merge conflict and the conflict is not detect by test. http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@5018 PS8, Line 5018: , > nit: add a space after comma "," Done http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java File fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java: http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@1995 PS8, Line 1995: // is IncompleteTable and addPartition event may also skip. > Do you mean sometimes we could see more events being skipped, e.g. 3 or 4? yes http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@2033 PS8, Line 2033: TEventProcessorMetricsSummaryResponse summaryResponse = : catalog_.getEventProcessorSummary(); : assertNotNull(summaryResponse); : assertTrue(response.getLast_synced_event_id() > lastEven > These two are unrelative to the skipped metric. I think we can remove them. ok http://gerrit.cloudera.org:8080/#/c/21045/8/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@2043 PS8, Line 2043: // event 5 alter partition > nit: remove semicolon ";" and reword to "ALTER events will be skipped." Done -- To view, visit http://gerrit.cloudera.org:8080/21045 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7aeb04e999b82187eb138c0b643ead259da22f1a Gerrit-Change-Number: 21045 Gerrit-PatchSet: 9 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Tue, 18 Jun 2024 15:26:15 +0000 Gerrit-HasComments: Yes
