Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 )
Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs ...................................................................... Patch Set 39: (7 comments) http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java File fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java: http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@924 PS38, Line 924: *isSk > nit: need a comment of "/*isSkipTableMetadataReload*/" to be consistent wit Ack http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@537 PS38, Line 537: return new AlterPartitionEvent(catalogOpExecutor_, metrics, event) { > Could you share how we will go here? NotificationEvent "event" comes from H This is not required since we are not doing the batching. http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1792 PS38, Line 1792: "So table schema reload can be skipped. SD before: {}, SD after: {}", > I'm thinking if we could split these into a separate patch for IMPALA-12487 Unfortunately, few tests I introduced in this patch are dependent on this logic. I'll make sure to backport this logic to older branches along with IMPALA-12487 http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@390 PS38, Line 390: Preconditions.checkNotNull(currentEvent.getDbName()); : Preconditions.checkNotNull(currentEvent.getTab > Shouldn't we use 'filteredEvents' instead? Will add a TODO comment with a follow-up Jira to address the batching logic while syncing events. http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1139 PS38, Line 1139: tbl will n > nit: "Hdfs table" might be misunderstood as the real table on HDFS. Maybe u Ack http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2644 PS38, Line 2644: > We won't reload the file metadata now since we have reverted the changes in Ack http://gerrit.cloudera.org:8080/#/c/20367/38/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@3230 PS38, Line 3230: _on_ddls=true for truncate operati > Could you turn this into a TODO comment with the Hive JIRA for skip droppin Ack -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 39 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Tue, 06 Feb 2024 00:33:52 +0000 Gerrit-HasComments: Yes
