Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21665 )
Change subject: IMPALA-12865: Fix wrong lastRefreshEventId set by firing RELOAD events ...................................................................... Patch Set 22: (5 comments) http://gerrit.cloudera.org:8080/#/c/21665/22/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/21665/22/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@3444 PS22, Line 3444: else { nit: don't need "else" http://gerrit.cloudera.org:8080/#/c/21665/20/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/21665/20/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7576 PS20, Line 7576: LOG.warn("Partition {} no longer exists in table {}. It might be " + > Ack Can we add a test for this scenario, e.g. using a REFRESH on multiple partitions (IMPALA-14089)? http://gerrit.cloudera.org:8080/#/c/21665/22/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/21665/22/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7569 PS22, Line 7569: partitionChanged This depends on whether other partitions are updated. Should't we use the return result of updatePartition() directly? i.e. if (hdfsTbl.updatePartition(partBuilder)) { http://gerrit.cloudera.org:8080/#/c/21665/22/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7573 PS22, Line 7573: partBuilder 'partBuilder' is a temp object and its update won't be finalized unless calling HdfsTable.updatePartition(). I think we should update the actual HdfsPartition instance here. if (hdfsTbl.updatePartition(partBuilder)) { partitionChanged = true; // Get the updated partition partition = hdfsTbl.getPartitionFromThriftPartitionSpec(partSpecList.get(i)); Preconditions.checkNotNull(partition, "Partition is null after update"); } partition.addToVersionsForInflightEvents(false, newCatalogVersion); http://gerrit.cloudera.org:8080/#/c/21665/22/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/21665/22/tests/custom_cluster/test_events_custom_configs.py@1329 PS22, Line 1329: Can we add another test similar to the above one but using a Hive statement not on the current table? So we can verify that the RELOAD events are skipped by the in-flight event list. Note that in the above test, the lastRefreshEventId is updated when processing the ALTER_PARTITION event. So the RELOAD event is actually skipped by lastRefreshEventId, not by the in-flight event list. -- To view, visit http://gerrit.cloudera.org:8080/21665 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I90039da77ec561c5aede44456f88c6650582815b Gerrit-Change-Number: 21665 Gerrit-PatchSet: 22 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Pranav Lodha <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Wed, 17 Dec 2025 11:44:54 +0000 Gerrit-HasComments: Yes
