Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 )
Change subject: IMPALA-10976: Sync db/table to latest HMS event for all DDL/DMLs ...................................................................... Patch Set 15: (2 comments) http://gerrit.cloudera.org:8080/#/c/20367/15/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/20367/15/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@376 PS15, Line 376: for (NotificationEvent event : events) { I think we can optimize this later in a separate JIRA to make sure the table will only be reloaded at most once. http://gerrit.cloudera.org:8080/#/c/20367/14/tests/custom_cluster/test_sync_to_latest_hms_events.py File tests/custom_cluster/test_sync_to_latest_hms_events.py: http://gerrit.cloudera.org:8080/#/c/20367/14/tests/custom_cluster/test_sync_to_latest_hms_events.py@37 PS14, Line 37: --file_metadata_reload_properties='' > This is a real problem with queries involving the 'Insert or Insert overwri I tried reproducing this and found we should run a query on this table in Impala after creating it (so it's in loaded state). It seems a bug that we update lastSyncedEventId too much beyound what it should be (so the INSERT is incorrectly skipped). I think if we just reload the table schema and without reloading the file metadata, we shouldn't update lastSyncedEventId to the latest event id. -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 15 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Thu, 14 Dec 2023 12:31:11 +0000 Gerrit-HasComments: Yes
