Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/23159 )
Change subject: IMPALA-14082: Support batch processing of RELOAD events on same table ...................................................................... Patch Set 11: (4 comments) http://gerrit.cloudera.org:8080/#/c/23159/6/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/23159/6/tests/custom_cluster/test_events_custom_configs.py@578 PS6, Line 578: > Thanks for the explanation! Could you please add a comment to summarize the Summary of this discussion: BatchPartitionEvent#isOlderEvent() requires enable_skipping_older_events=true to skip older reload events. http://gerrit.cloudera.org:8080/#/c/23159/7/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/23159/7/tests/custom_cluster/test_events_custom_configs.py@631 PS7, Line 631: # Test to verify if older events from hive are being skipped in event processor. > It's not just about naming. We set this to true in test_refresh_invalidate_ Yeah, you are right. The test would fail when --enable_sync_to_latest_event_on_ddls=false and enable_sync_to_latest_event_on_ddls=True. It because https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L3221 we won't fetch latest_hms_event_id when --enable_sync_to_latest_event_on_ddls=false, so we update the latestRefreshEventId on the table as current event id. So we end up processing all the events and fails at L#643 (table level refresh fails). http://gerrit.cloudera.org:8080/#/c/23159/10/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/23159/10/tests/custom_cluster/test_events_custom_configs.py@652 PS10, Line 652: self.hive_client.fire_listener_event(req) > What if the 10 RELOAD events arrive and be processed before this REFRESH st Yeah, it would fail. I have added pause, start commands for EP http://gerrit.cloudera.org:8080/#/c/23159/10/tests/custom_cluster/test_events_custom_configs.py@846 PS10, Line 846: # p=0 has two values 0, 4 and p=2 has value 1 > add "order by i" to make the results determined. Ack -- To view, visit http://gerrit.cloudera.org:8080/23159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie3e9a99b666a1c928ac2a136bded1e5420f77dab Gerrit-Change-Number: 23159 Gerrit-PatchSet: 11 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Mon, 15 Sep 2025 22:10:30 +0000 Gerrit-HasComments: Yes
