Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21662 )
Change subject: IMPALA-12758: Event processor is ignoring the prev_id while reloading existing partitions ...................................................................... Patch Set 5: Code-Review+1 (7 comments) The new test looks good to me. Just have some minor comments. http://gerrit.cloudera.org:8080/#/c/21662/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21662/5//COMMIT_MSG@7 PS5, Line 7: IMPALA-12758: Event processor is ignoring the prev_id while reloading Please update the commit message. This is not only used by the event-processor. Maybe "Fix catalogd not setting prev_id for reloaded partitions" is better. The commit message also needs to be updated. http://gerrit.cloudera.org:8080/#/c/21662/5//COMMIT_MSG@12 PS5, Line 12: state : store nit: "statestore" http://gerrit.cloudera.org:8080/#/c/21662/5//COMMIT_MSG@13 PS5, Line 13: impalads will : not know whether to invalidate the current partition and then request : the new partition information. It's unclear to me what do we mean by "request the new partition information". Coordinator won't send requests when processing statestore updates. http://gerrit.cloudera.org:8080/#/c/21662/1/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/21662/1/tests/custom_cluster/test_events_custom_configs.py@1324 PS1, Line 1324: self.client.execute("select * from {}".format(test_tbl)) > In the local catalog mode, I have created a partitioned table and loaded th Ack http://gerrit.cloudera.org:8080/#/c/21662/5/tests/custom_cluster/test_local_catalog.py File tests/custom_cluster/test_local_catalog.py: http://gerrit.cloudera.org:8080/#/c/21662/5/tests/custom_cluster/test_local_catalog.py@580 PS5, Line 580: hms_event_polling_interval_s=5 If the default interval doesn't work, let's set this to 0 to avoid flakiness. http://gerrit.cloudera.org:8080/#/c/21662/5/tests/custom_cluster/test_local_catalog.py@587 PS5, Line 587: self.client.execute("show partitions {}".format(test_tbl)) Please add a comment to explain the purpose of this, e.g. "Make the partition loaded in the coordinator, so the one instance needs to be invalidated when the partition is reloaded." http://gerrit.cloudera.org:8080/#/c/21662/5/tests/custom_cluster/test_local_catalog.py@589 PS5, Line 589: log_regex = r"Invalidated objects in cache: \[partition %s:p=\d \(id=%%d\)\]" \ nit: change "p=\d" to "p=0" and change "%%d" to "0" Then we can change "log_regex % 0" to "log_regex" -- To view, visit http://gerrit.cloudera.org:8080/21662 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia7d53f601b63d83e99cbd2d1f58076a0dc78c610 Gerrit-Change-Number: 21662 Gerrit-PatchSet: 5 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Tue, 15 Oct 2024 00:14:01 +0000 Gerrit-HasComments: Yes
