Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17645 )
Change subject: IMPALA-5476: Fix Catalogd restart bring about metadata is out of sync ...................................................................... Patch Set 14: (1 comment) http://gerrit.cloudera.org:8080/#/c/17645/10/tests/custom_cluster/test_restart_services.py File tests/custom_cluster/test_restart_services.py: http://gerrit.cloudera.org:8080/#/c/17645/10/tests/custom_cluster/test_restart_services.py@208 PS10, Line 208: LOG.info(str(e)) : if i == 5: : self.cluster.catalogd.restart() : > I tried many times, but the WARNING log was not printed. It may be because statestored will send topic update request to catalogd once it's registered. I ran the test and realized that we restart catalogd too quickly, i.e. catalogd hasn't been ready after restart but we restart it again. So impalad is actually talking to the second restarted catalogd. The problem is restart() returns too early. Connected to statestored doesn't mean catalogd is ready: https://github.com/apache/impala/blob/ca17e307ab3abb2c95c27b3ba749adf6bf16efc7/tests/common/impala_cluster.py#L569-L571 We probably need to add a metric like "catalog-server.ready" to indicate the start is finish. -- To view, visit http://gerrit.cloudera.org:8080/17645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9fe25f5a2a42fb432e306ef08ae35750c8f3c50c Gerrit-Change-Number: 17645 Gerrit-PatchSet: 14 Gerrit-Owner: liuyao <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 05 Aug 2021 03:28:10 +0000 Gerrit-HasComments: Yes
