liuyao has posted comments on this change. ( http://gerrit.cloudera.org:8080/17645 )
Change subject: IMPALA-5476: Fix Catalogd restart bring about metadata is out of sync ...................................................................... Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17645/4/tests/custom_cluster/test_restart_services.py File tests/custom_cluster/test_restart_services.py: http://gerrit.cloudera.org:8080/#/c/17645/4/tests/custom_cluster/test_restart_services.py@189 PS4, Line 189: # Wait enough time to synchronize the metadata : sleep(20) > We should not depend on sleep. This is what I mean in the previous > comment: https://gerrit.cloudera.org/c/17645/1//COMMIT_MSG#17 > > When AlterTable finishes, the metadata of this table should be > up-to-date. The current patch can't satify this. I think of three solutions: Solution 1. Directly trigger the full topic update, but you have to wait until the full update is completed before you can update the local cache, because the local cache has a higher version. But I did not find the api to trigger the full update, ImpalaServer::CatalogUpdateCallback is not applicable Solution 2. When restart catalogs, delay DDL execution time. After the ddl is executed, determine whether the catalog service id of the returned result is the same as the local one. If it is different, wait for the local catalog service id to change, and then return success Solution 3. When call exec_env_->frontend()->UpdateCatalogCache, if the catalog server id is different, the catalog object will be updated directly without comparing the versions -- To view, visit http://gerrit.cloudera.org:8080/17645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9fe25f5a2a42fb432e306ef08ae35750c8f3c50c Gerrit-Change-Number: 17645 Gerrit-PatchSet: 4 Gerrit-Owner: liuyao <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: liuyao <[email protected]> Gerrit-Comment-Date: Fri, 16 Jul 2021 03:20:31 +0000 Gerrit-HasComments: Yes
