liuyao has posted comments on this change. ( http://gerrit.cloudera.org:8080/17645 )
Change subject: IMPALA-5476: Fix Catalogd restart bring about metadata is out of sync ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/17645/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17645/1//COMMIT_MSG@16 PS1, Line 16: but did not submit full topic update : request > Shouldn't we fix this instead? The new catalog objects in the DDL > response come from the new catalogd instance. Will they be able to > update the local catalog cache? Or are they just got ignored due to > versions too small? we should fix this bug.The new catalog objects in the DDL response come from the new catalogd instance, they just got ignored due to versions too small > > I think updating the local catalog cache immediately using the DDL > response aims to let the coordinator get latest catalog > immediately. After this patch, the coordinator will need to wait > for the next statestore topic update to get the latest catalog, > which doesn't match the existing functionality. Did I miss > anything? This patch do not change the previous process. After the ddl is successfully executed, impalad will immediately modify the local catalog cache instead of waiting for the statestore notification. When the ddl is executed successfully, it is reasonable for impalad to modify the local cache, but it is redundant to modify the service id every time, and it will cause the object version to be very high, causing the local catalog cache to be out of sync http://gerrit.cloudera.org:8080/#/c/17645/1//COMMIT_MSG@24 PS1, Line 24: CatalogD > CatalogD? Done http://gerrit.cloudera.org:8080/#/c/17645/1/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/17645/1/be/src/service/impala-server.cc@a2133 PS1, Line 2133: : : : > Is a better solution to set the catalogServiceId_ only if firstRun is true Modifying the server id here may be redundant. Is it necessary to do so? -- To view, visit http://gerrit.cloudera.org:8080/17645 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9fe25f5a2a42fb432e306ef08ae35750c8f3c50c Gerrit-Change-Number: 17645 Gerrit-PatchSet: 2 Gerrit-Owner: liuyao <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: liuyao <[email protected]> Gerrit-Comment-Date: Tue, 13 Jul 2021 03:30:42 +0000 Gerrit-HasComments: Yes
