Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/19620 )
Change subject: KUDU-3459 Download superblock piece by piece ...................................................................... Patch Set 4: (6 comments) http://gerrit.cloudera.org:8080/#/c/19620/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19620/2//COMMIT_MSG@16 PS2, Line 16: wnload superblock piece by piece. A new > 1. You mean if found the error, transfer into using tablet_copy_download_su 1. If found superblock size is larlger than --rpc_max_message_size, download the superblock data by pieces automatically, it's not needed to wait an error occurs. 2. Thanks for the clarification. http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tools/kudu-tool-test.cc File src/kudu/tools/kudu-tool-test.cc: http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tools/kudu-tool-test.cc@9158 PS3, Line 9158: TestDownloadSuperblockInBatch > It has been checked. Please see line 9176. I mean how to check the superblock is downloaded by pieces actually? http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tserver/tablet_copy_client.cc File src/kudu/tserver/tablet_copy_client.cc: http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tserver/tablet_copy_client.cc@325 PS3, Line 325: if (!superblock_path.empty() && : dst_fs_manager_->env()->FileExists(superblock_path)) { > No. Superblock maybe download failed. Then the file of superblock_path will If download failed, it would return in L320, right? http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tserver/tablet_copy_client.cc@334 PS3, Line 334: path, > Superblock is encrypted default. Please see src/kudu/tablet/tablet_metadata If --encrypt_data_at_rest is not set, the superblock file will not be encrypted too. I mean is there a test to check "download super block piece by piece" when --encrypt_data_at_rest is set? http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tserver/tablet_copy_client.cc@928 PS3, Line 928: > I have added a MakeScopedCleanup to delete the file, please see line 324. If DownloadSuperBlock failed and returned in L320, MakeScopedCleanup in L324 will not happen. http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tserver/tablet_copy_source_session.cc File src/kudu/tserver/tablet_copy_source_session.cc: http://gerrit.cloudera.org:8080/#/c/19620/3/src/kudu/tserver/tablet_copy_source_session.cc@322 PS3, Line 322: string path = fs_manager_->GetTabletMetadataPath(tablet_id_); > But how to read tablet_superblock_ piece by piece? You can dump the in-memory struct to a temporary file when decide to download it by pieces. -- To view, visit http://gerrit.cloudera.org:8080/19620 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3a7c212784383e247566a4701965e2897de83303 Gerrit-Change-Number: 19620 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <[email protected]> Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Comment-Date: Tue, 27 Jun 2023 13:15:59 +0000 Gerrit-HasComments: Yes
