Yingchun Lai has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19479 )
Change subject: [KUDU-3447] Limit tablets copying speed ...................................................................... [KUDU-3447] Limit tablets copying speed Copying tablets from one cluster to another is a high resource consumed operation using the command : kudu local_replica copy_from_remote. If the data size is very large, the copying process will last for a long time. Other service maybe get impacted and become unavailable because of the tablets copying process cost too much disk and/or network bandwith. Therefore it is better to limit the tablets copying speed and make the system more stable. The goal is a trade-off the tablets copying speed and the resource consumption. As copy_from_remote is mainly downloading data from the remote cluster and writing the data to local file system, it is better to control the downloading speed to control the resource consumption. This patch use a throttler to limit tablet copying speed. Two paramters are added: --tablet_copy_throttler_bytes_per_sec limits the copying speed, and --tablet_copy_throttler_burst_factor limits the maximum copying speed at a single time. Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 Reviewed-on: http://gerrit.cloudera.org:8080/19479 Reviewed-by: Yingchun Lai <[email protected]> Tested-by: Yingchun Lai <[email protected]> --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_local_replica.cc M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h 5 files changed, 155 insertions(+), 12 deletions(-) Approvals: Yingchun Lai: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 Gerrit-Change-Number: 19479 Gerrit-PatchSet: 24 Gerrit-Owner: Wang Xixu <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <[email protected]> Gerrit-Reviewer: Yifan Zhang <[email protected]> Gerrit-Reviewer: Yingchun Lai <[email protected]>
