subject:"\[kudu\-CR\] \[KUDU\-3447\] Limit tablets copying speed"

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-30 Thread Yingchun Lai (Code Review)

Yingchun Lai has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Reviewed-on: http://gerrit.cloudera.org:8080/19479
Reviewed-by: Yingchun Lai 
Tested-by: Yingchun Lai 
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 155 insertions(+), 12 deletions(-)

Approvals:
  Yingchun Lai: Looks good to me, approved; Verified

--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 24
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-28 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 23: Verified+1 Code-Review+2

The failed test is not related:
LeaderRebalancerTest.AddTserver


--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 23
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Sat, 28 Oct 2023 15:00:43 +
Gerrit-HasComments: No

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-28 Thread Yingchun Lai (Code Review)

Yingchun Lai has removed a vote on this change.

Change subject: [KUDU-3447] Limit tablets copying speed
..


Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 23
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-27 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 23: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 23
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Fri, 27 Oct 2023 10:11:08 +
Gerrit-HasComments: No

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-27 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 23:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19479/22/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/22/src/kudu/tserver/tablet_copy_client.cc@91
PS22, Line 91: --tablet_copy_throttler_bytes_per_s
> nit: "--tablet_copy_throttler_bytes_per_sec"
Done



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 23
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Fri, 27 Oct 2023 10:08:54 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-27 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#23).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 155 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/23
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 23
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-27 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 22:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19479/22/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/22/src/kudu/tserver/tablet_copy_client.cc@91
PS22, Line 91: tablet_copy_throttler_bytes_per_sec
nit: "--tablet_copy_throttler_bytes_per_sec"



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 22
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Fri, 27 Oct 2023 09:26:16 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-25 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 22:

(20 comments)

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9360
PS20, Line 9360:
> nit for here and elsewhere in this new test scenario: introducing a variabl
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9377
PS20, Line 9377:
> nit: maybe, drop this extra space?
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9381
PS20, Line 9381:
> style nit: please move these parameters into their own line(s)
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9390
PS20, Line 9390: _cluster_->mini_tablet_server(target_idx)->Start());
   :   const vector& target_tablet_ids
> nit: probably, using FindOrNull() from map-util.h would be more readable?
target_tablet_ids is a vector.


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@347
PS11, Line 347:
> That NO_FATALS() in TabletCopyClientTest::SetUp() around TabletCopyTest::Se
Got it. Thanks for your explaination.


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@335
PS20, Line 335: table
> nit: does 'tool' is relevant in this test?
Change it to 'tablet-copy-test'.


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@337
PS20, Line 337: ASSERT_OK(ResetRemoteTabletCopyClient(_copy_client
> I meant 'consider wrapping this into ASSERT_OK()'
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@356
PS20, Line 356: must
> nit: must be
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@357
PS20, Line 357: FLAGS_tablet_copy_transfer_chunk_size_bytes
> This doesn't seem to be or right dimension, no?  Shouldn't the time that ha
Please see line 324, I have set the tablet copying speed to 
FLAGS_tablet_copy_transfer_chunk_size_bytes.


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@361
PS20, Line 361: 'client_
> nit: 'client_'
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@361
PS20, Line 361: d
> nit: remove this single quote symbol?
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@362
PS20, Line 362: d u
> uses
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@362
PS20, Line 362: the pointer
> a pointer to
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@363
PS20, Line 363: while being destr
> while being destroyed
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@363
PS20, Line 363: t.cc.
> It's going to be confusing in the future when another change in tablet_copy
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.h@328
PS20, Line 328:  pub
> style nit: the indent should be 2 spaces
Done


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.h@321
PS11, Line 321: // The throttler_ is shared among mult
> Great, then please add a comment for this field, please:
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.cc@84
PS20, Line 84: limit
> nit: limits
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.cc@85
PS20, Line 85: not limiting "
 :  "the speed
> nit: 'not limiting the speed' or 'there isn't any limit'
Done


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.cc@87
PS20, Line 87:
> Does 0 represent a special value for the semantics of this flag?  If so, pl
OK. The default value of tablet_copy_throttler_burst_factor should be better to 
be 1.0, which means the maximun rate is equals
"to tablet_copy_throttler_bytes_per_sec". I will change it.



--
To view, visit

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-25 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#22).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 161 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/22
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 22
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-24 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#21).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 160 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/21
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 21
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-24 Thread Alexey Serbin (Code Review)

Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 20:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9360
PS20, Line 9360: mini_cluster_->mini_tablet_server(0)
nit for here and elsewhere in this new test scenario: introducing a variable 
for mini_cluster_->mini_tablet_server(idx) would help to make this code more 
readable


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9377
PS20, Line 9377:
nit: maybe, drop this extra space?


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@337
PS20, Line 337: ResetRemoteTabletCopyClient(_copy_client_metrics);
> What this into ASSERT_OK()?
I meant 'consider wrapping this into ASSERT_OK()'



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 20
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Tue, 24 Oct 2023 23:02:02 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-24 Thread Alexey Serbin (Code Review)

Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 20:

(19 comments)

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9381
PS20, Line 9381: nullptr, )
style nit: please move these parameters into their own line(s)


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tools/kudu-tool-test.cc@9390
PS20, Line 9390: find(target_tablet_ids.begin(), target_tablet_ids.end(), 
tablet_id)
   : != target_tablet_ids.end()
nit: probably, using FindOrNull() from map-util.h would be more readable?


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@337
PS11, Line 337:  ResetRemoteTabletCopyClient(_copy_client_metrics);
  :
  :   ASSERT_OK(StartCopy());
  :   BlockId block_id = 
FirstColumnBlockId(*client_->remote_superblock_);
  :   Slice slice;
  :   faststring scratch;
  :
> Sub-class can not initialize the member filed of parent class in the initia
Indeed -- for some reason I assumed these fields were from the 
TabletCopyThrottlerTest class.


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@347
PS11, Line 347:
> It does not. Please see the function SetUp() in class TabletCopyClientTest,
That NO_FATALS() in TabletCopyClientTest::SetUp() around 
TabletCopyTest::SetUp() is to bail out of the function/method if error happened 
in TabletCopyTest::SetUp(), not continue executing the rest of the code of the 
TabletCopyClientTest::SetUp() method.

As I could see, gtest checks on the errors raised during SetUp() on itself, and 
if any error present, no further phases after SetUp() are run and test 
considered failed.  With that, NO_FATALS() isn't needed here since there isn't 
anything else in this SetUp() method except for the call of 
TabletCopyClientTest::SetUp().


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@335
PS20, Line 335: tool-
nit: does 'tool' is relevant in this test?


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@337
PS20, Line 337: ResetRemoteTabletCopyClient(_copy_client_metrics);
What this into ASSERT_OK()?


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@356
PS20, Line 356: must
nit: must be


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@357
PS20, Line 357: FLAGS_tablet_copy_transfer_chunk_size_bytes
This doesn't seem to be or right dimension, no?  Shouldn't the time that has 
been spent also be accounted for here?


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@361
PS20, Line 361: 'client_
nit: 'client_'


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@361
PS20, Line 361: '
nit: remove this single quote symbol?


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@362
PS20, Line 362: use
uses


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@362
PS20, Line 362: the point of
a pointer to


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@363
PS20, Line 363: when be destroyed
while being destroyed


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client-test.cc@363
PS20, Line 363:  line 231.
It's going to be confusing in the future when another change in 
tablet_copy_client.cc updates the code and the line number changes.  Please 
don't refer to the line numbers like this.


http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/20/src/kudu/tserver/tablet_copy_client.h@328
PS20, Line 328:
style nit: the indent should be 2 spaces


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.h@321
PS11, Line 321: std::shared_ptr throttler_;
> It is shared by multiple tablet copying tasks in the same session.
Great, then please add a comment for this field, please:

'The throttler_ is shared among multiple tablet copying tasks in the same 
session.'

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-23 Thread Yingchun Lai (Code Review)

Yingchun Lai has removed a vote on this change.

Change subject: [KUDU-3447] Limit tablets copying speed
..


Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 20
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-23 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 20: Verified+1 Code-Review+2

The failed tests are not related:
ClientTest.TestDeleteWithDeletedTableReserveSecondsWorks
TsTabletManagerITest.TestTableStats


--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 20
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Tue, 24 Oct 2023 02:29:29 +
Gerrit-HasComments: No

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-20 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#20).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 154 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/20
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 20
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-20 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#19).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 146 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/19
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 19
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-18 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 18: Code-Review+1

The failed test is related.


--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 18
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Wed, 18 Oct 2023 08:42:57 +
Gerrit-HasComments: No

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-18 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 18: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 18
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Wed, 18 Oct 2023 08:42:21 +
Gerrit-HasComments: No

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-16 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 18:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/tool_action_local_replica.cc@334
PS17, Line 334: shared_ptr throttler;
  : if (FLAGS_tablet_copy_throttler_bytes_per_sec > 0) {
  :   throttler = std::make_shared(MonoTime::Now(),
  :   0,
  :   
FLAGS_tablet_copy_throttler_bytes_per_sec,
  :   
FLAGS_tablet_copy_throttler_burst_factor);
  : }
  :
> I doubt different business has different limiting speed. Can replicas migra
We can disable this feature by default, but it would be great to provide the 
ability.
In the senario of Kudu servers are deployed with other IO sensitive servers, it 
would be nice to limit the disk bandwith consumption.
Of cource, you can do this in next patches.



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 18
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Mon, 16 Oct 2023 16:13:32 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-16 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 18:

(16 comments)

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/kudu-tool-test.cc@9384
PS10, Line 9384:   ASSERT_STR_CONTAINS(stderr, "Time spent Tablet copy 
throttler");
> I don't think we should add a new test just to verify the output message or
Done


http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/kudu-tool-test.cc@9361
PS17, Line 9361:   ASSERT_GT(source_tserver_tablet_count, 0);
> Better to use ASSERT_GT(), the actual value of source_tserver_tablet_count
Done


http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/kudu-tool-test.cc@9381
PS17, Line 9381: 10 * FLAGS_tablet_copy_transfer_chunk_size_byt
> nit: leave a space on both left and right sides of '*'.
Done


http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/tool_action_local_replica.cc@1386
PS10, Line 1386: .AddOptionalParameter("tablet_copy_throttler_bytes_per_sec")
   :   
.AddOptionalParameter("tablet_copy_throttler_burst_factor")
> nit: It's necessary to mention these two parameters in the commit message.
Done


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tools/tool_action_local_replica.cc@336
PS11, Line 336: throttler = std::make_shared(MonoTi
> Please use std::make_shared() here instead.
Done


http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/tool_action_local_replica.cc@334
PS17, Line 334: shared_ptr throttler;
  : if (FLAGS_tablet_copy_throttler_bytes_per_sec > 0) {
  :   throttler = std::make_shared(MonoTime::Now(),
  :   0,
  :   
FLAGS_tablet_copy_throttler_bytes_per_sec,
  :   
FLAGS_tablet_copy_throttler_burst_factor);
  : }
  :
> The throttler is only used in CLI tool? How about create a throttler intern
I doubt different business has different limiting speed. Can replicas 
migration/auto rebalancing share the same throttler?


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@259
PS11, Line 259:
> nit: remove the extra spaces
Done


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@270
PS11, Line 270:
  :   RaftPeerPB* cstate_leader;
  :   ConsensusStatePB cstate;
  :   
RETURN_NOT_OK(tablet_replica_->consensus()->ConsensusState());
  :   RETURN_NOT_OK(GetRaftConfigLeader(, _leader));
  :   leader_ = *cstate_leader;
  :   return Status::OK();
  : }
  :
  : Status TabletCopyClientTest::ResetLocalTabletCopyClient() {
  :   // client_ will be reset many times in test cases, we only 
shutdown mini_server_
  :   // at the first time, and will not start it again.
  :   if (mini_server_->is_started()) {
  :
> Is it possible to pass tablet_copy_client_metrics_.get() and  throttler_ as
Done


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@337
PS11, Line 337:  ASSERT_OK(StartCopy());
  :   BlockId block_id = 
FirstColumnBlockId(*client_->remote_superblock_);
  :   Slice slice;
  :   faststring scratch;
  :
  :   // Ensure the block wasn't there before (it shouldn't be, we 
use our own FsManager dir).
  :   Status s = ReadLocalBlockFile(fs_manager_.get(), block_id, 
, &
> Is it possible to move this into the initializers' list?
Sub-class can not initialize the member filed of parent class in the 
initializers list of sub-class. it will throw error:
error: class ‘kudu::tserver::TabletCopyThrottlerTest’ does not have any field 
named ‘mode_’.


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@347
PS11, Line 347: ockId new_
> I'm not sure this is needed

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-16 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#18).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 147 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/18
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 18
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-12 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 17:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/kudu-tool-test.cc@9361
PS17, Line 9361:   ASSERT_TRUE(source_tserver_tablet_count > 0);
Better to use ASSERT_GT(), the actual value of source_tserver_tablet_count will 
be output if the ASSERT_* failed.


http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/kudu-tool-test.cc@9381
PS17, Line 9381: 10*FLAGS_tablet_copy_transfer_chunk_size_bytes
nit: leave a space on both left and right sides of '*'.


http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/17/src/kudu/tools/tool_action_local_replica.cc@334
PS17, Line 334: shared_ptr throttler;
  : if (FLAGS_tablet_copy_throttler_bytes_per_sec > 0) {
  :   throttler = std::make_shared(MonoTime::Now(),
  :   0,
  :   
FLAGS_tablet_copy_throttler_bytes_per_sec,
  :   
FLAGS_tablet_copy_throttler_burst_factor);
  : }
  :
The throttler is only used in CLI tool? How about create a throttler internal 
of the cluster as well, use for replicas migration/auto rebalancing?



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 17
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Fri, 13 Oct 2023 04:25:51 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-12 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#17).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 147 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/17
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 17
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-11 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#16).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 146 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/16
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 16
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-11 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#15).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 151 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/15
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 15
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-10 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#14).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 150 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/14
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 14
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-10 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#13).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 146 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/13
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 13
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-09 Thread Wang Xixu (Code Review)

Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#12).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 146 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/12
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 12
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-09 Thread Alexey Serbin (Code Review)

Alexey Serbin has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 11:

(9 comments)

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tools/tool_action_local_replica.cc@336
PS11, Line 336: throttler.reset(new Throttler(MonoTime::Now(),
Please use std::make_shared() here instead.


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@259
PS11, Line 259:
nit: remove the extra spaces


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@270
PS11, Line 270:   if (use_throttler_) {
  : client_.reset(new RemoteTabletCopyClient(GetTabletId(),
  : fs_manager_.get(),
  : cmeta_manager,
  : messenger_,
  : 
tablet_copy_client_metrics_.get(),
  : throttler_));
  :   } else {
  : client_.reset(new RemoteTabletCopyClient(GetTabletId(),
  : fs_manager_.get(),
  : cmeta_manager,
  : messenger_,
  : nullptr /* no 
metrics */));
  :   }
Is it possible to pass tablet_copy_client_metrics_.get() and  throttler_ as the 
last two arguments instead of introducing this if/else clause?


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@337
PS11, Line 337:mode_ = TabletCopyMode::REMOTE;
  : use_throttler_ = true;
  : throttler_.reset(
  : new Throttler(MonoTime::Now(),
  :   0,
  :   
FLAGS_tablet_copy_transfer_chunk_size_bytes,
  :   2 * 
FLAGS_tablet_copy_transfer_chunk_size_bytes));
Is it possible to move this into the initializers' list?

Also, please use std::make_shared() to create a new instance of std::shared_ptr.


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@347
PS11, Line 347: NO_FATALS(
I'm not sure this is needed -- isn't gtest checking for any errors from SetUp() 
on itself?


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@368
PS11, Line 368: Tablet copy speed must less than throttler defined speed
nit: this seems to be an incomplete sentence


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client-test.cc@369
PS11, Line 369: ASSERT_GE(FLAGS_tablet_copy_transfer_chunk_size_bytes, 
current_speed);
With integer rounding, is this stable enough to not introduce any flakiness?  
Have you tried to run this test scenario with, say --stress_cpu_threads=16 
under RELEASE, DEBUG, ASAN, and TSAN builds?


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.h@321
PS11, Line 321: std::shared_ptr throttler_;
Why to use std::shared_ptr here?  Please describe what components share the 
ownership.

If it's not shared ownership, then consider using std::unique_ptr instead.


http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/11/src/kudu/tserver/tablet_copy_client.cc@84
PS11, Line 84: Limit tablet copy speed.
It would be great to include more details in this description, in particular:
  * Is this per-session limit (i.e. it limits the bandwidth of just a single 
table copying) or that's server-wide limit (i.e. this limits the total 
bandwidth of all tablet copying sessions at this tablet server)?
  * What's the semantics of value 0?



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 11
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin 
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-10-09 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#11).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.
Two paramters are added:
--tablet_copy_throttler_bytes_per_sec limits the copying speed,
and --tablet_copy_throttler_burst_factor limits the maximum
copying speed at a single time.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
5 files changed, 163 insertions(+), 16 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/11
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 11
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-27 Thread Yifan Zhang (Code Review)

Yifan Zhang has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 10:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/kudu-tool-test.cc@9384
PS10, Line 9384:   ASSERT_STR_CONTAINS(stderr, "Time spent Tablet copy 
throttler");
I don't think we should add a new test just to verify the output message or 
log. Maybe we can collect the number of bytes downloaded and the time used, and 
to verify the rate is within the limit?


http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tools/tool_action_local_replica.cc@1386
PS10, Line 1386: .AddOptionalParameter("tablet_copy_throttler_bytes_per_sec")
   :   
.AddOptionalParameter("tablet_copy_throttler_burst_factor")
nit: It's necessary to mention these two parameters in the commit message.


http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tserver/tablet_copy_client.h@320
PS10, Line 320:   std::shared_ptr throttler_;
Can we make it a 'std::unique_ptr'?

Maybe use it like this:

std::unique_ptr throttler_ = nullptr;
if (FLAGS_tablet_copy_throttler_bytes_per_sec > 0)
  throttler.reset(new Throttler()).


http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/10/src/kudu/tserver/tablet_copy_client.cc@83
PS10, Line 83: DEFINE_int64(tablet_copy_throttler_bytes_per_sec, 0,
 :  "Limit tablet copy speed.");
 : DEFINE_int64(tablet_copy_throttler_burst_factor, 0,
 :  "Burst factor for tablet copy throttling. The 
maximum rate the throttler "
 :  "allows within a token refill period (100ms) equals 
burst factor multiply "
 :  "base rate.");
Are these definitions necessary here? They are already defined in 
tool_action_local_replica.cc



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 10
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang 
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Wed, 27 Sep 2023 12:33:07 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-20 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 10:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc@9359
PS9, Line 9359: NO_FATALS(CreateTableWithFlushedData("table1", 
mini_cluster_.get(), 3, 1));
  :   int source_tserver_tablet_count = 
mini_cluster_->mini_tablet_server(0)->Lis
> Why create 2 tables? I guess you aim to make both of the 2 tservers have so
Done


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc@9361
PS9, Line 9361: RT_TRUE(source_tserver_tabl
> Then check source_tserver_tablet_count doesn't equal to 0.
Done


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc@9375
PS9, Line 9375: "-fs_data_dirs=$2 -fs_wal_dir=$3 -num_threads=3 "
  :  "-tablet_copy_throttler_bytes_per_sec=$4
> The test is just to verify the 2 flags are valid, but dosen't verify the ra
Done


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.h@355
PS9, Line 355:   // in a temporary local file 'superblock_path' and then loaded 
into the memory.
> I think the LocalTabletCopyClient has the same requirement as well, how abo
Done


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.cc@1008
PS9, Line 1008:   
dst_tablet_copy_metrics_->bytes_fetched->IncrementBy(chunk_size);
> How about adding LOG_TIMING when throttle happened?
Done



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 10
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Wed, 20 Sep 2023 10:00:00 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-20 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#10).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
4 files changed, 91 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/10
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 10
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-19 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 9:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc@9359
PS9, Line 9359: NO_FATALS(CreateTableWithFlushedData("table1", 
mini_cluster_.get(), 3, 1));
  :   NO_FATALS(CreateTableWithFlushedData("table2", 
mini_cluster_.get(), 3, 1));
Why create 2 tables? I guess you aim to make both of the 2 tservers have some 
tablets distributed on them, add some comments to explain this.


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc@9361
PS9, Line 9361: source_tserver_tablet_count
Then check source_tserver_tablet_count doesn't equal to 0.


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tools/kudu-tool-test.cc@9375
PS9, Line 9375: "-tablet_copy_throttler_bytes_per_sec=$4 "
  :  "-tablet_copy_throttler_burst_factor=1 ",
The test is just to verify the 2 flags are valid, but dosen't verify the rate 
limiter works, right?


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.h@355
PS9, Line 355:   std::shared_ptr throttler_;
I think the LocalTabletCopyClient has the same requirement as well, how about 
move the throttler_ to the base class TabletCopyClient?


http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/9/src/kudu/tserver/tablet_copy_client.cc@1008
PS9, Line 1008:   while (!throttler_->Take(MonoTime::Now(), 0, chunk_size)) 
{
How about adding LOG_TIMING when throttle happened?



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 9
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Wed, 20 Sep 2023 03:47:09 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-18 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#9).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
4 files changed, 79 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/9
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 9
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-18 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#8).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
4 files changed, 79 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/8
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 8
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-17 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 7:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG@9
PS5, Line 9: one cluster to another is a high reso
> nit: one cluster to another
Done


http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG@13
PS5, Line 13: get impacted and become unavailable
> It is necessary to mention the reason, I guess it's because of the tablets
Yes, you are right.


http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG@15
PS5, Line 15:  limit the
> How about: "a tradeoff of"
Done


http://gerrit.cloudera.org:8080/#/c/19479/5/thirdparty/build-definitions.sh
File thirdparty/build-definitions.sh:

http://gerrit.cloudera.org:8080/#/c/19479/5/thirdparty/build-definitions.sh@1248
PS5, Line 1248:
> Have you ever try class Throttler in src/kudu/util/throttler.h, does it mee
Throttler is a simple rate limiter. To avoid dependence on the third party 
library, here use throttler as the rate limiter.



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 7
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Mon, 18 Sep 2023 04:36:30 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-17 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#7).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
4 files changed, 79 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/7
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 7
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-17 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#6).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from one cluster to another is a high resource
consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable because of the tablets
copying process cost too much disk and/or network bandwith.
Therefore it is better to limit the tablets copying speed and make
the system more stable. The goal is a trade-off the tablets copying
speed and the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
This patch use a throttler to limit tablet copying speed.

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
M thirdparty/build-definitions.sh
5 files changed, 82 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/6
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 6
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-09-15 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 5:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG@9
PS5, Line 9: an old cluster to another new cluster
nit: one cluster to another


http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG@13
PS5, Line 13: get impacted and become unavailable
It is necessary to mention the reason, I guess it's because of the tablets 
copying process cost too much disk and/or network bandwith.


http://gerrit.cloudera.org:8080/#/c/19479/5//COMMIT_MSG@15
PS5, Line 15: to balance
How about: "a tradeoff of"


http://gerrit.cloudera.org:8080/#/c/19479/5/thirdparty/build-definitions.sh
File thirdparty/build-definitions.sh:

http://gerrit.cloudera.org:8080/#/c/19479/5/thirdparty/build-definitions.sh@1248
PS5, Line 1248: build_folly() {
Have you ever try class Throttler in src/kudu/util/throttler.h, does it meet 
the usage?



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 5
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Fri, 15 Sep 2023 11:24:04 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-07-13 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#5).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from an old cluster to another new cluster is a
high resource consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable. Therefore it is
better to limit the tablets copying speed and make the system
more stable. The goal is to balance the tablets copying speed and
the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
There are some algorithms to implement a rate limiter. This patch
will use the token bucket algorithm implemented by Facebook Folly
library: https://github.com/facebook/folly/blob/main/folly/TokenBucket.h

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M CMakeLists.txt
A cmake_modules/FindDoubleConversion.cmake
A cmake_modules/FindFmt.cmake
A cmake_modules/FindFolly.cmake
A cmake_modules/FindLibEvent.cmake
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
M thirdparty/build-definitions.sh
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
M thirdparty/vars.sh
13 files changed, 376 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/5
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 5
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-07-12 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Yingchun Lai, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#4).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from an old cluster to another new cluster is a
high resource consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable. Therefore it is
better to limit the tablets copying speed and make the system
more stable. The goal is to balance the tablets copying speed and
the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writing the data to local file system, it is better to
control the downloading speed to control the resource consumption.
There are some algorithms to implement a rate limiter. This patch
will use the token bucket algorithm implemented by Facebook Folly
library: https://github.com/facebook/folly/blob/main/folly/TokenBucket.h

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M CMakeLists.txt
A cmake_modules/FindDoubleConversion.cmake
A cmake_modules/FindFmt.cmake
A cmake_modules/FindFolly.cmake
A cmake_modules/FindLibEvent.cmake
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
M thirdparty/build-definitions.sh
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
M thirdparty/vars.sh
13 files changed, 378 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/4
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 4
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-06-12 Thread Yingchun Lai (Code Review)

Yingchun Lai has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19479/3/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/3/src/kudu/tserver/tablet_copy_client.cc@960
PS3, Line 960:   int64 burst_size = std::max(2*limit_speed, 
static_cast(chunk_size));
How about parameterize the burst_size as well?



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 3
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai 
Gerrit-Comment-Date: Mon, 12 Jun 2023 16:45:52 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-02-16 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 3:

> Patch Set 2:
>
> (8 comments)
>
> > Patch Set 1:
> >
> > (8 comments)
> >
> > Thank you for adding this. It would really help if you could add a little 
> > more details about the following:
> > 1. I understand that you have mentioned about other services getting 
> > impacted due to network jam caused by tablet copy operation, but that 
> > doesn't explain why tablet copy operation would become a preferred 
> > bandwidth user over other services. Is there an existing case where this 
> > kind of behaviour was seen and what was making linux kernel to do that?
> > 2. Adding detailed information (in commit message) about the new library 
> > APIs would be quite helpful for posterity.
> > 3. Limiting the network speed might cause frequent context switching. Do we 
> > have any data on that from the performance point of view?

Performance test was done, please see: 
https://issues.apache.org/jira/browse/KUDU-3447


--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 3
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Comment-Date: Thu, 16 Feb 2023 09:36:49 +
Gerrit-HasComments: No

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-02-09 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#3).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from an old cluster to another new cluster is a
high resource consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable. Therefore it is
better to limit the tablets copying speed and make the system
more stable. The goal is to balance the tablets copying speed and
the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writes the data to local file system, it is better to
control the downloading speed to control the resource consumption.
There are some algorithms to implement a rate limiter. This patch
will use the token bucket algorithm implemented by Facebook Folly
library: https://github.com/facebook/folly/blob/main/folly/TokenBucket.h

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M CMakeLists.txt
A cmake_modules/FindDoubleConversion.cmake
A cmake_modules/FindFmt.cmake
A cmake_modules/FindFolly.cmake
A cmake_modules/FindLibEvent.cmake
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
M thirdparty/build-definitions.sh
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
M thirdparty/vars.sh
13 files changed, 400 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/3
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 3
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-02-09 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 2:

> Patch Set 1:
>
> (8 comments)
>
> Thank you for adding this. It would really help if you could add a little 
> more details about the following:
> 1. I understand that you have mentioned about other services getting impacted 
> due to network jam caused by tablet copy operation, but that doesn't explain 
> why tablet copy operation would become a preferred bandwidth user over other 
> services. Is there an existing case where this kind of behaviour was seen and 
> what was making linux kernel to do that?
> 2. Adding detailed information (in commit message) about the new library APIs 
> would be quite helpful for posterity.
> 3. Limiting the network speed might cause frequent context switching. Do we 
> have any data on that from the performance point of view?

About 1 and 2, please view: https://issues.apache.org/jira/browse/KUDU-3447
about 3, I will supplement some performance tests.


--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 2
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Comment-Date: Thu, 09 Feb 2023 09:31:23 +
Gerrit-HasComments: No

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-02-09 Thread Wang Xixu (Code Review)

Wang Xixu has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/19479 )

Change subject: [KUDU-3447] Limit tablets copying speed
..


Patch Set 2:

(8 comments)

> Patch Set 1:
>
> (8 comments)
>
> Thank you for adding this. It would really help if you could add a little 
> more details about the following:
> 1. I understand that you have mentioned about other services getting impacted 
> due to network jam caused by tablet copy operation, but that doesn't explain 
> why tablet copy operation would become a preferred bandwidth user over other 
> services. Is there an existing case where this kind of behaviour was seen and 
> what was making linux kernel to do that?
> 2. Adding detailed information (in commit message) about the new library APIs 
> would be quite helpful for posterity.
> 3. Limiting the network speed might cause frequent context switching. Do we 
> have any data on that from the performance point of view?

http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG@9
PS1, Line 9: ter
> nit: does
Done


http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG@11
PS1, Line 11: y_fro
> nit: very
Done


http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG@15
PS1, Line 15: le. The goal is to balance the
> Do you want to explain it a little further. In the sense, what would owning
Done


http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:

http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/kudu-tool-test.cc@8871
PS1, Line 8871: tablet_ids_str,
> Instead of modifying existing test case, could you please add a new one and
Done


http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:

http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/tool_action_local_replica.cc@686
PS1, Line 686:   if (FLAGS_enable_network_speed_limit && 
FLAGS_limit_network_speed <= 0) {
> A similar check can be added to ensure either both flags are set or neither
OK. A GROUP_FLAG_VALIDATOR will be added.


http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/tool_action_local_replica.cc@686
PS1, Line 686: FLAGS_limit_network_speed <= 0)
> What if this value to too high?
Just allow it as big a value as int64 can hold


http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:

http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tserver/tablet_copy_client.cc@83
PS1, Line 83: If true, copying tablet
> Either set this to some legit value (e.g. 100 MB/s, 1000 MB/s, etc) or make
OK. Set the default value of limit_network_speed 1MB/s.


http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tserver/tablet_copy_client.cc@933
PS1, Line 933: lta::FromMilliseconds(FL
> I have no idea what this method does.
Done



--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 2
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Comment-Date: Thu, 09 Feb 2023 09:29:00 +
Gerrit-HasComments: Yes

[kudu-CR] [KUDU-3447] Limit tablets copying speed

2023-02-09 Thread Wang Xixu (Code Review)

Hello Ashwani Raina, Kudu Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/19479

to look at the new patch set (#2).

Change subject: [KUDU-3447] Limit tablets copying speed
..

[KUDU-3447] Limit tablets copying speed

Copying tablets from an old cluster to another new cluster is a
high resource consumed operation using the command : kudu
local_replica copy_from_remote. If the data size is very large,
the copying process will last for a long time. Other service
maybe get impacted and become unavailable. Therefore it is
better to limit the tablets copying speed and make the system
more stable. The goal is to balance the tablets copying speed and
the resource consumption.

As copy_from_remote is mainly downloading data from the remote
cluster and writes the data to local file system, it is better to
control the downloading speed to control the resource consumption.
There are some algorithms to implement a rate limiter. This patch
will use the token bucket algorithm implemented by Facebook Folly
library: https://github.com/facebook/folly/blob/main/folly/TokenBucket.h

Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
---
M CMakeLists.txt
A cmake_modules/FindDoubleConversion.cmake
A cmake_modules/FindFmt.cmake
A cmake_modules/FindFolly.cmake
A cmake_modules/FindLibEvent.cmake
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_local_replica.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
M thirdparty/build-definitions.sh
M thirdparty/build-thirdparty.sh
M thirdparty/download-thirdparty.sh
M thirdparty/vars.sh
13 files changed, 398 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/2
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 2
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina 
Gerrit-Reviewer: Kudu Jenkins (120)

48 matches

Mail list logo