[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. This is effective both for the source tablet and destination tablet during the copy operation. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Reviewed-on: http://gerrit.cloudera.org:8080/21356 Reviewed-by: Alexey Serbin Tested-by: Alexey Serbin --- M src/kudu/integration-tests/tablet_copy-itest.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/tablet_copy_service.cc M src/kudu/tserver/tablet_copy_source_session.cc M src/kudu/tserver/tablet_copy_source_session.h 6 files changed, 85 insertions(+), 2 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 15 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has removed a vote on this change. Change subject: [metrics] Add metrics for tablet copy op time .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 14 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 14: Verified+1 unrelated test failures -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 14 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 06 Jun 2024 16:42:47 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 14: Verified-1 Build Failed http://jenkins.kudu.apache.org/job/pre_commit/207/ : FAILURE -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 14 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 06 Jun 2024 03:10:16 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 14: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 14 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 06 Jun 2024 02:21:30 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 14: Build Started http://jenkins.kudu.apache.org/job/pre_commit/207/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 14 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 06 Jun 2024 02:16:08 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Alexey Serbin, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#14). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. This is effective both for the source tablet and destination tablet during the copy operation. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/integration-tests/tablet_copy-itest.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/tablet_copy_service.cc M src/kudu/tserver/tablet_copy_source_session.cc M src/kudu/tserver/tablet_copy_source_session.h 6 files changed, 85 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/14 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 14 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
KeDeng has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 14: (1 comment) Thanks for your reviews. http://gerrit.cloudera.org:8080/#/c/21356/13/src/kudu/integration-tests/tablet_copy-itest.cc File src/kudu/integration-tests/tablet_copy-itest.cc: http://gerrit.cloudera.org:8080/#/c/21356/13/src/kudu/integration-tests/tablet_copy-itest.cc@152 PS13, Line 152: METRIC_DECLARE_histogram(tablet_copy_duration); : METRIC_DECLARE_histogram(tablet_copy_source_dur > nit: swap these two lines to make them alphabetically ordered, as the rest Done -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 14 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 06 Jun 2024 02:16:50 + Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 13: Code-Review+1 (2 comments) Just one tiny need and the patch should be good to go! http://gerrit.cloudera.org:8080/#/c/21356/13/src/kudu/integration-tests/tablet_copy-itest.cc File src/kudu/integration-tests/tablet_copy-itest.cc: http://gerrit.cloudera.org:8080/#/c/21356/13/src/kudu/integration-tests/tablet_copy-itest.cc@152 PS13, Line 152: METRIC_DECLARE_histogram(tablet_copy_source_duration); : METRIC_DECLARE_histogram(tablet_copy_duration); nit: swap these two lines to make them alphabetically ordered, as the rest of this metrics list http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.cc File src/kudu/tablet/tablet_metrics.cc: http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.cc@258 PS11, Line 258: o CLIENT_PROPAGATED.", > Hi Alexey, Hi KeDeng, Thanks a lot for addressing the feedback and revving the patch! To me, the new version looks much better. -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 13 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 05 Jun 2024 21:33:15 + Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 13: Verified-1 Build Failed http://jenkins.kudu.apache.org/job/pre_commit/199/ : FAILURE -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 13 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 05 Jun 2024 09:12:47 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Alexey Serbin, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#13). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. This is effective both for the source tablet and destination tablet during the copy operation. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/integration-tests/tablet_copy-itest.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/tablet_copy_service.cc M src/kudu/tserver/tablet_copy_source_session.cc M src/kudu/tserver/tablet_copy_source_session.h 6 files changed, 85 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/13 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 13 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 13: Build Started http://jenkins.kudu.apache.org/job/pre_commit/199/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 13 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 05 Jun 2024 07:41:34 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 12: Verified-1 Build Failed http://jenkins.kudu.apache.org/job/pre_commit/198/ : FAILURE -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 12 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 05 Jun 2024 07:24:51 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
KeDeng has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 12: (1 comment) http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.cc File src/kudu/tablet/tablet_metrics.cc: http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.cc@258 PS11, Line 258: o CLIENT_PROPAGATED.", > Well, after looking a bit more into the code it seems this is vice-versa: t Hi Alexey, As we discussed on Slack, I have re-implemented the server-level histogram monitoring item. To ensure there are no misunderstandings like before, I have opted to validate the new monitoring item through unit tests to confirm its effectiveness. Since the current version differs significantly from the one you reviewed earlier, I apologize for not addressing each comment from the previous review. I would greatly appreciate it if you could review my new implementation again. Thank you very much. -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 12 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 05 Jun 2024 06:39:10 + Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 12: Build Started http://jenkins.kudu.apache.org/job/pre_commit/198/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 12 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 05 Jun 2024 06:32:54 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Alexey Serbin, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#12). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. This is effective both for the source tablet and destination tablet during the copy operation. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/integration-tests/tablet_copy-itest.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/tablet_copy_service.cc M src/kudu/tserver/tablet_copy_source_session.cc M src/kudu/tserver/tablet_copy_source_session.h 6 files changed, 83 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/12 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 12 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/21356 )
Change subject: [metrics] Add metrics for tablet copy op time
..
Patch Set 11:
(7 comments)
It seems the measured tablet copy duration is actually measured for the
destination tablet, not for the source one. I was misguided by the fact the
the tablet-level metric was a histogram -- I apologize for realizing that only
at PS11 after many iterations.
Please check out corresponding comments, and let me know if you think it's not
correct.
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.h
File src/kudu/tablet/tablet_metrics.h:
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.h@114
PS11, Line 114: scoped_refptr tablet_copy_duration;
I guess this histogram will contain just a single number, no? If so, then
histogram isn't a proper way to maintain such a metric. Probably, AtomicGauge
of integer type should be used instead.
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.cc
File src/kudu/tablet/tablet_metrics.cc:
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tablet/tablet_metrics.cc@258
PS11, Line 258: tablet replica as a source
Well, after looking a bit more into the code it seems this is vice-versa: this
is from the measurement done at the target/destination side.
Also, since it's going to be just a single number, it doesn't make sense to use
a histogram for that. From the other side, that's a bit strange to have a
'constant' per-table metric that's set only once upon copying the tablet from
the source, and never set to anything but 0 after a tablet server is restarted.
Maybe, it makes more sense to keep that sort of statistics not at the tablet
level, but rather at the service level, along with other metrics like
tablet_copy_bytes_fetched and tablet_copy_open_client_sessions?
Let me know if I'm wrong, but otherwise this needs to be updated accordingly.
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc@237
PS8, Line 237: // Test the metric that tracks the duration of a tablet copy
session at the source side.
: TEST_F(TabletCopyMetricTest, TestRunTimeMetricFinishState) {
: const auto before_cnt = CopyRunTime()->TotalCount();
: string session_id;
: ASSERT_OK(DoBeginValidTabletCopySession(_id));
:
: EndTabletCopySessionResponsePB resp;
: RpcController controller;
: ASSERT_OK(DoEndTabletCopySession(session_id, true, nullptr,
, ));
:
> Do you mean verifying in the unit test that the count of the tablet_copy_du
Ah, please ignore that.
As I already mentioned in the comment for PS11, I was mistakenly thinking this
is all about the metric at the source tablet replica. However, I realized
that's not so.
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tserver/tablet_copy_service-test.cc@221
PS11, Line 221: copy_run_time_ =
:
tablet_replica_->tablet()->metrics()->tablet_copy_duration;
nit: join these two lines or fix the indent for the second line?
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tserver/tablet_copy_service-test.cc@225
PS11, Line 225: void TearDown() override {
: TabletCopyServiceTest::TearDown();
: }
Why to override this if there is nothing specific but just a call to the
TearDown() of the parent class? Wouldn't it be the same effect if not
overriding TearDown() in this class at all?
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tserver/tablet_copy_service-test.cc@229
PS11, Line 229: scoped_refptr
nit: could this be 'const scoped_refptr&' ?
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tserver/tablet_copy_source_session.cc
File src/kudu/tserver/tablet_copy_source_session.cc:
http://gerrit.cloudera.org:8080/#/c/21356/11/src/kudu/tserver/tablet_copy_source_session.cc@512
PS11, Line 512: Prevent core dumps during deletion in the copy process.
nit: maybe, change this to be
If tablet hasn't been started yet, the TabletReplica::tablet_ field is null.
This code might encounter such a condition when the table is dropped while a
tablet copy session for any of its tablets is in progress.
--
To view, visit http://gerrit.cloudera.org:8080/21356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c
Gerrit-Change-Number: 21356
Gerrit-PatchSet: 11
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 11: Verified-1 Build Failed http://jenkins.kudu.apache.org/job/pre_commit/189/ : FAILURE -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 11 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 30 May 2024 09:23:44 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
KeDeng has posted comments on this change. (
http://gerrit.cloudera.org:8080/21356 )
Change subject: [metrics] Add metrics for tablet copy op time
..
Patch Set 11:
(7 comments)
Thanks for your reviews.
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.h
File src/kudu/tablet/tablet_metrics.h:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.h@114
PS10, Line 114: tablet_copy_duration;
> nit: the details are available in the description of the metric (and it's p
Done
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc
File src/kudu/tablet/tablet_metrics.cc:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc@255
PS10, Line 255: tablet_copy_duration,
> nit: the details are available in the description, so this could be just
Done
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc@256
PS10, Line 256: Tablet Copy Duration",
> nit: the details are available in the description, so this could be just
Done
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc@258
PS10, Line 258: Duration of tablet copying when using this tablet replica
> Duration of tablet copying when using this tablet replica as a source.
Done
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc@237
PS8, Line 237: // Test the metric that tracks the duration of a tablet copy
session at the source side.
: TEST_F(TabletCopyMetricTest, TestRunTimeMetricFinishState) {
: const auto before_cnt = CopyRunTime()->TotalCount();
: string session_id;
: ASSERT_OK(DoBeginValidTabletCopySession(_id));
:
: EndTabletCopySessionResponsePB resp;
: RpcController controller;
: ASSERT_OK(DoEndTabletCopySession(session_id, true, nullptr,
, ));
:
> Yep, this makes sense.
Do you mean verifying in the unit test that the count of the
tablet_copy_duration metric corresponding to the shard generated by the copy is
zero?
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_service-test.cc@237
PS10, Line 237: at the source
> nit: at the source side
Done
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_source_session.cc
File src/kudu/tserver/tablet_copy_source_session.cc:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_source_session.cc@512
PS10, Line 512: // Prevent core dumps during deletion in the
> Is there any situaion when RemoteTabletCopySourceSession::UpdateTabletMetri
It leads core down in TsTabletManagerITest.TestDeleteTableDuringTabletCopy, so
it is necessary to avoid the impact of deletion operations during the copying
process.
--
To view, visit http://gerrit.cloudera.org:8080/21356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c
Gerrit-Change-Number: 21356
Gerrit-PatchSet: 11
Gerrit-Owner: KeDeng
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Thu, 30 May 2024 09:03:14 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 11: Build Started http://jenkins.kudu.apache.org/job/pre_commit/189/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 11 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 30 May 2024 08:17:58 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Alexey Serbin, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#11). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. This is only effective for the source tablet during the copy operation. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_service-test.cc M src/kudu/tserver/tablet_copy_service.cc M src/kudu/tserver/tablet_copy_source_session.cc M src/kudu/tserver/tablet_copy_source_session.h 6 files changed, 67 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/11 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 11 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/21356 )
Change subject: [metrics] Add metrics for tablet copy op time
..
Patch Set 10:
(7 comments)
Almost there! Just a few nits.
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.h
File src/kudu/tablet/tablet_metrics.h:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.h@114
PS10, Line 114: tablet_copy_duration_as_source
nit: the details are available in the description of the metric (and it's
propagated into the auto-generated documentation), so this could be just
tablet_copy_duration
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc
File src/kudu/tablet/tablet_metrics.cc:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc@255
PS10, Line 255: tablet_copy_duration_as_source
nit: the details are available in the description, so this could be just
tablet_copy_duration
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc@256
PS10, Line 256: Tablet Copy Operation Duration As Source Tablet
nit: the details are available in the description, so this could be just
Tablet Copy Duration
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tablet/tablet_metrics.cc@258
PS10, Line 258: Duration of the tablet copying operation as source tablet.
Duration of tablet copying when using this tablet replica as a source.
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc@237
PS8, Line 237: // Test the metric that tracks the duration of a tablet copy
session on source side.
: TEST_F(TabletCopyMetricTest, TestRunTimeMetricFinishState) {
: const auto before_cnt = CopyRunTime()->TotalCount();
: string session_id;
: ASSERT_OK(DoBeginValidTabletCopySession(_id));
:
: EndTabletCopySessionResponsePB resp;
: RpcController controller;
: ASSERT_OK(DoEndTabletCopySession(session_id, true, nullptr,
, ));
:
> Yes, this metric is designed to be effective only on the source tablet duri
Yep, this makes sense.
My question was whether we want to track similar metric at the target replica,
making sure it stays zero after the copying completed and the target replica is
up and running.
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_service-test.cc@237
PS10, Line 237: on source side
nit: at the source side
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_source_session.cc
File src/kudu/tserver/tablet_copy_source_session.cc:
http://gerrit.cloudera.org:8080/#/c/21356/10/src/kudu/tserver/tablet_copy_source_session.cc@512
PS10, Line 512: if (PREDICT_TRUE(tablet_replica_->tablet()))
Is there any situaion when RemoteTabletCopySourceSession::UpdateTabletMetrics()
could be called when tablet_replica_->tablet() is nullptr? Do you mind adding
a comment to explain why this if() clause is needed?
Thank you!
--
To view, visit http://gerrit.cloudera.org:8080/21356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c
Gerrit-Change-Number: 21356
Gerrit-PatchSet: 10
Gerrit-Owner: KeDeng
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Wed, 29 May 2024 20:13:44 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 10: Verified+1 Build Successful http://jenkins.kudu.apache.org/job/pre_commit/176/ : SUCCESS -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 10 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Fri, 24 May 2024 07:07:17 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 10: Build Started http://jenkins.kudu.apache.org/job/pre_commit/176/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 10 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Fri, 24 May 2024 06:12:59 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 9: Verified+1 Build Successful http://jenkins.kudu.apache.org/job/pre_commit/155/ : SUCCESS -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 9 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 20 May 2024 04:36:16 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
KeDeng has posted comments on this change. (
http://gerrit.cloudera.org:8080/21356 )
Change subject: [metrics] Add metrics for tablet copy op time
..
Patch Set 9:
(6 comments)
Thanks for your reviews.
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc
File src/kudu/tablet/tablet_metrics.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@227
PS8, Line 227: _source,
> Would 'Tablet Copy Duration' be good enough?
Done
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@229
PS8, Line 229:
> Does it make sense to mention this is the duration as seen from the source
Done
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@229
PS8, Line 229: :kMilliseco
> tablet copying
Done
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@231
PS8, Line 231: kudu::Metr
> Yingchun already pointed at that in PS5, but it seems there is still room f
Done
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc@237
PS8, Line 237: // Test the metric that tracks the duration of a tablet copy
session on source side.
: TEST_F(TabletCopyMetricTest, TestRunTimeMetricFinishState) {
: const auto before_cnt = CopyRunTime()->TotalCount();
: string session_id;
: ASSERT_OK(DoBeginValidTabletCopySession(_id));
:
: EndTabletCopySessionResponsePB resp;
: RpcController controller;
: ASSERT_OK(DoEndTabletCopySession(session_id, true, nullptr,
, ));
:
> Does it make sense to verify the corresponding metric both at the destinati
Yes, this metric is designed to be effective only on the source tablet during
the copy execution process and will not be displayed on the target shard. This
is primarily determined by the copy process, as there are additional operations
after the copy execution is completed to ensure that the target tablet
transitions to a normal working state. Updating the metric before this
transition seems meaningless.
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_source_session.cc
File src/kudu/tserver/tablet_copy_source_session.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_source_session.cc@514
PS8, Line 514: PREDICT_TRUE(metrics != nullptr)) {
: int64_t elapsed_ms = (MonoTime::Now() - start_time_
> In tablet_metrics.cc, the tablet_copy_duration is defined in microseconds i
Done
--
To view, visit http://gerrit.cloudera.org:8080/21356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c
Gerrit-Change-Number: 21356
Gerrit-PatchSet: 9
Gerrit-Owner: KeDeng
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Mon, 20 May 2024 03:43:34 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Alexey Serbin, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#9). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. This is only effective for the source tablet during the copy operation. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_service-test.cc M src/kudu/tserver/tablet_copy_service.cc M src/kudu/tserver/tablet_copy_source_session.cc M src/kudu/tserver/tablet_copy_source_session.h 6 files changed, 66 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/9 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 9 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 9: Build Started http://jenkins.kudu.apache.org/job/pre_commit/155/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 9 Gerrit-Owner: KeDeng Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 20 May 2024 03:42:57 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Alexey Serbin has posted comments on this change. (
http://gerrit.cloudera.org:8080/21356 )
Change subject: [metrics] Add metrics for tablet copy op time
..
Patch Set 8:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc
File src/kudu/tablet/tablet_metrics.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@227
PS8, Line 227: Tablet Copy Operation Duration
Would 'Tablet Copy Duration' be good enough?
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@229
PS8, Line 229: on this tablet
Does it make sense to mention this is the duration as seen from the source
tablet replica?
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@229
PS8, Line 229: copy tablet
tablet copying
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tablet/tablet_metrics.cc@231
PS8, Line 231: 6000LU
Yingchun already pointed at that in PS5, but it seems there is still room for
improvement.
With current settings, 6000 stands for maximum value of 1 minute (60
seconds). Are you sure this makes sense? I suspect that for a large tablet
replica being copied over a slow network it might take tens of minutes to
complete the operation, so an hour for the maximum duration would be prudent.
Also, I don't think it makes a lot of sense to have microseconds for the unit
for the duration here. I'd think it to be milliseconds at most.
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc
File src/kudu/tserver/tablet_copy_service-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_service-test.cc@237
PS8, Line 237: TEST_F(TabletCopyMetricTest, TestRunTimeMetricFinishState) {
: const auto before_cnt = CopyRunTime()->TotalCount();
: string session_id;
: ASSERT_OK(DoBeginValidTabletCopySession(_id));
:
: EndTabletCopySessionResponsePB resp;
: RpcController controller;
: ASSERT_OK(DoEndTabletCopySession(session_id, true, nullptr,
, ));
: ASSERT_EQ(before_cnt + 1, CopyRunTime()->TotalCount());
: }
Does it make sense to verify the corresponding metric both at the destination
tablet replica as well? I guess it should show 0 for TotalCount(), right?
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_source_session.cc
File src/kudu/tserver/tablet_copy_source_session.cc:
http://gerrit.cloudera.org:8080/#/c/21356/8/src/kudu/tserver/tablet_copy_source_session.cc@514
PS8, Line 514: int64_t elapsed_ms = (MonoTime::Now() -
start_time_).ToMilliseconds();
: metrics->tablet_copy_duration->Increment(elapsed_ms);
In tablet_metrics.cc, the tablet_copy_duration is defined in microseconds is
PS8, so this code is inconsistent.
--
To view, visit http://gerrit.cloudera.org:8080/21356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c
Gerrit-Change-Number: 21356
Gerrit-PatchSet: 8
Gerrit-Owner: KeDeng
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Fri, 17 May 2024 18:14:55 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 8: Verified-1 Build Failed http://jenkins.kudu.apache.org/job/pre_commit/153/ : FAILURE -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 8 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Fri, 17 May 2024 10:32:58 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#8). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_service-test.cc M src/kudu/tserver/tablet_copy_service.cc M src/kudu/tserver/tablet_copy_source_session.cc M src/kudu/tserver/tablet_copy_source_session.h 6 files changed, 62 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/8 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 8 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 8: Build Started http://jenkins.kudu.apache.org/job/pre_commit/153/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 8 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Fri, 17 May 2024 03:14:11 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 7: Verified-1 Build Failed http://jenkins.kudu.apache.org/job/pre_commit/135/ : FAILURE -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 7 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sat, 11 May 2024 10:46:41 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#7). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. The addition of monitoring items will aid in historical issue tracking and analysis, as well as facilitate the configuration of monitoring alarms. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 112 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/7 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 7 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 7: Build Started http://jenkins.kudu.apache.org/job/pre_commit/135/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 7 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sat, 11 May 2024 09:22:56 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Yingchun Lai has posted comments on this change. (
http://gerrit.cloudera.org:8080/21356 )
Change subject: [metrics] Add metrics for tablet copy op time
..
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/21356/6/src/kudu/tserver/tablet_copy_client.h
File src/kudu/tserver/tablet_copy_client.h:
http://gerrit.cloudera.org:8080/#/c/21356/6/src/kudu/tserver/tablet_copy_client.h@197
PS6, Line 197: const tablet::TabletMetrics* TabletMetrics() { return
tablet_metrics_; }
Is this necessary? It seems TabletMetrics() is only used in TabletCopyClient
class, we can access tablet_metrics_ directly.
--
To view, visit http://gerrit.cloudera.org:8080/21356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c
Gerrit-Change-Number: 21356
Gerrit-PatchSet: 6
Gerrit-Owner: KeDeng
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Sat, 11 May 2024 03:18:52 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 6: Verified-1 Build Failed http://jenkins.kudu.apache.org/job/pre_commit/67/ : FAILURE -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 6 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Tue, 30 Apr 2024 14:59:57 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 6: Build Started http://jenkins.kudu.apache.org/job/pre_commit/67/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 6 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Tue, 30 Apr 2024 12:35:12 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 6: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/57/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 6 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Tue, 30 Apr 2024 08:57:37 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
KeDeng has posted comments on this change. (
http://gerrit.cloudera.org:8080/21356 )
Change subject: [metrics] Add metrics for tablet copy op time
..
Patch Set 6:
(5 comments)
Thanks for your reviews.
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tablet/tablet_metrics.cc
File src/kudu/tablet/tablet_metrics.cc:
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tablet/tablet_metrics.cc@231
PS5, Line 231: 600
> While the unit is kMicroseconds, so the 'max_val' maybe 60,000,000, or even
Done
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc
File src/kudu/tserver/tablet_copy_client-test.cc:
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc@336
PS5, Line 336: void TearDown() override {
: client_.reset();
: TabletCopyClientTest::TearDown();
: }
> Can we define these in SetUp() ?
Done
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc@345
PS5, Line 345:
> Can we define it in TearDown() ?
Done
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc@561
PS5, Line 561: int64_t dur = GetMonoTimeMicros() - start_time_micros_;
> This metric is in microsecond unit [1], don't divide by 1000.
Done
http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc@605
PS5, Line 605: int64_t dur = GetMonoTimeMicros() - start_time_micros_;
> Ditto.
Done
--
To view, visit http://gerrit.cloudera.org:8080/21356
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c
Gerrit-Change-Number: 21356
Gerrit-PatchSet: 6
Gerrit-Owner: KeDeng
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Mon, 29 Apr 2024 07:09:18 +
Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#6). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 118 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/6 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 6 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 6: Build Started http://jenkins.kudu.apache.org/job/pre_commit/57/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 6 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 29 Apr 2024 07:09:01 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 5: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/56/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 5 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 29 Apr 2024 07:08:53 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tablet/tablet_metrics.cc File src/kudu/tablet/tablet_metrics.cc: http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tablet/tablet_metrics.cc@231 PS5, Line 231: 6LU While the unit is kMicroseconds, so the 'max_val' maybe 60,000,000, or even larger. A tablet disk space is possible to be dozens of GiB, copy a tablet may consume dozens of minitues, so we can define the 'max_val' to several hours. http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc File src/kudu/tserver/tablet_copy_client-test.cc: http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc@336 PS5, Line 336: ASSERT_OK(ResetTabletCopyClient(tablet_replica_->tablet()->metrics())); : auto copy_run_time = : tablet_replica_->tablet()->metrics()->tablet_copy_duration; : const auto before_cnt = copy_run_time->TotalCount(); Can we define these in SetUp() ? http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client-test.cc@345 PS5, Line 345: client_.reset(); Can we define it in TearDown() ? http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc File src/kudu/tserver/tablet_copy_client.cc: http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc@561 PS5, Line 561: int64_t dur = (GetMonoTimeMicros() - start_time_micros_) / 1000; This metric is in microsecond unit [1], don't divide by 1000. 1. https://gerrit.cloudera.org/c/21356/5/src/kudu/tablet/tablet_metrics.cc#228 http://gerrit.cloudera.org:8080/#/c/21356/5/src/kudu/tserver/tablet_copy_client.cc@605 PS5, Line 605: int64_t dur = (GetMonoTimeMicros() - start_time_micros_) / 1000; Ditto. -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 5 Gerrit-Owner: KeDeng Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 29 Apr 2024 03:55:44 + Gerrit-HasComments: Yes
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#5). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/5 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 5 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#4). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add tablet-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/4 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 4 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#3). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/3 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 3 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 5: Build Started http://jenkins.kudu.apache.org/job/pre_commit/56/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 5 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:54:17 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 4: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/55/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 4 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:54:12 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 4: Build Started http://jenkins.kudu.apache.org/job/pre_commit/55/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 4 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:51:52 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 3: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/54/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 3 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:51:43 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 3: Build Started http://jenkins.kudu.apache.org/job/pre_commit/54/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 3 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:50:57 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 2: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/53/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 2 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 10:50:52 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/21356 to look at the new patch set (#2). Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tablet/tablet_metrics.cc M src/kudu/tablet/tablet_metrics.h M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M src/kudu/tserver/ts_tablet_manager.cc 6 files changed, 107 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/2 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 2 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 1: Build Aborted http://jenkins.kudu.apache.org/job/pre_commit/43/ : ABORTED -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 1 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 09:13:24 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 2: Build Started http://jenkins.kudu.apache.org/job/pre_commit/53/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 2 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sun, 28 Apr 2024 09:13:32 + Gerrit-HasComments: No
[kudu-CR] [metrics] Add metrics for tablet copy op time
KeDeng has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21356 Change subject: [metrics] Add metrics for tablet copy op time .. [metrics] Add metrics for tablet copy op time Add server-level statistics to track the time consumption of copy tablet operations. These monitoring metrics will be very useful for us to track CPU usage. Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c --- M src/kudu/tserver/tablet_copy_client-test.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h 3 files changed, 91 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/21356/1 -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 1 Gerrit-Owner: KeDeng
[kudu-CR] [metrics] Add metrics for tablet copy op time
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21356 ) Change subject: [metrics] Add metrics for tablet copy op time .. Patch Set 1: Build Started http://jenkins.kudu.apache.org/job/pre_commit/43/ -- To view, visit http://gerrit.cloudera.org:8080/21356 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I088f6a9a8a07ad39ca95ae8b4995ce00d1a0d00c Gerrit-Change-Number: 21356 Gerrit-PatchSet: 1 Gerrit-Owner: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 25 Apr 2024 03:44:29 + Gerrit-HasComments: No
