[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest maybe core
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19451 Change subject: [KUDU-3438] TabletCopyClientAbortTest maybe core .. [KUDU-3438] TabletCopyClientAbortTest maybe core TabletCopyClientTest::SetUp() of TabletCopyClientAbortTest may fail, for example, because of the full disk. TabletCopyClient will be not initialized. Therefore using TabletCopyClient in StartCopy() will cause coredump. It is better to Add NO_FATALS() for TabletCopyClientTest::SetUp() to avoid nullptr TabletCopyClient. Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c --- M src/kudu/tserver/tablet_copy_client-test.cc 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/51/19451/1 -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest maybe core
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19451 to look at the new patch set (#2). Change subject: [KUDU-3438] TabletCopyClientAbortTest maybe core .. [KUDU-3438] TabletCopyClientAbortTest maybe core TabletCopyClientTest::SetUp() of TabletCopyClientAbortTest may fail, for example, because of the full disk. TabletCopyClient will be not initialized. Therefore using TabletCopyClient in StartCopy() will cause coredump. It is better to Add NO_FATALS() for TabletCopyClientTest::SetUp() to avoid nullptr TabletCopyClient. Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c --- M src/kudu/tserver/tablet_copy_client-test.cc 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/51/19451/2 -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest maybe core
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19451 ) Change subject: [KUDU-3438] TabletCopyClientAbortTest maybe core .. Patch Set 2: > Patch Set 1: > > (1 comment) It needs to capture the error inside a function using NO_FATALS(), or it will execute the code continuely. -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 30 Jan 2023 03:54:08 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest maybe core
Hello Tidy Bot, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19451 to look at the new patch set (#3). Change subject: [KUDU-3438] TabletCopyClientAbortTest maybe core .. [KUDU-3438] TabletCopyClientAbortTest maybe core TabletCopyClientTest::SetUp() of TabletCopyClientAbortTest may fail, for example, because of the full disk. TabletCopyClient will be not initialized. Therefore using TabletCopyClient in StartCopy() will cause coredump. It is better to Add NO_FATALS() for TabletCopyClientTest::SetUp() to avoid nullptr TabletCopyClient. Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c --- M src/kudu/tserver/tablet_copy_client-test.cc 1 file changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/51/19451/3 -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest maybe core
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19451 ) Change subject: [KUDU-3438] TabletCopyClientAbortTest maybe core .. Patch Set 2: > Patch Set 2: > > (1 comment) done -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Mon, 30 Jan 2023 03:58:01 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest maybe core
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19451 ) Change subject: [KUDU-3438] TabletCopyClientAbortTest maybe core .. Patch Set 3: > Patch Set 3: > > (1 comment) Line 598 throws the error, not line 121. -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Wed, 01 Feb 2023 09:43:54 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest might crash
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19451 ) Change subject: [KUDU-3438] TabletCopyClientAbortTest might crash .. Patch Set 4: (3 comments) > Patch Set 3: Code-Review+2 > > (3 comments) > > Thank you for the fix! > > A few nits in the commit description, otherwise looks good. http://gerrit.cloudera.org:8080/#/c/19451/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19451/3//COMMIT_MSG@7 PS3, Line 7: might cras > nit: might crash Done http://gerrit.cloudera.org:8080/#/c/19451/3//COMMIT_MSG@12 PS3, Line 12: a crash. > nit: a crash Done http://gerrit.cloudera.org:8080/#/c/19451/3//COMMIT_MSG@14 PS3, Line 14: It is better to Add NO_FATALS() for TabletCopyClientTest::SetUp() : to avoid nullptr TabletCopyClient. > Right, but I guess that leads only to exiting earlier from TabletCopyClient Done -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Thu, 02 Feb 2023 01:54:18 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3438] TabletCopyClientAbortTest might crash
Hello Tidy Bot, Alexey Serbin, Yingchun Lai, Yifan Zhang, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19451 to look at the new patch set (#4). Change subject: [KUDU-3438] TabletCopyClientAbortTest might crash .. [KUDU-3438] TabletCopyClientAbortTest might crash TabletCopyClientTest::SetUp() of TabletCopyClientAbortTest may fail, for example, because of the full disk. TabletCopyClient will be not initialized. Therefore using TabletCopyClient in StartCopy() will cause a crash. It is better to Add NO_FATALS() for TabletCopyClientTest::SetUp() to avoid nullptr TabletCopyClient. Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c --- M src/kudu/tserver/tablet_copy_client-test.cc 1 file changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/51/19451/4 -- To view, visit http://gerrit.cloudera.org:8080/19451 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id1eeea183bf9fc0b566e6a70f2f7520f34b2ae3c Gerrit-Change-Number: 19451 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [Tool] Limit tablets copying speed
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19479 Change subject: [Tool] Limit tablets copying speed .. [Tool] Limit tablets copying speed Copying tablets from a remote cluster to local cluster will not limit the network speed. Sometimes, it will cause network jam when the data is every large. Other services will be disturbed. Therefore, it needs to limit the tables copying speed. This patch uses token bucket algorithm to implement a network limiter. One host owns one token bucket. Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 --- M CMakeLists.txt A cmake_modules/FindDoubleConversion.cmake A cmake_modules/FindFmt.cmake A cmake_modules/FindFolly.cmake A cmake_modules/FindLibEvent.cmake M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_local_replica.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 13 files changed, 337 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/1 -- To view, visit http://gerrit.cloudera.org:8080/19479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 Gerrit-Change-Number: 19479 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [KUDU-3447] Limit tablets copying speed
Hello Ashwani Raina, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19479 to look at the new patch set (#2). Change subject: [KUDU-3447] Limit tablets copying speed .. [KUDU-3447] Limit tablets copying speed Copying tablets from an old cluster to another new cluster is a high resource consumed operation using the command : kudu local_replica copy_from_remote. If the data size is very large, the copying process will last for a long time. Other service maybe get impacted and become unavailable. Therefore it is better to limit the tablets copying speed and make the system more stable. The goal is to balance the tablets copying speed and the resource consumption. As copy_from_remote is mainly downloading data from the remote cluster and writes the data to local file system, it is better to control the downloading speed to control the resource consumption. There are some algorithms to implement a rate limiter. This patch will use the token bucket algorithm implemented by Facebook Folly library: https://github.com/facebook/folly/blob/main/folly/TokenBucket.h Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 --- M CMakeLists.txt A cmake_modules/FindDoubleConversion.cmake A cmake_modules/FindFmt.cmake A cmake_modules/FindFolly.cmake A cmake_modules/FindLibEvent.cmake M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_local_replica.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 13 files changed, 398 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/2 -- To view, visit http://gerrit.cloudera.org:8080/19479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 Gerrit-Change-Number: 19479 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [KUDU-3447] Limit tablets copying speed
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19479 )
Change subject: [KUDU-3447] Limit tablets copying speed
..
Patch Set 2:
(8 comments)
> Patch Set 1:
>
> (8 comments)
>
> Thank you for adding this. It would really help if you could add a little
> more details about the following:
> 1. I understand that you have mentioned about other services getting impacted
> due to network jam caused by tablet copy operation, but that doesn't explain
> why tablet copy operation would become a preferred bandwidth user over other
> services. Is there an existing case where this kind of behaviour was seen and
> what was making linux kernel to do that?
> 2. Adding detailed information (in commit message) about the new library APIs
> would be quite helpful for posterity.
> 3. Limiting the network speed might cause frequent context switching. Do we
> have any data on that from the performance point of view?
http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG@9
PS1, Line 9: ter
> nit: does
Done
http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG@11
PS1, Line 11: y_fro
> nit: very
Done
http://gerrit.cloudera.org:8080/#/c/19479/1//COMMIT_MSG@15
PS1, Line 15: le. The goal is to balance the
> Do you want to explain it a little further. In the sense, what would owning
Done
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/kudu-tool-test.cc@8871
PS1, Line 8871: tablet_ids_str,
> Instead of modifying existing test case, could you please add a new one and
Done
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/tool_action_local_replica.cc
File src/kudu/tools/tool_action_local_replica.cc:
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/tool_action_local_replica.cc@686
PS1, Line 686: if (FLAGS_enable_network_speed_limit &&
FLAGS_limit_network_speed <= 0) {
> A similar check can be added to ensure either both flags are set or neither
OK. A GROUP_FLAG_VALIDATOR will be added.
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tools/tool_action_local_replica.cc@686
PS1, Line 686: FLAGS_limit_network_speed <= 0)
> What if this value to too high?
Just allow it as big a value as int64 can hold
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tserver/tablet_copy_client.cc
File src/kudu/tserver/tablet_copy_client.cc:
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tserver/tablet_copy_client.cc@83
PS1, Line 83: If true, copying tablet
> Either set this to some legit value (e.g. 100 MB/s, 1000 MB/s, etc) or make
OK. Set the default value of limit_network_speed 1MB/s.
http://gerrit.cloudera.org:8080/#/c/19479/1/src/kudu/tserver/tablet_copy_client.cc@933
PS1, Line 933: lta::FromMilliseconds(FL
> I have no idea what this method does.
Done
--
To view, visit http://gerrit.cloudera.org:8080/19479
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3
Gerrit-Change-Number: 19479
Gerrit-PatchSet: 2
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Comment-Date: Thu, 09 Feb 2023 09:29:00 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3447] Limit tablets copying speed
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19479 ) Change subject: [KUDU-3447] Limit tablets copying speed .. Patch Set 2: > Patch Set 1: > > (8 comments) > > Thank you for adding this. It would really help if you could add a little > more details about the following: > 1. I understand that you have mentioned about other services getting impacted > due to network jam caused by tablet copy operation, but that doesn't explain > why tablet copy operation would become a preferred bandwidth user over other > services. Is there an existing case where this kind of behaviour was seen and > what was making linux kernel to do that? > 2. Adding detailed information (in commit message) about the new library APIs > would be quite helpful for posterity. > 3. Limiting the network speed might cause frequent context switching. Do we > have any data on that from the performance point of view? About 1 and 2, please view: https://issues.apache.org/jira/browse/KUDU-3447 about 3, I will supplement some performance tests. -- To view, visit http://gerrit.cloudera.org:8080/19479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 Gerrit-Change-Number: 19479 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Comment-Date: Thu, 09 Feb 2023 09:31:23 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3447] Limit tablets copying speed
Hello Ashwani Raina, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19479 to look at the new patch set (#3). Change subject: [KUDU-3447] Limit tablets copying speed .. [KUDU-3447] Limit tablets copying speed Copying tablets from an old cluster to another new cluster is a high resource consumed operation using the command : kudu local_replica copy_from_remote. If the data size is very large, the copying process will last for a long time. Other service maybe get impacted and become unavailable. Therefore it is better to limit the tablets copying speed and make the system more stable. The goal is to balance the tablets copying speed and the resource consumption. As copy_from_remote is mainly downloading data from the remote cluster and writes the data to local file system, it is better to control the downloading speed to control the resource consumption. There are some algorithms to implement a rate limiter. This patch will use the token bucket algorithm implemented by Facebook Folly library: https://github.com/facebook/folly/blob/main/folly/TokenBucket.h Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 --- M CMakeLists.txt A cmake_modules/FindDoubleConversion.cmake A cmake_modules/FindFmt.cmake A cmake_modules/FindFolly.cmake A cmake_modules/FindLibEvent.cmake M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_local_replica.cc M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/tablet_copy_client.h M thirdparty/build-definitions.sh M thirdparty/build-thirdparty.sh M thirdparty/download-thirdparty.sh M thirdparty/vars.sh 13 files changed, 400 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/79/19479/3 -- To view, visit http://gerrit.cloudera.org:8080/19479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 Gerrit-Change-Number: 19479 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
[kudu-CR] [Client] Add query id to trace the whole query process
Hello Mahesh Reddy, Tidy Bot, Jian Zhang, Yuqi Du, Alexey Serbin, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18846 to look at the new patch set (#29). Change subject: [Client] Add query id to trace the whole query process .. [Client] Add query id to trace the whole query process In Impala, a sql will be translated into a query plan. The query plan contains multiple execution units and will be executed distributly in different hosts of a cluster. There exists an unique query id to trace the execution process of every execution unit. It is useful to analyze which step get stucked or failed, when a slow or failed query happened. Query id can associate all log records in different hosts, which belongs to one query. Kudu is the scan node of a query plan, when defines Kudu table as external table in Impala. And a scanning on a table will be splited into multiple scanning on different tablets. The scanner id is used to trace a scanning on a tablet. But there is a gap between Impala and Kudu. We can't trace the whole scanning process in Kudu of a query from Impala. There is not unique id to associate the execution of all scanners in Kudu. Therefore, this patch post query id from Impala to Kudu, and binding with scanner id. Then we can trace a query from Impala to Kudu for troubleshooting and debugging. Change-Id: I9dbae801596726fec1c85ee547128da3179345d9 --- M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/client.proto M src/kudu/client/scan_token-internal.cc M src/kudu/client/scan_token-internal.h M src/kudu/client/scan_token-test.cc M src/kudu/client/scanner-internal.cc M src/kudu/tserver/tablet_server-test.cc M src/kudu/tserver/tablet_service.cc M src/kudu/tserver/tserver.proto 11 files changed, 207 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/46/18846/29 -- To view, visit http://gerrit.cloudera.org:8080/18846 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9dbae801596726fec1c85ee547128da3179345d9 Gerrit-Change-Number: 18846 Gerrit-PatchSet: 29 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Client] Add query id to trace the whole query process
Hello Mahesh Reddy, Tidy Bot, Jian Zhang, Yuqi Du, Alexey Serbin, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18846 to look at the new patch set (#30). Change subject: [Client] Add query id to trace the whole query process .. [Client] Add query id to trace the whole query process In Impala, a sql will be translated into a query plan. The query plan contains multiple execution units and will be executed distributly in different hosts of a cluster. There exists an unique query id to trace the execution process of every execution unit. It is useful to analyze which step get stucked or failed, when a slow or failed query happened. Query id can associate all log records in different hosts, which belongs to one query. Kudu is the scan node of a query plan, when defines Kudu table as external table in Impala. And a scanning on a table will be splited into multiple scanning on different tablets. The scanner id is used to trace a scanning on a tablet. But there is a gap between Impala and Kudu. We can't trace the whole scanning process in Kudu of a query from Impala. There is not unique id to associate the execution of all scanners in Kudu. Therefore, this patch post query id from Impala to Kudu, and binding with scanner id. Then we can trace a query from Impala to Kudu for troubleshooting and debugging. Change-Id: I9dbae801596726fec1c85ee547128da3179345d9 --- M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/client.proto M src/kudu/client/scan_token-internal.cc M src/kudu/client/scan_token-internal.h M src/kudu/client/scan_token-test.cc M src/kudu/client/scanner-internal.cc M src/kudu/tserver/tablet_server-test.cc M src/kudu/tserver/tablet_service.cc M src/kudu/tserver/tserver.proto 11 files changed, 179 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/46/18846/30 -- To view, visit http://gerrit.cloudera.org:8080/18846 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9dbae801596726fec1c85ee547128da3179345d9 Gerrit-Change-Number: 18846 Gerrit-PatchSet: 30 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19493 Change subject: [Tools] Remove enter key when listing non-existed table .. [Tools] Remove enter key when listing non-existed table Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 --- M src/kudu/tools/tool_action_table.cc 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/19493/1 -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19493 to look at the new patch set (#2). Change subject: [Tools] Remove enter key when listing non-existed table .. [Tools] Remove enter key when listing non-existed table When Kudu listing a non-existed table using command line tool, it ouputs an enter key and a new line appears. But earlier version does not output an enter key. Therefore it is better to be compatible with old versions of Kudu. Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 --- M src/kudu/tools/tool_action_table.cc 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/19493/2 -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [Client] Add query id to trace the whole query process
Hello Mahesh Reddy, Tidy Bot, Jian Zhang, Yuqi Du, Alexey Serbin, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Abhishek Chennaka, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/18846 to look at the new patch set (#31). Change subject: [Client] Add query id to trace the whole query process .. [Client] Add query id to trace the whole query process In Impala, an SQL is translated into a query plan. The query plan contains multiple execution units and is executed at different hosts of a cluster in a distributed manner. There exists an unique query id to trace the execution process of every execution unit. It is useful in analysis of stuck, failed, and otherwise non-performant queries. Query id can associate all log records in different hosts, which belongs to one query. Scanning a Kudu table is split into multiple scan operations on appropriate tablets, one scanner per tablet. When scanning a Kudu table using Impala, corresponding scan nodes appear for every scanner in the Impala's query plan. Scanner id is used to trace the scan operation on a tablet. But there is a gap between Impala and Kudu. We can't trace the whole scanning process in Kudu of a query originated from Impala. There is not unique id to associate the execution of all scanners in Kudu. Therefore, with this patch it is now possible to post Impala's query identifier to Kudu, so it can be associated with Kudu's scanner id. Then we can trace a query from Impala to Kudu for troubleshooting and debugging. Change-Id: I9dbae801596726fec1c85ee547128da3179345d9 --- M src/kudu/client/client-test.cc M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/client.proto M src/kudu/client/scan_token-internal.cc M src/kudu/client/scan_token-internal.h M src/kudu/client/scan_token-test.cc M src/kudu/client/scanner-internal.cc M src/kudu/tserver/tablet_server-test.cc M src/kudu/tserver/tablet_service.cc M src/kudu/tserver/tserver.proto 11 files changed, 197 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/46/18846/31 -- To view, visit http://gerrit.cloudera.org:8080/18846 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9dbae801596726fec1c85ee547128da3179345d9 Gerrit-Change-Number: 18846 Gerrit-PatchSet: 31 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Jian Zhang Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Client] Add query id to trace the whole query process
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/18846 )
Change subject: [Client] Add query id to trace the whole query process
..
Patch Set 31:
(22 comments)
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@9
PS30, Line 9: is tra
> is
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@9
PS30, Line 9: an SQ
> an SQL query
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@10
PS30, Line 10: is
> is
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@11
PS30, Line 11: h
> at
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@11
PS30, Line 11: at differen
> in a distributed manner
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@13
PS30, Line 13: very execution unit. It is useful
: in analysis of stuck, failed, and otherwise non-performa
> It is useful in analysis of stuck, failed, and otherwise non-performant que
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@18
PS30, Line 18: Scanning a Kudu table is split into multiple scan operations
: on appropriate tablets, one scanner per tablet. When scanning
: a Kudu table using Impala, corresponding scan nodes appear
: for every scanner in the Impala's query pla
> Scanning a Kudu table is split into multiple scan operations on appropriate
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@24
PS30, Line 24: . We
> originated from
Done
http://gerrit.cloudera.org:8080/#/c/18846/30//COMMIT_MSG@27
PS30, Line 27:
:
> with this patch it is now possible to post Impala's query identifier to Kud
Done
http://gerrit.cloudera.org:8080/#/c/18846/27/src/kudu/client/client.h
File src/kudu/client/client.h:
http://gerrit.cloudera.org:8080/#/c/18846/27/src/kudu/client/client.h@3376
PS27, Line 3376:
> Missed this comment?
Done
http://gerrit.cloudera.org:8080/#/c/18846/27/src/kudu/client/client.h@3377
PS27, Line 3377: ole process.
> Missed this comment?
Done
http://gerrit.cloudera.org:8080/#/c/18846/27/src/kudu/client/client.h@3380
PS27, Line 3380:
> Missed this comment?
Done
http://gerrit.cloudera.org:8080/#/c/18846/27/src/kudu/client/client.cc
File src/kudu/client/client.cc:
http://gerrit.cloudera.org:8080/#/c/18846/27/src/kudu/client/client.cc@2019
PS27, Line 2019: }
:
: VLOG(2) << "Beginning " << data_->DebugString();
:
> It seems this comment hasn't been answered (it was missed?).
Sorry. I maybe not understand what you mean exactly. Could you give more
detail? Here ,I just want to set a random query id when the user does not set
the query id. Every query needs a unique query id even if the user does not
set the query id.
And Java client will be added the same functionality at the next commit.
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/client.proto
File src/kudu/client/client.proto:
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/client.proto@201
PS30, Line 201: Identifier of a top-level query
> How about:
Done
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/scan_token-test.cc
File src/kudu/client/scan_token-test.cc:
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/scan_token-test.cc@703
PS30, Line 703: replication factor.
> nit: but the code below sets the replication factor to 1?
Done
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/scan_token-test.cc@709
PS30, Line 709: workload.Start();
: ASSERT_EVENTUALLY([&]() { ASSERT_GE
> Is this crucial to customize these in the context of this test? If yes, pl
Done
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/scan_token-test.cc@729
PS30, Line 729: unique_ptr scanner;
: ASSERT_OK(IntoUniqueScanner(client_.get(), *token,
&scanner));
> Does it make sense to check for the 'query_id' field in scanner->data_->nex
Good advise. A check will be added: ASSERT_EQ("query-id-for-test",
scanner->data_->next_req_.query_id());
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/scan_token-test.cc@731
PS30, Line 731: ASSERT_OK(scanner->Open());
: ASSERT_EQ("query-id-for-test",
scanner->data_->next_req_.query_id());
: KuduScanBatch batch;
: ASSERT_TRUE(scanner->HasMoreRows());
:
:
> nit: maybe, check for the number of scanned rows to equal to workload.rows_
Done
http://gerrit.cloudera.org:8080/#/c/18846/30/src/kudu/client/scan_token-test.cc@746
PS30, Line 746: vector tokens;
: KuduScanTokenBuilder builder(table.get());
> Does it make sense to check for the 'query_id' field in scanner->data_->nex
Good advise. A check will be added:
ASSERT_FALSE(scanner->data_->next_req_.query_id
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19501 Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the table id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_tablet.cc 8 files changed, 110 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/1 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [Tool] Show the information of a tablet
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#2). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the table id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool_action_tablet.cc 8 files changed, 110 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/2 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Hello Yuqi Du, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19493 to look at the new patch set (#3). Change subject: [Tools] Remove enter key when listing non-existed table .. [Tools] Remove enter key when listing non-existed table When Kudu listing a non-existed table using command line tool, it ouputs an enter key and a new line appears. But earlier version does not output an enter key. Therefore it is better to be compatible with old versions of Kudu. Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 --- M src/kudu/tools/tool_action_table.cc 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/19493/3 -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Hello Yuqi Du, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19493 to look at the new patch set (#4). Change subject: [Tools] Remove enter key when listing non-existed table .. [Tools] Remove enter key when listing non-existed table When Kudu listing a non-existed table using command line tool, it ouputs an enter key and a new line appears. But earlier version does not output an enter key. Therefore it is better to be compatible with old versions of Kudu. Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 --- M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool_action_table.cc 2 files changed, 3 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/19493/4 -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19493 ) Change subject: [Tools] Remove enter key when listing non-existed table .. Patch Set 4: (1 comment) > Patch Set 2: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/19493/2/src/kudu/tools/tool_action_table.cc File src/kudu/tools/tool_action_table.cc: http://gerrit.cloudera.org:8080/#/c/19493/2/src/kudu/tools/tool_action_table.cc@361 PS2, Line 361: if (!tables_info_pb.tables().empty()) > what about: Good advice. -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 16 Feb 2023 08:46:41 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3447] Limit tablets copying speed
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19479 ) Change subject: [KUDU-3447] Limit tablets copying speed .. Patch Set 3: > Patch Set 2: > > (8 comments) > > > Patch Set 1: > > > > (8 comments) > > > > Thank you for adding this. It would really help if you could add a little > > more details about the following: > > 1. I understand that you have mentioned about other services getting > > impacted due to network jam caused by tablet copy operation, but that > > doesn't explain why tablet copy operation would become a preferred > > bandwidth user over other services. Is there an existing case where this > > kind of behaviour was seen and what was making linux kernel to do that? > > 2. Adding detailed information (in commit message) about the new library > > APIs would be quite helpful for posterity. > > 3. Limiting the network speed might cause frequent context switching. Do we > > have any data on that from the performance point of view? Performance test was done, please see: https://issues.apache.org/jira/browse/KUDU-3447 -- To view, visit http://gerrit.cloudera.org:8080/19479 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1f4834bfb0718a2b6b1d946975287a11f6be1fe3 Gerrit-Change-Number: 19479 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Comment-Date: Thu, 16 Feb 2023 09:36:49 + Gerrit-HasComments: No
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Hello Yuqi Du, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19493 to look at the new patch set (#5). Change subject: [Tools] Remove enter key when listing non-existed table .. [Tools] Remove enter key when listing non-existed table When Kudu listing a non-existed table using command line tool, it ouputs an enter key and a new line appears. But earlier version does not output an enter key. Therefore it is better to be compatible with old versions of Kudu. Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 --- M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool_action_table.cc 2 files changed, 4 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/93/19493/5 -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19493 ) Change subject: [Tools] Remove enter key when listing non-existed table .. Patch Set 5: (1 comment) > Patch Set 4: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/19493/4/src/kudu/tools/tool_action_table.cc File src/kudu/tools/tool_action_table.cc: http://gerrit.cloudera.org:8080/#/c/19493/4/src/kudu/tools/tool_action_table.cc@362 PS4, Line 362: cout << ToPrettyFormat(tables_info_pb) << endl; > nit: add brackets Done -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 17 Feb 2023 02:18:20 + Gerrit-HasComments: Yes
[kudu-CR] [Client] Add query id to trace the whole query process
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19518 Change subject: [Client] Add query id to trace the whole query process .. [Client] Add query id to trace the whole query process This patch continues the work: https://gerrit.cloudera.org/c/18846. Query id is used to trace a query from Impala to Kudu for troubleshooting and debugging. Change-Id: Iddbd348e766bad1b5648b4091635679319b1e2fd --- M java/kudu-client/src/main/java/org/apache/kudu/client/AbstractKuduScannerBuilder.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanToken.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestScanToken.java 6 files changed, 127 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/19518/1 -- To view, visit http://gerrit.cloudera.org:8080/19518 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Iddbd348e766bad1b5648b4091635679319b1e2fd Gerrit-Change-Number: 19518 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [Client] Add query id to trace the whole query process
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19518 to look at the new patch set (#2). Change subject: [Client] Add query id to trace the whole query process .. [Client] Add query id to trace the whole query process This patch continues the work: https://gerrit.cloudera.org/c/18846. Query id is used to trace a query from Impala to Kudu for troubleshooting and debugging. Change-Id: Iddbd348e766bad1b5648b4091635679319b1e2fd --- M java/kudu-client/src/main/java/org/apache/kudu/client/AbstractKuduScannerBuilder.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduScanner.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanToken.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduScanner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestScanToken.java 6 files changed, 125 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/18/19518/2 -- To view, visit http://gerrit.cloudera.org:8080/19518 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iddbd348e766bad1b5648b4091635679319b1e2fd Gerrit-Change-Number: 19518 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19493 ) Change subject: [Tools] Remove enter key when listing non-existed table .. Patch Set 5: (1 comment) > Patch Set 5: Code-Review+1 > > (1 comment) http://gerrit.cloudera.org:8080/#/c/19493/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19493/5//COMMIT_MSG@10 PS5, Line 10: it ouputs an enter key and a new line appears > I'm curious about what issue this new line cause? In my Python program, it uses this command line to judge whether a table existed. In old versions, it returns empty result if a table not existed, but in new versions, it return an enter key. -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 22 Feb 2023 01:54:08 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.17.x) [docs] Add release notes for 1.17.0
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19512 ) Change subject: [docs] Add release notes for 1.17.0 .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/19512/1/docs/release_notes.adoc File docs/release_notes.adoc: http://gerrit.cloudera.org:8080/#/c/19512/1/docs/release_notes.adoc@111 PS1, Line 111: specify the partial tables that need to be rebuilt, and it will not take effect on the existing other tables on Kudu master. rebuit the metadata of specified tables on Kudu master, and it has no effect on the other tables. http://gerrit.cloudera.org:8080/#/c/19512/1/docs/release_notes.adoc@125 PS1, Line 125: in batches in batch http://gerrit.cloudera.org:8080/#/c/19512/1/docs/release_notes.adoc@168 PS1, Line 168: `--list_detail` flag the `--list_detail` flag http://gerrit.cloudera.org:8080/#/c/19512/1/docs/release_notes.adoc@168 PS1, Line 168: crashes crash -- To view, visit http://gerrit.cloudera.org:8080/19512 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: comment Gerrit-Change-Id: I4381e7824dcf7f500165aa3af9dca09a86ac1dd9 Gerrit-Change-Number: 19512 Gerrit-PatchSet: 1 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Hongjiang Zhang Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Khazar Mammadli Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Mahesh Reddy Gerrit-Reviewer: Marton Greber Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Wenzhe Zhou Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: Zoltan Chovan Gerrit-Reviewer: bsglz <18031...@qq.com> Gerrit-Reviewer: harinreddy Gerrit-Reviewer: yejiabao Gerrit-Reviewer: zhanghongbin <1079039...@qq.com> Gerrit-Reviewer: Ádám Bakai Gerrit-Comment-Date: Wed, 22 Feb 2023 03:52:15 + Gerrit-HasComments: Yes
[kudu-CR] [fs] make CommitDeletedBlocks output parameter nullable
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19520 ) Change subject: [fs] make CommitDeletedBlocks output parameter nullable .. Patch Set 3: Code-Review+1 LGTM -- To view, visit http://gerrit.cloudera.org:8080/19520 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia92c36011cf8fc63d58fad7da0e08554b9d5dad6 Gerrit-Change-Number: 19520 Gerrit-PatchSet: 3 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 22 Feb 2023 04:06:42 + Gerrit-HasComments: No
[kudu-CR] [Tools] Remove enter key when listing non-existed table
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19493 ) Change subject: [Tools] Remove enter key when listing non-existed table .. Patch Set 5: > Patch Set 5: -Code-Review > > (1 comment) A new line. -- To view, visit http://gerrit.cloudera.org:8080/19493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19493 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 24 Feb 2023 04:40:34 + Gerrit-HasComments: No
[kudu-CR] [Tool] Show the information of a tablet
Hello Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#3). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the table id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 9 files changed, 169 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/3 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19501 )
Change subject: [Tool] Show the information of a tablet
..
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19501/2/src/kudu/client/client.h
File src/kudu/client/client.h:
http://gerrit.cloudera.org:8080/#/c/19501/2/src/kudu/client/client.h@1180
PS2, Line 1180: const std::string& table_id() const;
: const std::string& table_name() const;
> Expose functions to user client should be very carefully, we have to mainte
KuduTablet::Data is private to CLI tools. Friendly class seems to be no
suitable also.
A similar data structure: TabletInfo defined in catalog_manager.h also contains
the information of table. KuduTablet contains table_id, table_name may be ok.
http://gerrit.cloudera.org:8080/#/c/19501/2/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/19501/2/src/kudu/master/master.proto@449
PS2, Line 449: optional string table_name = 9;
> If we want to use the feature we have to upgrade the servers, can we gather
Currently no suitable functions found to get the table info according tablet id.
http://gerrit.cloudera.org:8080/#/c/19501/2/src/kudu/tools/tool_action_tablet.cc
File src/kudu/tools/tool_action_tablet.cc:
http://gerrit.cloudera.org:8080/#/c/19501/2/src/kudu/tools/tool_action_tablet.cc@300
PS2, Line 300: for (const auto& replica : tablet->replicas()) {
: TabletInfoPB::LocationIn
> nit: return cmatrix.PrintTo(cout); ?
The out parameter of PrintTo() is ostream
--
To view, visit http://gerrit.cloudera.org:8080/19501
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe
Gerrit-Change-Number: 19501
Gerrit-PatchSet: 3
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai
Gerrit-Comment-Date: Fri, 24 Feb 2023 06:42:17 +
Gerrit-HasComments: Yes
[kudu-CR] [Tool] Delete destination table before copy
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19544 Change subject: [Tool] Delete destination table before copy .. [Tool] Delete destination table before copy Add a new flag: --delete_dst_table_before_copy to table copy. Whether the destination table existed or not when flags: --create_table and --delete_dst_table_before_copy are true, it will delete destination table before table copy. Change-Id: I5cb8487e7cb2a616ea2ef1ebac2379e79cdbd265 --- M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/table_scanner.cc M src/kudu/tools/tool_action_table.cc 3 files changed, 36 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/44/19544/1 -- To view, visit http://gerrit.cloudera.org:8080/19544 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5cb8487e7cb2a616ea2ef1ebac2379e79cdbd265 Gerrit-Change-Number: 19544 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [Tool] Delete destination table before copy
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19544 ) Change subject: [Tool] Delete destination table before copy .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/19544/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19544/1//COMMIT_MSG@7 PS1, Line 7: Delete destination table before copy > Is there any specific objective of adding this apart from avoiding to run t In my case, we use the command: 'kudu table copy' to migrate data from an old Kudu cluster to a new Kudu cluster. Sometimes, the copy maybe fail, and it will try again, but the table already existed in the new cluster. The try will fail. But anyway, you are right. Combine two commands into one command will make Kudu become fat. Kudu should focus on storage and performance. Thank you for your comments. -- To view, visit http://gerrit.cloudera.org:8080/19544 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5cb8487e7cb2a616ea2ef1ebac2379e79cdbd265 Gerrit-Change-Number: 19544 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Comment-Date: Tue, 28 Feb 2023 01:57:05 + Gerrit-HasComments: Yes
[kudu-CR] [Tool] Delete destination table before copy
Wang Xixu has abandoned this change. ( http://gerrit.cloudera.org:8080/19544 ) Change subject: [Tool] Delete destination table before copy .. Abandoned not suitable -- To view, visit http://gerrit.cloudera.org:8080/19544 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: abandon Gerrit-Change-Id: I5cb8487e7cb2a616ea2ef1ebac2379e79cdbd265 Gerrit-Change-Number: 19544 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
[kudu-CR] [Tool] Show the information of a tablet
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#4). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 287 insertions(+), 76 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/4 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19501 ) Change subject: [Tool] Show the information of a tablet .. Patch Set 4: (9 comments) http://gerrit.cloudera.org:8080/#/c/19501/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19501/3//COMMIT_MSG@13 PS3, Line 13: tablet i > tablet id? Done http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/client/client.h File src/kudu/client/client.h: http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/client/client.h@1180 PS3, Line 1180: const std::string& id() const; : > I don't think these new methods are needed in the public client API if only Done http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/client/tablet-internal.h File src/kudu/client/tablet-internal.h: http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/client/tablet-internal.h@31 PS3, Line 31: Data > +1 Done http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/master/master.proto File src/kudu/master/master.proto: PS3: > The newly added fields TabletLocationsPB::table_id and TabletLocationsPB::t Done http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/master/master.proto@448 PS3, Line 448: // DEPRECATED. > Why do we need to assign values from 8? The field interned_replicas is 7. http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/master/master.proto@448 PS3, Line 448: // DEPRECATED. : // Used only if interned replic > Whether is it necessary to move them to the end? Done http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/tools/tool_action_tablet.cc File src/kudu/tools/tool_action_tablet.cc: http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/tools/tool_action_tablet.cc@295 PS3, Line 295: K(CreateKudu > 'FLAGS_format' has many values as below. Done http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/tools/tool_action_tablet.cc@321 PS3, Line 321: cerr << "Replaced tablet " << tablet_id << " with tablet "; > nit: remote an empty line Done http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/tools/tool_action_tablet.cc@409 PS3, Line 409: ilable.\n > Alternatively, it could be just 'info'? Done -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Wed, 01 Mar 2023 09:59:39 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.17.x) [Tools] Remove enter key when listing non-existed table
Hello Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/19577 to review the following change. Change subject: [Tools] Remove enter key when listing non-existed table .. [Tools] Remove enter key when listing non-existed table When Kudu listing a non-existed table using command line tool, it ouputs an enter key and a new line appears. But earlier version does not output an enter key. Therefore it is better to be compatible with old versions of Kudu. Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Reviewed-on: http://gerrit.cloudera.org:8080/19493 Tested-by: Kudu Jenkins Reviewed-by: Yingchun Lai Reviewed-by: Yuqi Du Reviewed-by: Yifan Zhang --- M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool_action_table.cc 2 files changed, 4 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/77/19577/1 -- To view, visit http://gerrit.cloudera.org:8080/19577 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: newchange Gerrit-Change-Id: I86fafc3c7df26d380b424b6d6b3f80a1b8d78dc8 Gerrit-Change-Number: 19577 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [LBM] append CREATE/DELETE records in batch to metadata
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19519 ) Change subject: [LBM] append CREATE/DELETE records in batch to metadata .. Patch Set 10: (1 comment) http://gerrit.cloudera.org:8080/#/c/19519/10/src/kudu/fs/log_block_manager.cc File src/kudu/fs/log_block_manager.cc: http://gerrit.cloudera.org:8080/#/c/19519/10/src/kudu/fs/log_block_manager.cc@1318 PS10, Line 1318: unable to append block's metadata How about: unable to append block's metadata for created records -- To view, visit http://gerrit.cloudera.org:8080/19519 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie24adab3b1dbbea55108a1f525093136fcf5a726 Gerrit-Change-Number: 19519 Gerrit-PatchSet: 10 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Sat, 04 Mar 2023 09:31:46 + Gerrit-HasComments: Yes
[kudu-CR] [server] add 'slow scans' metric for tablet server
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19545 ) Change subject: [server] add 'slow_scans' metric for tablet server .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19545 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51e64bc4602f0e1dd99207f7d4cf8883085eca9a Gerrit-Change-Number: 19545 Gerrit-PatchSet: 4 Gerrit-Owner: KeDeng Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Comment-Date: Sat, 04 Mar 2023 09:33:56 + Gerrit-HasComments: No
[kudu-CR](branch-1.17.x) update
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19580 Change subject: update .. update Change-Id: Icb846e34399309749352c8c2e2dfa9bde69f44ad --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-tool-test.cc 2 files changed, 28 insertions(+), 16 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/19580/1 -- To view, visit http://gerrit.cloudera.org:8080/19580 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: newchange Gerrit-Change-Id: Icb846e34399309749352c8c2e2dfa9bde69f44ad Gerrit-Change-Number: 19580 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR](branch-1.17.x) update
Wang Xixu has abandoned this change. ( http://gerrit.cloudera.org:8080/19580 ) Change subject: update .. Abandoned test -- To view, visit http://gerrit.cloudera.org:8080/19580 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: abandon Gerrit-Change-Id: Icb846e34399309749352c8c2e2dfa9bde69f44ad Gerrit-Change-Number: 19580 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.17.x) [KUDU-3452] Create tablet without enough healthy tservers
Wang Xixu has abandoned this change. ( http://gerrit.cloudera.org:8080/19579 ) Change subject: [KUDU-3452] Create tablet without enough healthy tservers .. Abandoned test -- To view, visit http://gerrit.cloudera.org:8080/19579 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: abandon Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 Gerrit-Change-Number: 19579 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [KUDU-3452] A tool to show creating tables
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19584 Change subject: [KUDU-3452] A tool to show creating tables .. [KUDU-3452] A tool to show creating tables When creating a new table or partition, new tablets will be created. If the healthy tablet servers are not enough to create multiple replicas. The creating task will retry continuously for a long time. Therefore it needs a tool to show the creating tables. And users can decide to cancel the creating tasks or not. This patch adds a new flag: --list_creating_tables for command: kudu table list to show the creating tables. It will show the table id, table name, the number of total tablets, the number of creating tables and state. Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_table.cc 7 files changed, 223 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/19584/1 -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [LBM] append CREATE/DELETE records in batch to metadata
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19519 ) Change subject: [LBM] append CREATE/DELETE records in batch to metadata .. Patch Set 11: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19519 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie24adab3b1dbbea55108a1f525093136fcf5a726 Gerrit-Change-Number: 19519 Gerrit-PatchSet: 11 Gerrit-Owner: Yingchun Lai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 06 Mar 2023 11:24:20 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19584 to look at the new patch set (#2). Change subject: [KUDU-3452] A tool to show on creating tables .. [KUDU-3452] A tool to show on creating tables When creating a new table or partition, new tablets will be created. If the healthy tablet servers are not enough to create multiple replicas. The creating task will retry continuously for a long time. Therefore it needs a tool to show the creating tables. And users can decide to cancel the creating tasks or not. This patch adds a new flag: --list_creating_tables for command: kudu table list to show the creating tables. It will show the table id, table name, the number of total tablets, the number of creating tables and state. Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_table.cc 7 files changed, 223 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/19584/2 -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19584 ) Change subject: [KUDU-3452] A tool to show on creating tables .. Patch Set 2: > Patch Set 1: > > (2 comments) done -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Comment-Date: Tue, 07 Mar 2023 02:13:08 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19594 Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table when one of three tablet servers will timeout. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 62 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/1 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <1450306...@qq.com>
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#2). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table when one of three tablet servers will timeout. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 67 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/2 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [Tool] Show the information of a tablet
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#5). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 287 insertions(+), 76 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/5 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19501 ) Change subject: [Tool] Show the information of a tablet .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/19501/4/src/kudu/master/master.proto File src/kudu/master/master.proto: http://gerrit.cloudera.org:8080/#/c/19501/4/src/kudu/master/master.proto@459 PS4, Line 459: field > nit: field Done -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Tue, 07 Mar 2023 11:05:27 + Gerrit-HasComments: Yes
[kudu-CR] [codegen] fix build on Ubuntu 22.04
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19582 ) Change subject: [codegen] fix build on Ubuntu 22.04 .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19582 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If65f02b169098afec0d5682a7be6066568420982 Gerrit-Change-Number: 19582 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Comment-Date: Wed, 08 Mar 2023 09:18:19 + Gerrit-HasComments: No
[kudu-CR] KUDU-3437 Set default block cache metrics policy to reset
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19585 )
Change subject: KUDU-3437 Set default block_cache_metrics_policy to reset
..
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19585/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19585/3//COMMIT_MSG@9
PS3, Line 9: empty local directories
Why a new master joins the cluster, it will execute Master::Init() twice?
Cloud you give more detail?
http://gerrit.cloudera.org:8080/#/c/19585/3/src/kudu/master/master_options.cc
File src/kudu/master/master_options.cc:
http://gerrit.cloudera.org:8080/#/c/19585/3/src/kudu/master/master_options.cc@43
PS3, Line 43: :
block_cache_metrics_policy_(Cache::ExistingMetricsPolicy::kReset) {
Maybe it needs a unit test to prove this modification available. Or a related
unit test has exited already.
--
To view, visit http://gerrit.cloudera.org:8080/19585
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idc32d7ab02201382debcbe36311579550353bf71
Gerrit-Change-Number: 19585
Gerrit-PatchSet: 3
Gerrit-Owner: Ádám Bakai
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Attila Bukor
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Comment-Date: Wed, 08 Mar 2023 09:49:50 +
Gerrit-HasComments: Yes
[kudu-CR] [java] KUDU-3455 Improve space complexity about prune hash partitions for in-list predicate
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19568 ) Change subject: [java] KUDU-3455 Improve space complexity about prune hash partitions for in-list predicate .. Patch Set 4: (8 comments) http://gerrit.cloudera.org:8080/#/c/19568/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19568/4//COMMIT_MSG@7 PS4, Line 7: Improve Reduce? http://gerrit.cloudera.org:8080/#/c/19568/4//COMMIT_MSG@7 PS4, Line 7: prune nit: pruning http://gerrit.cloudera.org:8080/#/c/19568/4//COMMIT_MSG@9 PS4, Line 9: this nit: the current http://gerrit.cloudera.org:8080/#/c/19568/4//COMMIT_MSG@12 PS4, Line 12: those make which makes http://gerrit.cloudera.org:8080/#/c/19568/4//COMMIT_MSG@15 PS4, Line 15: method nit: algorithm http://gerrit.cloudera.org:8080/#/c/19568/4/java/kudu-client/src/main/java/org/apache/kudu/client/PartitionPruner.java File java/kudu-client/src/main/java/org/apache/kudu/client/PartitionPruner.java: http://gerrit.cloudera.org:8080/#/c/19568/4/java/kudu-client/src/main/java/org/apache/kudu/client/PartitionPruner.java@767 PS4, Line 767: Iterator it = indexList.iterator(); You can use indexList.get(i) to any elements. http://gerrit.cloudera.org:8080/#/c/19568/4/java/kudu-client/src/main/java/org/apache/kudu/client/PartitionPruner.java@770 PS4, Line 770: assert (it.hasNext()); assert will cause system exit. Is it suitable? http://gerrit.cloudera.org:8080/#/c/19568/4/java/kudu-client/src/main/java/org/apache/kudu/client/PartitionPruner.java@777 PS4, Line 777: assert (level < columnIdxs.size()); Remove it. it will never happen. -- To view, visit http://gerrit.cloudera.org:8080/19568 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icd6f213cb705e1b2a001562cc7cebe4164281723 Gerrit-Change-Number: 19568 Gerrit-PatchSet: 4 Gerrit-Owner: Yuqi Du Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 09 Mar 2023 03:45:29 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Create tablet without enough healthy tservers
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19571 ) Change subject: [KUDU-3452] Create tablet without enough healthy tservers .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/master/catalog_manager.cc@5930 PS2, Line 5930: master_->ts_manager()->GetDescriptorsAvailableForPlacement(&ts_descs); > Why move the statement from outer to inner loop? I infer that at a unsteady Why move the statement from outer to inner loop? It used healthy tablet servers totally before. But now the case is different. For example, there are 3 tablet servers, one is unhealthy. Some tablet defined three-replicas, some defined two-replicas. The case is different for different tablets. Therefore, the logic is: firstly, try to use the healthy tablet servers, if it is not enough, try to use unhealthy tablet servers. -- To view, visit http://gerrit.cloudera.org:8080/19571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 Gerrit-Change-Number: 19571 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 10 Mar 2023 06:01:54 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3452 Create tablet without enough healthy tservers
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19571 )
Change subject: KUDU-3452 Create tablet without enough healthy tservers
..
Patch Set 3:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/19571/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19571/2//COMMIT_MSG@7
PS2, Line 7: KUDU-3452 Create tablet without enough healthy tservers
> add a tag like: [master] ?
Done
http://gerrit.cloudera.org:8080/#/c/19571/2//COMMIT_MSG@21
PS2, Line 21: create tablet without enough
: healthy tservers can success. And its status is under
: replicating
> We don't need the number of healthy tserver greater or equal 'replication_f
ok
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/master/catalog_manager.cc@5930
PS2, Line 5930: const int alive_num_ts = ts_descs.size();
> Why move the statement from outer to inner loop? It used healthy tablet ser
Done
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/master/catalog_manager.cc@5931
PS2, Line 5931: PlacementPolicy policy(std::move(ts_descs), &rng_);
> Whether the lock can put into { TableMetadataLock table_guard(tablet->tab
Done
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/master/catalog_manager.cc@5932
PS2, Line 5932: for (auto& tablet : deferred.needs_create_rpc) {
: int nreplicas = 0;
: // Get replica factor.
: {
: TableMetadataLock table_guard(tablet->table().get(),
LockMode::READ);
:
> At this, I have different opinion about this which may relation to lines fr
Done
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/tools/kudu-tool-test.cc@9182
PS2, Line 9182: .
> 'catalog_manager_check_ts_count_for_create_table''s default value is true.
Yes
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/tools/kudu-tool-test.cc@9203
PS2, Line 9203: // Sleep for 2s to make the state of one tab
> replacing the 'SleepFor' by checking 'ksck not OK' ?
Ksck can not make master know that tablet server is offline. Tablet server will
report its status through heart beat every tserver_unresponsive_timeout_ms time.
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/tools/kudu-tool-test.cc@9209
PS2, Line 9209: workload2.set_table_name(kTableName2);
> IMO, it's better start a write thread and write some records to prove the t
Done
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/tools/kudu-tool-test.cc@9213
PS2, Line 9213: 2.rows_inserted(
> replacing the 'SleepFor' by checking 'ksck not OK' ?
Ksck can not make master know that tablet server is offline. Tablet server will
report its status through heart beat every tserver_unresponsive_timeout_ms time.
--
To view, visit http://gerrit.cloudera.org:8080/19571
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6
Gerrit-Change-Number: 19571
Gerrit-PatchSet: 3
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Mon, 13 Mar 2023 07:44:02 +
Gerrit-HasComments: Yes
[kudu-CR] [master] exclude tservers at MAINTENANCE MODE when leader rebalancing
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19608 )
Change subject: [master] exclude tservers at MAINTENANCE_MODE when leader
rebalancing
..
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer-test.cc
File src/kudu/master/auto_leader_rebalancer-test.cc:
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer-test.cc@215
PS4, Line 215: MaintenanceMode
Need prefix: Test
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer-test.cc@239
PS4, Line 239: ASSERT_OK(Subprocess::Call(Substitute("$0/kudu tserver state
enter_maintenance $1 $2",
Add some comments.
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer-test.cc@253
PS4, Line 253: SleepFor(MonoDelta::FromSeconds(10 *
FLAGS_auto_rebalancing_interval_seconds));
This is a slow test. Maybe need add SKIP_IF_SLOW_NOT_ALLOWED();
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer.h
File src/kudu/master/auto_leader_rebalancer.h:
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer.h@79
PS4, Line 79: exclude_dest_uuids
How about: exclude_tserver_uuids?
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer.cc
File src/kudu/master/auto_leader_rebalancer.cc:
http://gerrit.cloudera.org:8080/#/c/19608/4/src/kudu/master/auto_leader_rebalancer.cc@251
PS4, Line 251: count
How about using: ContainsKey()
--
To view, visit http://gerrit.cloudera.org:8080/19608
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2f85a675e69fd02a62e2625881dad2ca5e27acd9
Gerrit-Change-Number: 19608
Gerrit-PatchSet: 4
Gerrit-Owner: Yuqi Du
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mahesh Reddy
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Mon, 13 Mar 2023 08:04:38 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19584 ) Change subject: [KUDU-3452] A tool to show on creating tables .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@7 PS2, Line 7: A tool to show on creating tables > Is this for tablets or tables that are in process of creation? Yes http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@11 PS2, Line 11: The creating task will retry continuously for a long : time. > What is the timeout here? The client side will timeout, but the server side will never timeout. http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@15 PS2, Line 15: cancel the creating tasks > Can you please elaborate on this, how cancelation of creation task can happ For example, a Kudu cluster with 3 masters and 3 tservers. if one tserver gets offline, creating a three-replicas will timeout on client side, but on server side, the creating task is still running for a long time. If the unhealthy tserver can never become healthy agin. The creating task will exist. It is meaningless. It needs to cancel this creating task. -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 2 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Tue, 14 Mar 2023 01:47:39 + Gerrit-HasComments: Yes
[kudu-CR] [Tool] Compact metadata file for old versions of Kudu
Hello Tidy Bot, Yingchun Lai, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19327 to look at the new patch set (#5). Change subject: [Tool] Compact metadata file for old versions of Kudu .. [Tool] Compact metadata file for old versions of Kudu Some old versions of Kudu do not support compacting metadata file in runtime. A long time later, metadata file becomes very big, for example, more than 100G. When restarting Kudu tablet server, it will read all metadata files to check whick blocks are still alive and compact metadata files into new small metadata files which only record alive blocks. As the metadata files are very big, compacting metadata files will cost a lot of time, maybe 2-3 hours. That will cause the service unavailabe for a long time, which is unacceptable. Runtime metadata file compacting has already been supported above version of 1.16.0, see jira: #KUDU-3318. Upgrading old version to above Kudu 1.16.0 still needs to restart Kudu. This tool helps to compact metdata file for old versions of Kudu. It supports 4 operations: --print: Statistic the total blocks and alive blocks for every container. --compact: Compact metadata files when Kudu is still running --merge: Stop Kudu and write incremental records into compacted metadata files. Then backup old metadata files. --clear: Delete metadata backup files. Restarting old version of Kudu maybe cost 2-3 hours, but using compact tool, service stopped only lasts 3 minutes. Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4 --- M src/kudu/fs/log_block_manager-test.cc M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h M src/kudu/tools/tool_action_fs.cc M src/kudu/util/pb_util.cc M src/kudu/util/pb_util.h 6 files changed, 585 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/19327/5 -- To view, visit http://gerrit.cloudera.org:8080/19327 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4 Gerrit-Change-Number: 19327 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai
[kudu-CR] [Tool] Show the information of a tablet
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#6). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 290 insertions(+), 76 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/6 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 6 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#7). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. The follow is an exmple: Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table UUID | Host | Port | Is Leader --+-+---+--- 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 293 insertions(+), 78 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/7 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 7 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19501 )
Change subject: [Tool] Show the information of a tablet
..
Patch Set 7:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/19501/5//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19501/5//COMMIT_MSG@9
PS5, Line 9: This tool is used to show the table name, table id and all
: replicas on different tablet servers according to the tablet
: id.
> Could give s sample output?
Done
http://gerrit.cloudera.org:8080/#/c/19501/6/src/kudu/client/client-internal.cc
File src/kudu/client/client-internal.cc:
http://gerrit.cloudera.org:8080/#/c/19501/6/src/kudu/client/client-internal.cc@298
PS6, Line 298: Status KuduClient::Data::GetTablet(KuduClient* client,
> warning: method 'GetTablet' can be made const [readability-make-member-func
Done
http://gerrit.cloudera.org:8080/#/c/19501/6/src/kudu/client/client-internal.cc@373
PS6, Line 373: Status KuduClient::Data::GetTabletServer(KuduClient* client,
> warning: method 'GetTabletServer' can be made const [readability-make-membe
Done
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/master/master.proto@460
PS5, Line 460: es id = 1;
> I guess this field rather belongs to GetTabletLocationsRequestPB?
You are right. I make a mismake.
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/master/master.proto@461
PS5, Line 461: optional string name = 2;
: }
> It would be great to create a dedicated message for this and add it just as
Done
http://gerrit.cloudera.org:8080/#/c/19501/6/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19501/6/src/kudu/tools/kudu-tool-test.cc@3967
PS6, Line 3967: ASSERT_STR_CONTAINS(stdout, workload_table->id());
> warning: 'opts' used after it was moved [bugprone-use-after-move]
Done
http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/tools/tool_action_tablet.cc
File src/kudu/tools/tool_action_tablet.cc:
http://gerrit.cloudera.org:8080/#/c/19501/3/src/kudu/tools/tool_action_tablet.cc@409
PS3, Line 409: { kTablet
> +1
Done
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/tools/tool_action_tablet.cc
File src/kudu/tools/tool_action_tablet.cc:
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/tools/tool_action_tablet.cc@101
PS5, Line 101: ShowTabletI
> This is C++, not Java, so I don't understand why to introduce otherwise use
OK. I will use friend function.
That is funny. Shower is an equipment of bathing. So learn English is not only
about grammar but also culture.
Thank you for your comments.
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/tools/tool_action_tablet.cc@126
PS5, Line 126: DataTable cmatrix({ "UUID", "Host
> Why would you need this? std::cout is std::ostream as well, so it fits whe
Done
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/tools/tool_action_tablet.cc@127
PS5, Line 127: r (
> Would it be enough using 'cout' here?
Done
http://gerrit.cloudera.org:8080/#/c/19501/5/src/kudu/tools/tool_action_tablet.cc@136
PS5, Line 136: na
> nit: wrong indent?
Done
--
To view, visit http://gerrit.cloudera.org:8080/19501
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe
Gerrit-Change-Number: 19501
Gerrit-PatchSet: 7
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Tue, 14 Mar 2023 13:23:58 +
Gerrit-HasComments: Yes
[kudu-CR] [Tool] Show the information of a tablet
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#8). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. The following is an exmple: Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table Tablet Server UUID| Host | Port | Is Leader --+-+---+--- 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 293 insertions(+), 78 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/8 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 8 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#9). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. The following is an exmple: Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table Tablet Server UUID| Host | Port | Is Leader --+-+---+--- 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 294 insertions(+), 79 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/9 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 9 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19501 ) Change subject: [Tool] Show the information of a tablet .. Patch Set 9: (4 comments) > Patch Set 8: > > (1 comment) http://gerrit.cloudera.org:8080/#/c/19501/7//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19501/7//COMMIT_MSG@17 PS7, Line 17: follow > following Done http://gerrit.cloudera.org:8080/#/c/19501/7/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: http://gerrit.cloudera.org:8080/#/c/19501/7/src/kudu/master/catalog_manager.cc@6276 PS7, Line 6276: > nit: you could use std::move() here to prevent copying: Done http://gerrit.cloudera.org:8080/#/c/19501/7/src/kudu/tools/tool_action_tablet.cc File src/kudu/tools/tool_action_tablet.cc: http://gerrit.cloudera.org:8080/#/c/19501/7/src/kudu/tools/tool_action_tablet.cc@126 PS7, Line 126: Tabl > nit: Tablet Server UUID Done http://gerrit.cloudera.org:8080/#/c/19501/7/src/kudu/tools/tool_action_tablet.cc@414 PS7, Line 414: Show information on the table which the tablet is a > nit: maybe change this to be more specific Done. Thanks for your comments. -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 9 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 20 Mar 2023 03:19:06 + Gerrit-HasComments: Yes
[kudu-CR] [Tool] Show the information of a tablet
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#10). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. The following is an exmple: Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table Tablet Server UUID| Host | Port | Is Leader --+-+---+--- 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 283 insertions(+), 79 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/10 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 10 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19501 )
Change subject: [Tool] Show the information of a tablet
..
Patch Set 9:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/client/client-internal.cc
File src/kudu/client/client-internal.cc:
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/client/client-internal.cc@298
PS9, Line 298: Status KuduClient::Data::GetTablet(KuduClient* client,
> I'm not clear why to move this function from KuduClient to KuduClient::Data
Just to keep the same style. Client.cc exposes the interface to the user, and
client-internal.cc implements the interface. Please see the following function:
GetTabletServer()
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/client/client-internal.cc@301
PS9, Line 301: const
> At least, it can be static (in L307, use client->default_admin_operation_ti
Done L307.
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/master/catalog_manager.h
File src/kudu/master/catalog_manager.h:
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/master/catalog_manager.h@797
PS9, Line 797: get_extra_info
> Could you plz add some comments to describe what dose 'get_extra_info' mean
Done
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/master/catalog_manager.cc@6241
PS9, Line 6241: R
> nit: indent
Done
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/tools/tool_action_tablet.cc
File src/kudu/tools/tool_action_tablet.cc:
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/tools/tool_action_tablet.cc@109
PS9, Line 109: if (FLAGS_json) {
> DataTable is support to output in JSON format(see DataTable::PrintTo), why
Done
http://gerrit.cloudera.org:8080/#/c/19501/9/src/kudu/tools/tool_action_tablet.cc@124
PS9, Line 124: cout << Substitute("Table id: $0, Table name: $1",
> So if --json is enabled, table id and name will not be output?
To simply the code, I will output table id and table name for every replica.
Maybe the data has some duplication, but it does matter. The following is an
example:
pretty format:
Table Id | Table Name |Tablet Server UUID
| Host | Port | Is Leader
--++--+---+---+---
883e5ffc9e2f41f780ddffd8a8744f13 | test_table |
5116514e00024a71926c722f5ccfaf44 | 127.42.53.193 | 41263 | false
883e5ffc9e2f41f780ddffd8a8744f13 | test_table |
7041740e966d45d38be3cc6c79c1f67f | 127.42.53.194 | 35579 | true
883e5ffc9e2f41f780ddffd8a8744f13 | test_table |
4c4d43e7e0bc4d41b23f072ad2dfe85d | 127.42.53.195 | 40701 | false
json format:
[{"Table Id":"53092288a6944385bcb78fdc6a2d5322","Table
Name":"test_table","Tablet Server
UUID":"909a581760ae40ceb8837d72cc989ede","Host":"127.45.86.65","Port":"44353","Is
Leader":"true"},{"Table Id":"53092288a6944385bcb78fdc6a2d5322","Table
Name":"test_table","Tablet Server
UUID":"b509cc1057af4dda9cc596c7f87849a1","Host":"127.45.86.66","Port":"42719","Is
Leader":"false"},{"Table Id":"53092288a6944385bcb78fdc6a2d5322","Table
Name":"test_table","Tablet Server
UUID":"9413e042a42549409367c722a50785e4","Host":"127.45.86.67","Port":"36483","Is
Leader":"false"}]
--
To view, visit http://gerrit.cloudera.org:8080/19501
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe
Gerrit-Change-Number: 19501
Gerrit-PatchSet: 9
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Thu, 23 Mar 2023 12:04:03 +
Gerrit-HasComments: Yes
[kudu-CR] [Tool] Show the information of a tablet
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#11). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. The following is an exmple: Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table Tablet Server UUID| Host | Port | Is Leader --+-+---+--- 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 279 insertions(+), 79 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/11 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 11 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] KUDU-3437 Set default block cache metrics policy to reset
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19585 ) Change subject: KUDU-3437 Set default block_cache_metrics_policy to reset .. Patch Set 3: > Patch Set 3: > > (1 comment) I got it. Thanks. Maybe you can destroy the singleton BlockCache before initiating the master at the second time, just like: ClearLocalSystemCatalogAndCopy(leader_hp). -- To view, visit http://gerrit.cloudera.org:8080/19585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc32d7ab02201382debcbe36311579550353bf71 Gerrit-Change-Number: 19585 Gerrit-PatchSet: 3 Gerrit-Owner: Ádám Bakai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: Zoltan Chovan Gerrit-Reviewer: Ádám Bakai Gerrit-Comment-Date: Fri, 24 Mar 2023 02:42:28 + Gerrit-HasComments: No
[kudu-CR] KUDU-3452 Create tablet without enough healthy tservers
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19571 )
Change subject: KUDU-3452 Create tablet without enough healthy tservers
..
Patch Set 4:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/19571/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19571/3//COMMIT_MSG@7
PS3, Line 7: Create tablet without enough healthy tservers
> Having a Kudu cluster with just 3 tablet servers is already quite limiting
Thank you for your comments. Please visit the related Jira and see the
discussion.
http://gerrit.cloudera.org:8080/#/c/19571/3//COMMIT_MSG@10
PS3, Line 10: will retry continuously
> It seems that the catalog manager's 'bgtasks' thread will continuously try
Yes
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@417
PS3, Line 417:
DEFINE_bool(enable_create_tablet_without_enough_healthy_tservers, false,
> We should also tag this new flag 'hidden', just as the 'catalog_manager_che
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@418
PS3, Line 418: creati
> creating
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@5933
PS3, Line 5933: // NOTE: if we fail to select replicas on the first pass
(due to
: // insufficient Tablet Servers being online), we will
still try
: // again unless the tablet/table creation is cancelled.
: RETURN_NOT_OK_PREPEND(SelectReplicasForTablet(policy,
tablet.get()),
: Substitute("error selecting
replicas for tablet $0",
:tablet->id()));
: }
: }
:
: // Update the sys catalog with the new set of
tablets/metadata.
: {
: SysCatalogTable::Actions actions;
: actions.tablets_to_add = deferred.tablets_to_add;
: actions.tablets_to_update = deferred.tablets_to_update;
:
RETURN_NOT_OK_PREPEND(sys_catalog_->Write(std::move(actions)),
: "error persisting updated tablet
metadata");
: }
:
: // Ex
> How about encapsulate the logic into SelectNReplicasForTablet()? since it's
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@5933
PS3, Line 5933: // NOTE: if we fail to select replicas on the first pass
(due to
: // insufficient Tablet Servers being online), we will
still try
: // again unless the tablet/table creation is cancelled.
: RETURN_NOT_OK_PREPEND(SelectReplicasForTablet(policy,
tablet.get()),
: Substitute("error selecting
replicas for tablet $0",
:tablet->id()));
: }
: }
:
: // Update the sys catalog with the new set of
tablets/metadata.
: {
: SysCatalogTable::Actions actions;
: actions.tablets_to_add = deferred.tablets_to_add;
: actions.tablets_to_update = deferred.tablets_to_update;
:
RETURN_NOT_OK_PREPEND(sys_catalog_->Write(std::move(actions)),
: "error persisting updated tablet
metadata");
: }
:
: // Ex
> +1
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@6006
PS3, Line 6006: "alive tablet
servers",
> I didn't find any where else using SelectReplicasForTablet(), why not remov
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@6006
PS3, Line 6006: "alive tablet
servers",
> +1
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/tools/kudu-tool-test.cc@9204
PS3, Line 9204: NO_FATALS(cluster_->tablet_server(0)->Shutdown());
> nit: use cluster_->WaitForTabletServerCount() to ensure this.
WaitForTabletServerCount is a function of InternalMiniCluster, not a function
of ExternalMiniCluster. And I have not found a way to start an
InternalMiniCluster with specified flags, like
--catalog_manager_check_ts_count_for_create_table.
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/tools/kudu-tool-test.cc@9228
PS3, Line 9228: ASSERT_EQ("UNDER_REPLICATED", items[i]["health"]);
> Would be great to test adding range partitions
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19584 )
Change subject: [KUDU-3452] A tool to show on creating tables
..
Patch Set 2:
(12 comments)
http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@7
PS2, Line 7: tables
> Since there two cases, creating tables and adding range partions, so 'table
In fact, creating a table is to creating tablets. Table is a logic conception.
But this tool shows the information of on creating tables.
Please see the commit, there is an output example.
http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@7
PS2, Line 7: A tool to show on creating tables
> You mean this tool is to show list of both tables as well as tablets?
In fact, creating a table is to creating tablets. Table is a logic conception.
But this tool shows the information of on-creating tables.
This tool only shows on-creating table not finished table.
http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@11
PS2, Line 11: The creating task will retry continuously for a long
: time.
> The client side will timeout, but the server side will never timeout.
Done
http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@15
PS2, Line 15: cancel the creating tasks
> I understand the need for cancelation. However, I am not clear on how cance
Currently, I have no clear idea. The creating tables in the tablet server is a
series of tasks. Maybe we can interrupt the tasks. But I do not know the risk.
That needs some tests.
http://gerrit.cloudera.org:8080/#/c/19584/2//COMMIT_MSG@18
PS2, Line 18: kudu table list
> nit: 'kudu table list'
Done
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/client/client-internal.h
File src/kudu/client/client-internal.h:
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/client/client-internal.h@171
PS2, Line 171: list_creating_tables
> nit: list_tables_being_created
Done
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/client/client-internal.cc
File src/kudu/client/client-internal.cc:
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/client/client-internal.cc@483
PS2, Line 483: list_creating_tables
> nit: list_creating_tablets?
This tool lists on-creating tables not tablets.
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/client/client-internal.cc@519
PS2, Line 519: if (list_creating_tables) {
> CLI tool is allowed to access servers in old version, it's needed to deal t
OK. Here, I will check whether the filed 'state' existed to be compatible with
the old version of kudu server.
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/client/client-internal.cc@520
PS2, Line 520: info.id = table.id();
> IMO table id is an internal concept, but not user faced, why expose it to e
In the future, maybe it needs to cancel the creating task according to the
table id. Therefore, here expose the table id to the user.
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/master/catalog_manager.cc@3949
PS2, Line 3949: return Status::OK();
> Why return here, the infos bellow e.g. soft deleted tables, can be skipped
Yes. We only need to collect on-creating tables, not need to collect other
information.
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/master/master.proto@634
PS2, Line 634: optional bool list_creating_tables = 5;
> nit: add '[default = false]' to explictly declare it, deal the case of usin
Done
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/tools/tool_action_table.cc
File src/kudu/tools/tool_action_table.cc:
http://gerrit.cloudera.org:8080/#/c/19584/2/src/kudu/tools/tool_action_table.cc@281
PS2, Line 281: continue;
> It seems --list_creating_tables is not work with some other flags, use GROU
Done
--
To view, visit http://gerrit.cloudera.org:8080/19584
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5
Gerrit-Change-Number: 19584
Gerrit-PatchSet: 2
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Mon, 27 Mar 2023 11:52:28 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19584 to look at the new patch set (#3). Change subject: [KUDU-3452] A tool to show on creating tables .. [KUDU-3452] A tool to show on creating tables When creating a new table or partition, new tablets will be created. If the healthy tablet servers are not enough to create multiple replicas. The creating task will retry continuously for a long time. Therefore it needs a tool to show the creating tables. And users can decide to cancel the creating tasks or not. This patch adds a new flag: --list_creating_tables for command: 'kudu table list' to show the creating tables. It will show the table id, table name, the number of total tablets, the number of creating tables and state. The following is an output example: NoEnoughTServersTable id:3c56ed6c159f4b35885912c34259de46 num_tablets:1 num_creating_tablets:1 state:RUNNING AddNewPartitionTable id:6c2209cc62af4398a60ebdc8f26050f3 num_tablets:2 num_creating_tablets:1 state:ALTERING Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_table.cc 7 files changed, 234 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/19584/3 -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19584 to look at the new patch set (#4). Change subject: [KUDU-3452] A tool to show on creating tables .. [KUDU-3452] A tool to show on creating tables When creating a new table or partition, new tablets will be created. If the healthy tablet servers are not enough to create multiple replicas. The creating task will retry continuously for a long time. Therefore it needs a tool to show the creating tables. And users can decide to cancel the creating tasks or not. This patch adds a new flag: --list_creating_tables for command: 'kudu table list' to show the creating tables. It will show the table id, table name, the number of total tablets, the number of creating tables and state. The following is an output example: NoEnoughTServersTable id:3c56ed6c159f4b35885912c34259de46 num_tablets:1 num_creating_tablets:1 state:RUNNING AddNewPartitionTable id:6c2209cc62af4398a60ebdc8f26050f3 num_tablets:2 num_creating_tablets:1 state:ALTERING Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_table.cc 7 files changed, 231 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/19584/4 -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19584 ) Change subject: [KUDU-3452] A tool to show on creating tables .. Patch Set 3: > Patch Set 3: > > (1 comment) done -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 27 Mar 2023 14:29:21 + Gerrit-HasComments: No
[kudu-CR] KUDU-3452 Create tablet without enough healthy tservers
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19571 )
Change subject: KUDU-3452 Create tablet without enough healthy tservers
..
Patch Set 6:
(13 comments)
http://gerrit.cloudera.org:8080/#/c/19571/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19571/4//COMMIT_MSG@11
PS4, Line 11: unavailabl
> nit: unavailable
Done
http://gerrit.cloudera.org:8080/#/c/19571/4//COMMIT_MSG@11
PS4, Line 11: tablet servers
> tservers become
Done
http://gerrit.cloudera.org:8080/#/c/19571/4//COMMIT_MSG@15
PS4, Line 15: le. So
> So creating
Done
http://gerrit.cloudera.org:8080/#/c/19571/4//COMMIT_MSG@16
PS4, Line 16: replicas ta
> when there are only 2 healthy tservers
Done
http://gerrit.cloudera.org:8080/#/c/19571/4//COMMIT_MSG@16
PS4, Line 16: 2 tabl
> nit: supported
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@5933
PS3, Line 5933: // NOTE: if we fail to select replicas on the first pass
(due to
: // insufficient Tablet Servers being online), we will
still try
: // again unless the tablet/table creation is cancelled.
: RETURN_NOT_OK_PREPEND(SelectReplicasForTablet(policy,
tablet.get()),
: Substitute("error selecting
replicas for tablet $0",
:tablet->id()));
: }
: }
:
: // Update the sys catalog with the new set of
tablets/metadata.
: {
: SysCatalogTable::Actions actions;
: actions.tablets_to_add = deferred.tablets_to_add;
: actions.tablets_to_update = deferred.tablets_to_update;
:
RETURN_NOT_OK_PREPEND(sys_catalog_->Write(std::move(actions)),
: "error persisting updated tablet
metadata");
: }
:
: // Ex
> Done
Done
http://gerrit.cloudera.org:8080/#/c/19571/3/src/kudu/master/catalog_manager.cc@6006
PS3, Line 6006: "alive tablet
servers",
> Done
Done
http://gerrit.cloudera.org:8080/#/c/19571/4/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19571/4/src/kudu/master/catalog_manager.cc@5988
PS4, Line 5988:T
> nit: unexpected extra space ?
Done
http://gerrit.cloudera.org:8080/#/c/19571/4/src/kudu/master/catalog_manager.cc@6001
PS4, Line 6001: replication_fa
> nit: replication_factor
Done
http://gerrit.cloudera.org:8080/#/c/19571/4/src/kudu/master/catalog_manager.cc@6002
PS4, Line 6002: of
> Otherwise,
Done
http://gerrit.cloudera.org:8080/#/c/19571/4/src/kudu/master/catalog_manager.cc@6004
PS4, Line 6004: protocol
> nit: protocol
Done
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19571/2/src/kudu/tools/kudu-tool-test.cc@9213
PS2, Line 9213: name(kTableName2
> So?SleepFor (2 * FLAGS_tserver_unresponsive_timeout_ms ) ? And above( L920
tserver_unresponsive_timeout_ms is a server flag. CLI can not get the value
directly.
http://gerrit.cloudera.org:8080/#/c/19571/4/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19571/4/src/kudu/tools/kudu-tool-test.cc@9239
PS4, Line 9239:
> It's no need restart all kudu cluster. The case should make sure 'cluster_
Done
--
To view, visit http://gerrit.cloudera.org:8080/19571
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6
Gerrit-Change-Number: 19571
Gerrit-PatchSet: 6
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Tue, 28 Mar 2023 03:04:20 +
Gerrit-HasComments: Yes
[kudu-CR] KUDU-3437 Set default block cache metrics policy to reset
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19585 ) Change subject: KUDU-3437 Set default block_cache_metrics_policy to reset .. Patch Set 4: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc32d7ab02201382debcbe36311579550353bf71 Gerrit-Change-Number: 19585 Gerrit-PatchSet: 4 Gerrit-Owner: Ádám Bakai Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yuqi Du Gerrit-Reviewer: Zoltan Chovan Gerrit-Reviewer: Ádám Bakai Gerrit-Comment-Date: Thu, 30 Mar 2023 03:46:57 + Gerrit-HasComments: No
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#3). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table will be timeout when one of three tablet servers. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 67 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/3 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19594 ) Change subject: [KUDU-3452] Make validate table creating task not affected .. Patch Set 3: (7 comments) http://gerrit.cloudera.org:8080/#/c/19594/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19594/2//COMMIT_MSG@7 PS2, Line 7: Make validate table creating task not affected > I think you can make a summary about jira issue: KUDU-3452 and then introdu Done http://gerrit.cloudera.org:8080/#/c/19594/2//COMMIT_MSG@9 PS2, Line 9: Create a three-replicas table will be timeout when one of three : tablet server > Some words need be added and adjusted? Done http://gerrit.cloudera.org:8080/#/c/19594/2//COMMIT_MSG@10 PS2, Line 10: tablet servers. After that, create a two-replicas table also will : timeout > nit: will be timeout Done http://gerrit.cloudera.org:8080/#/c/19594/2/src/kudu/tools/kudu-admin-test.cc File src/kudu/tools/kudu-admin-test.cc: http://gerrit.cloudera.org:8080/#/c/19594/2/src/kudu/tools/kudu-admin-test.cc@1913 PS2, Line 1913: const > nit: indent Done http://gerrit.cloudera.org:8080/#/c/19594/2/src/kudu/tools/kudu-admin-test.cc@1916 PS2, Line 1916: 1000 > nit: define a variable, eg: kTimeout. because below can use it. Done http://gerrit.cloudera.org:8080/#/c/19594/2/src/kudu/tools/kudu-admin-test.cc@1934 PS2, Line 1934: 2000 > nit: 2 * kTimeout Done http://gerrit.cloudera.org:8080/#/c/19594/2/src/kudu/tools/kudu-admin-test.cc@1944 PS2, Line 1944: ASSERT_STR_CONTAINS(s.ToString(), "Timed out waiting for Table Creation"); > It seems better that add a checker for the table's status by using 'IsCreat Done -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 3 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Fri, 31 Mar 2023 08:42:14 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#4). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table will be timeout when one of three tablet servers. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 88 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/4 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19584 to look at the new patch set (#5). Change subject: [KUDU-3452] A tool to show on creating tables .. [KUDU-3452] A tool to show on creating tables When creating a new table or partition, new tablets will be created. If the healthy tablet servers are not enough to create multiple replicas. The creating task will retry continuously for a long time. Therefore it needs a tool to show the creating tables. And users can decide to cancel the creating tasks or not. This patch adds a new flag: --list_creating_tables for command: 'kudu table list' to show the creating tables. It will show the table id, table name, the number of total tablets, the number of creating tables and state. The following is an output example: NoEnoughTServersTable id:3c56ed6c159f4b35885912c34259de46 num_tablets:1 num_creating_tablets:1 state:RUNNING AddNewPartitionTable id:6c2209cc62af4398a60ebdc8f26050f3 num_tablets:2 num_creating_tablets:1 state:ALTERING Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_table.cc 8 files changed, 239 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/19584/5 -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19584 )
Change subject: [KUDU-3452] A tool to show on creating tables
..
Patch Set 5:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/master/catalog_manager.cc@3929
PS4, Line 3929: !tablet_lock.data().is_creating()) {
: continue;
> tablets in state PREPARING and CREATING are needed, right? how about adding
Good ideal.
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/master/catalog_manager.cc@3947
PS4, Line 3947: }
> How about just response pb.state(), transfer to string in CLI tool to reduc
Done
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/kudu-admin-test.cc
File src/kudu/tools/kudu-admin-test.cc:
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/kudu-admin-test.cc@1933
PS4, Line 1933:
> nit: just pass '{}'
Done
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/kudu-admin-test.cc@1983
PS4, Line 1983: ASSERT_TRUE(s.IsTimedOut()) << s.ToString();
> nit: add one more check, ASSERT_TRUE(s.IsTimedOut()) << s.ToString();
Done
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/kudu-admin-test.cc@1999
PS4, Line 1999: ASSERT_STR_CONTAINS(s.ToString(), "Timed out: Timed out
waiting for AlterTable");
> nit: add one more check, ASSERT_TRUE(s.IsTimedOut()) << s.ToString();
Done
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/kudu-admin-test.cc@2029
PS4, Line 2029: else if (GetParam() ==
> nit: how about use ASSERT_STR_MATCHES?
It is not suitable. Here needs to check two different kinds of table state.
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/tool.proto
File src/kudu/tools/tool.proto:
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/tool.proto@546
PS4, Line 546: 6
> Why skip 6?
Done
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/tool_action_table.cc
File src/kudu/tools/tool_action_table.cc:
http://gerrit.cloudera.org:8080/#/c/19584/4/src/kudu/tools/tool_action_table.cc@239
PS4, Line 239: bool ValidateListTable() {
> nit: What's the purpose of this limit? To avoid too large output?
When the flag --list_creating_tables is set, other flags is useless. So it no
needs to set other flags.
--
To view, visit http://gerrit.cloudera.org:8080/19584
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5
Gerrit-Change-Number: 19584
Gerrit-PatchSet: 5
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Ashwani Raina
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Sun, 02 Apr 2023 08:00:07 +
Gerrit-HasComments: Yes
[kudu-CR] [Tool] Show the information of a tablet
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#12). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. The following is an exmple: Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table Tablet Server UUID| Host | Port | Is Leader --+-+---+--- 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 273 insertions(+), 79 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/12 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 12 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19501 ) Change subject: [Tool] Show the information of a tablet .. Patch Set 12: (2 comments) http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/catalog_manager.h File src/kudu/master/catalog_manager.h: http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/catalog_manager.h@802 PS11, Line 802: // Replace the given tablet with a new, empty one. The replaced tablet is > Remove one blank line. Done http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/master.proto File src/kudu/master/master.proto: http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/master.proto@459 PS11, Line 459: optional ListTablesResponsePB.TableInfo table_info = 8; > typo: If reusing ListTablesResponsePB, the new CLI tool is compatiable with Done -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 12 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Sun, 02 Apr 2023 08:35:34 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#5). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table will be timeout when one of three tablet servers. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 101 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/5 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19594 )
Change subject: [KUDU-3452] Make validate table creating task not affected
..
Patch Set 5:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/19594/4/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19594/4/src/kudu/master/catalog_manager.cc@5934
PS4, Line 5934: // to PREPARING, or its state will become CREATING
later.
> Why not just remove the tablet from needs_create_rpc to reduce an extra var
Done
http://gerrit.cloudera.org:8080/#/c/19594/4/src/kudu/master/catalog_manager.cc@5938
PS4, Line 5938: << ",
> What the meaning of 'old logic'? The 'old logic' is only reasonable in this
OK. I will remove it.
http://gerrit.cloudera.org:8080/#/c/19594/4/src/kudu/master/catalog_manager.cc@5940
PS4, Line 5940: {
> I not sure if I can get the point, could you please explain about it?
In the origin logic, if the state of a tablet is CREATING(persisted by
lock_out.Commit()), it means it has finished selecting replicas and prepares to
send creating replica PRCs to the tablet servers. If the tablet does not finish
selecting replicas, it should recover its state to PREPARING. Or it will wait
timeout(defined by --tablet_creation_timeout_ms) and create a new tablet to
replace the old one. Please see the function: HandleAssignCreatingTablet().
http://gerrit.cloudera.org:8080/#/c/19594/4/src/kudu/master/catalog_manager.cc@5941
PS4, Line 5941: it++;
> Print the returned Status to clearify why select replicas failed.
Done
--
To view, visit http://gerrit.cloudera.org:8080/19594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9
Gerrit-Change-Number: 19594
Gerrit-PatchSet: 5
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Sun, 02 Apr 2023 14:10:41 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#6). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table will be timeout when one of three tablet servers. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 100 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/6 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 6 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19594 ) Change subject: [KUDU-3452] Make validate table creating task not affected .. Patch Set 5: > Patch Set 5: > > (1 comment) done -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 5 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Mon, 03 Apr 2023 01:48:00 + Gerrit-HasComments: No
[kudu-CR] KUDU-3452 Create tablet without enough healthy tservers
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19571 )
Change subject: KUDU-3452 Create tablet without enough healthy tservers
..
Patch Set 6:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19571/6/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19571/6/src/kudu/master/catalog_manager.cc@6003
PS6, Line 6003: if (policy.ts_num() < (nreplicas + 1) / 2) {
> nit: According to discussions in the JIRA ticket, we should also check for
It has been implemented in Line 6012.
http://gerrit.cloudera.org:8080/#/c/19571/6/src/kudu/tools/kudu-tool-test.cc
File src/kudu/tools/kudu-tool-test.cc:
http://gerrit.cloudera.org:8080/#/c/19571/6/src/kudu/tools/kudu-tool-test.cc@9175
PS6, Line 9175: Restart the stopped tserver to bring the table to healthy.
> nit: How about 'The table will become healthy once the unavailable tserver
Done
http://gerrit.cloudera.org:8080/#/c/19571/6/src/kudu/tools/kudu-tool-test.cc@9290
PS6, Line 9290: colum
> nit: column
Done
--
To view, visit http://gerrit.cloudera.org:8080/19571
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6
Gerrit-Change-Number: 19571
Gerrit-PatchSet: 6
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Thu, 06 Apr 2023 13:51:54 +
Gerrit-HasComments: Yes
[kudu-CR] [Tool] Show the information of a tablet
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19501 to look at the new patch set (#13). Change subject: [Tool] Show the information of a tablet .. [Tool] Show the information of a tablet This tool is used to show the table name, table id and all replicas on different tablet servers according to the tablet id. Sometimes, the log prints the tablet id when a query failed or the system got something wrong. But it is difficult to find useful information according to the tablet id. This tool will be helpful for debuging. The following is an exmple: Command: kudu tablet info $master_address $tablet_id The result is: Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table Tablet Server UUID| Host | Port | Is Leader --+-+---+--- 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/client/client.cc M src/kudu/client/client.h M src/kudu/client/tablet-internal.cc M src/kudu/client/tablet-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/tools/kudu-tool-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_tablet.cc 13 files changed, 273 insertions(+), 79 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/01/19501/13 -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 13 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [Tool] Show the information of a tablet
Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/19501 ) Change subject: [Tool] Show the information of a tablet .. Patch Set 12: (5 comments) http://gerrit.cloudera.org:8080/#/c/19501/12//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/19501/12//COMMIT_MSG@9 PS12, Line 9: This tool is used to show the table name, table id and all : replicas on different tablet servers according to the tablet : id. > I review this patch again. This commit is done as you advised. I maybe miss something. Could you give more explain in detail? http://gerrit.cloudera.org:8080/#/c/19501/12//COMMIT_MSG@17 PS12, Line 17: This tool will be helpful for debuging. The following is an : exmple: : : Table id: 9112ec4b6eac48f5967da5a1f20610b0, Table name: test_table : Tablet Server UUID| Host | Port | Is Leader : --+-+---+--- : 4cd055e0cf3b4165823747e3c349dc6f | 127.241.215.131 | 45963 | true : 65a671158cd74ac3995f5c68180621b8 | 127.241.215.130 | 42211 | false : 6b6ef15e7d4a471f92f4c1eca1a46638 | 127.241.215.129 | 36321 | false > How about add examples of command lines?This will give us a more direct und Done http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/catalog_manager.h File src/kudu/master/catalog_manager.h: http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/catalog_manager.h@802 PS11, Line 802: // Replace the given tablet with a new, empty one. The replaced tablet is > Done Done http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/master.proto File src/kudu/master/master.proto: http://gerrit.cloudera.org:8080/#/c/19501/11/src/kudu/master/master.proto@459 PS11, Line 459: optional ListTablesResponsePB.TableInfo table_info = 8; > I meant if reuse ListTablesRequestPB to collect table info (not add an extr Do you mean the client send 2 RPCs to get related information? GetTabletLocationsRequestPB RPC gets tablet location information, and ListTablesRequestPB RPC gets the table information? But we don't know the table name when finished GetTabletLocationsRequestPB RPC. How to send ListTablesRequestPB RPC. http://gerrit.cloudera.org:8080/#/c/19501/12/src/kudu/master/master.proto File src/kudu/master/master.proto: http://gerrit.cloudera.org:8080/#/c/19501/12/src/kudu/master/master.proto@503 PS12, Line 503: table_info_in_response > It seems 'GetTabletLocationsResponsePB' has no such field to save 'table_in table_info is stored in TabletLocationsPB of GetTabletLocationsResponsePB. Please see line 459. -- To view, visit http://gerrit.cloudera.org:8080/19501 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib5ae5f61f50a44c4787843df76adaa61700ae9fe Gerrit-Change-Number: 19501 Gerrit-PatchSet: 12 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du Gerrit-Comment-Date: Thu, 06 Apr 2023 14:32:26 + Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#7). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table will be timeout when one of three tablet servers. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 97 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/7 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 7 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19594 )
Change subject: [KUDU-3452] Make validate table creating task not affected
..
Patch Set 7:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/19594/4/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19594/4/src/kudu/master/catalog_manager.cc@5940
PS4, Line 5940: ERROR) << "Error selecting r
> Thanks for your explain.
TabletInfo is a CowObject. Function commit() will finally persist its state. If
nothing has changed, function commit() maybe do nothing.
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/master/catalog_manager.cc@5926
PS6, Line 5926: auto it = deferred.needs_create_rpc.begin()
> nit: simplify by
Done
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/master/catalog_manager.cc@5934
PS6, Line 5934: et RPC finished. If a tablet selects re
> nit: Also need to elaborate on why its state shouldn't be CREATING.
Done
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc
File src/kudu/tools/kudu-admin-test.cc:
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1917
PS6, Line 1917: "--allow_unsafe_replication_factor=true",
> nit: Could we also test this case by creating a table with only one replica
Cloud you explain why we need test the case with one replica? 2-replica is also
unsafe replication factor.
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1962
PS6, Line 1962:
> Is it necessary to create a table with range partitions? I guess it's not r
You are right. I will remove it.
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1978
PS6, Line 1978:
> It would be great to check 2 of the 3 replicas have been created successful
I guess you maybe mix this commit with commit: 19571: KUDU-3452 Create tablet
without enough healthy tservers.
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1982
PS6, Line 1982: {
> Same.
Done
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1990
PS6, Line 1990:
> Call WaitStarted() to ensure it has been started.
WaitStarted() is not a member function of ExternalTabletServer.
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1992
PS6, Line 1992: ring> master_f
> How about cache the variable at first?
Done
--
To view, visit http://gerrit.cloudera.org:8080/19594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9
Gerrit-Change-Number: 19594
Gerrit-PatchSet: 7
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Sat, 08 Apr 2023 14:10:14 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#8). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table will be timeout when one of three tablet servers. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 113 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/8 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 8 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Wang Xixu has posted comments on this change. (
http://gerrit.cloudera.org:8080/19594 )
Change subject: [KUDU-3452] Make validate table creating task not affected
..
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc
File src/kudu/tools/kudu-admin-test.cc:
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1917
PS6, Line 1917: "--allow_unsafe_replication_factor=true",
> I think there is no need to set "--allow_unsafe_replication_factor=true" if
I guess you maybe worry about creating an odd number of replicas failed. OK, I
will add a test case with one replica.
http://gerrit.cloudera.org:8080/#/c/19594/6/src/kudu/tools/kudu-admin-test.cc@1990
PS6, Line 1990: ASSERT_EQ(string("HEALTHY"), items[i]["health"].
> Oh, sorry. But how about use WaitForTabletServers() and WaitForReplicasAndU
Done
--
To view, visit http://gerrit.cloudera.org:8080/19594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9
Gerrit-Change-Number: 19594
Gerrit-PatchSet: 8
Gerrit-Owner: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: KeDeng
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <1450306...@qq.com>
Gerrit-Reviewer: Yifan Zhang
Gerrit-Reviewer: Yingchun Lai
Gerrit-Reviewer: Yuqi Du
Gerrit-Comment-Date: Sat, 15 Apr 2023 14:46:47 +
Gerrit-HasComments: Yes
[kudu-CR] [KUDU-3452] Make validate table creating task not affected
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19594 to look at the new patch set (#9). Change subject: [KUDU-3452] Make validate table creating task not affected .. [KUDU-3452] Make validate table creating task not affected Create a three-replicas table will be timeout when one of three tablet servers. After that, create a two-replicas table also will timeout even if there are enough tablet servers to place its replicas. The validate two-replicas table will be affected by invalidate three-replicas table. This patch fix this problem. If a task of creating tablet fail, it will not return immediately, but let other tasks of creating other tablets keep on running. Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 --- M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-admin-test.cc 2 files changed, 112 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/19594/9 -- To view, visit http://gerrit.cloudera.org:8080/19594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I64668651d0e8f58b92cfb841bdb20617de6776f9 Gerrit-Change-Number: 19594 Gerrit-PatchSet: 9 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] KUDU-3452 Create tablet without enough healthy tservers
Hello Alexey Serbin, Yuqi Du, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19571 to look at the new patch set (#10). Change subject: KUDU-3452 Create tablet without enough healthy tservers .. KUDU-3452 Create tablet without enough healthy tservers Currently, creating a three-replicas table when there are not enough tablet servers will fail and the system catalog will retry continuously and always fail until the unavailable tablet servers becomes healthy again. An under-replicated table is still available for reading and writing, so it's enough to place just a majority of replicas for each tablet at healthy tablet servers to make a newly created table ready to use. This patch adds a new flag: --allow_creating_under_replicated_tables to support this feature. The original logic is kept the same. When this flag is set true, it's possible to create a tablet placing just a majority of replicas at healthy tablet servers. Even if the new tablet is created under-replicated, it's still available for read and write operations. Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 --- M src/kudu/client/client.h M src/kudu/master/catalog_manager.cc M src/kudu/tools/kudu-tool-test.cc 3 files changed, 205 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/71/19571/10 -- To view, visit http://gerrit.cloudera.org:8080/19571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I742ba1ff770f5c8b1be5800334c29bec96e195c6 Gerrit-Change-Number: 19571 Gerrit-PatchSet: 10 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
[kudu-CR] [KUDU-3452] A tool to show on creating tables
Hello Tidy Bot, Alexey Serbin, Yuqi Du, Ashwani Raina, Yingchun Lai, Yifan Zhang, Kudu Jenkins, KeDeng, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19584 to look at the new patch set (#6). Change subject: [KUDU-3452] A tool to show on creating tables .. [KUDU-3452] A tool to show on creating tables When creating a new table or partition, new tablets will be created. If the healthy tablet servers are not enough to create multiple replicas. The creating task will retry continuously for a long time. Therefore it needs a tool to show the creating tables. And users can decide to cancel the creating tasks or not. This patch adds a new flag: --list_tables_with_creating_tablets for command: 'kudu table list' to show the creating tables. It will show the table id, table name, the number of total tablets, the number of creating tables and state. The following is an output example: NoEnoughTServersTable id:3c56ed6c159f4b35885912c34259de46 num_tablets:1 num_creating_tablets:1 state:RUNNING AddNewPartitionTable id:6c2209cc62af4398a60ebdc8f26050f3 num_tablets:2 num_creating_tablets:1 state:ALTERING Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 --- M src/kudu/client/client-internal.cc M src/kudu/client/client-internal.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master.proto M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool.proto M src/kudu/tools/tool_action_table.cc 8 files changed, 239 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/19584/6 -- To view, visit http://gerrit.cloudera.org:8080/19584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I348b69f48e6ce36ed869097f9f798c5946136de5 Gerrit-Change-Number: 19584 Gerrit-PatchSet: 6 Gerrit-Owner: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Ashwani Raina Gerrit-Reviewer: KeDeng Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <1450306...@qq.com> Gerrit-Reviewer: Yifan Zhang Gerrit-Reviewer: Yingchun Lai Gerrit-Reviewer: Yuqi Du
