[kudu-CR] [fs] KUDU-3161: Include FileSystem Path in UUID Mismatch Error
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17606 ) Change subject: [fs] KUDU-3161: Include FileSystem Path in UUID Mismatch Error .. [fs] KUDU-3161: Include FileSystem Path in UUID Mismatch Error This patch enhances current error logging to include filesystem path if the UUID of the instance file in any of the filesystem roots differs from the UUID of other filesystem roots by including the filesystem path. This helps in troubleshooting scenarios where we need to know the exact directory to look for. Current log: Mismatched UUIDs across filesystem roots: aefaaab3fe774f7f87d5a3da4ae1aaae vs. 2af886a28b784b92b64432c612597e2d; configuring multiple Kudu processes with the same directory is not supported Log after the patch: Mismatched UUIDs across filesystem roots: The path /tserver-0/data contains UUID aefaaab3fe774f7f87d5a3da4ae1aaae vs. the path /tserver-1/data contains UUID 2af886a28b784b92b64432c612597e2d; configuring multiple Kudu processes with the same directory is not supported Change-Id: I02ccd93b8cbb2c2f17573d9d56f955ef4723481f Reviewed-on: http://gerrit.cloudera.org:8080/17606 Reviewed-by: Bankim Bhavsar Reviewed-by: Grant Henke Tested-by: Grant Henke --- M src/kudu/fs/fs_manager.cc 1 file changed, 6 insertions(+), 4 deletions(-) Approvals: Bankim Bhavsar: Looks good to me, approved Grant Henke: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17606 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I02ccd93b8cbb2c2f17573d9d56f955ef4723481f Gerrit-Change-Number: 17606 Gerrit-PatchSet: 2 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [fs] KUDU-3161: Include FileSystem Path in UUID Mismatch Error
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17606 ) Change subject: [fs] KUDU-3161: Include FileSystem Path in UUID Mismatch Error .. Patch Set 1: Verified+1 Code-Review+2 Unrelated TestKuduMetrics failure -- To view, visit http://gerrit.cloudera.org:8080/17606 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I02ccd93b8cbb2c2f17573d9d56f955ef4723481f Gerrit-Change-Number: 17606 Gerrit-PatchSet: 1 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Thu, 24 Jun 2021 15:22:04 + Gerrit-HasComments: No
[kudu-CR] [fs] KUDU-3161: Include FileSystem Path in UUID Mismatch Error
Grant Henke has removed a vote on this change. Change subject: [fs] KUDU-3161: Include FileSystem Path in UUID Mismatch Error .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/17606 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I02ccd93b8cbb2c2f17573d9d56f955ef4723481f Gerrit-Change-Number: 17606 Gerrit-PatchSet: 1 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [metrics] KUDU-3269: Add UUID and hostname of the server into the metrics output
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17607 ) Change subject: [metrics] KUDU-3269: Add UUID and hostname of the server into the metrics output .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/17607/4/src/kudu/server/server_base.cc File src/kudu/server/server_base.cc: http://gerrit.cloudera.org:8080/#/c/17607/4/src/kudu/server/server_base.cc@94 PS4, Line 94: #include "kudu/util/net/net_util.h" This is already included at line 82 above -- To view, visit http://gerrit.cloudera.org:8080/17607 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7d6d1e96f8067a7b1593da4f9d0e1931d3001016 Gerrit-Change-Number: 17607 Gerrit-PatchSet: 4 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Thu, 24 Jun 2021 14:01:51 + Gerrit-HasComments: Yes
[kudu-CR] [metrics] KUDU-3269: Add UUID and hostname of the server into the metrics output
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17607 ) Change subject: [metrics] KUDU-3269: Add UUID and hostname of the server into the metrics output .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/17607/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17607/2//COMMIT_MSG@16 PS2, Line 16: "hostname": "MacBook-Pro.local", > I filed KUDU-3269 with the rationale to add identification for collected ta makes sense, lets keep it simple -- To view, visit http://gerrit.cloudera.org:8080/17607 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7d6d1e96f8067a7b1593da4f9d0e1931d3001016 Gerrit-Change-Number: 17607 Gerrit-PatchSet: 2 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Wed, 23 Jun 2021 21:53:33 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.15.x) [version] Bump up the version to 1.15.1-SNAPSHOT for next release
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17623 ) Change subject: [version] Bump up the version to 1.15.1-SNAPSHOT for next release .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17623 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: Ied5778beb7ad3fce1f0c58c32bd52d34d7e2c085 Gerrit-Change-Number: 17623 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 23 Jun 2021 02:16:38 + Gerrit-HasComments: No
[kudu-CR] [thirdparty] fix building LLVM9 with GCC 10
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17627 ) Change subject: [thirdparty] fix building LLVM9 with GCC 10 .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/17627/1/thirdparty/patches/llvm-demangle.patch File thirdparty/patches/llvm-demangle.patch: http://gerrit.cloudera.org:8080/#/c/17627/1/thirdparty/patches/llvm-demangle.patch@4 PS1, Line 4: #include "llvm/Demangle/DemangleConfig.h" Has this patch landed in an LLVM repo anywhere yet? Can you reference the pull request or commit? -- To view, visit http://gerrit.cloudera.org:8080/17627 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3332556229dbf2bce65d52f37c48d62624465f70 Gerrit-Change-Number: 17627 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 23 Jun 2021 02:16:10 + Gerrit-HasComments: Yes
[kudu-CR] [thirdparty] fix building crcutil with GCC 10
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17626 ) Change subject: [thirdparty] fix building crcutil with GCC 10 .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/17626/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17626/1//COMMIT_MSG@9 PS1, Line 9: This patch fixes building crcutil with GCC 10. Prior to this patch, Is it possible to land this change in the http://github.com/cloudera/crcutil repo and then update the hash in vars.sh instead of patching it in the Kudu build? -- To view, visit http://gerrit.cloudera.org:8080/17626 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibaba794ce00eb64a29e8038a8a2f4883f35c975a Gerrit-Change-Number: 17626 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 23 Jun 2021 02:14:05 + Gerrit-HasComments: Yes
[kudu-CR] [thirdparty] upgrade libunwind to 1.5.0
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17625 ) Change subject: [thirdparty] upgrade libunwind to 1.5.0 .. Patch Set 1: Code-Review+2 Looks like there may be a TSAN issue. -- To view, visit http://gerrit.cloudera.org:8080/17625 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5964776133134d8ab17d69fc7ee0bc7457f98626 Gerrit-Change-Number: 17625 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 23 Jun 2021 02:11:58 + Gerrit-HasComments: No
[kudu-CR] [docs] Updates to the RELEASING doc
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17624 ) Change subject: [docs] Updates to the RELEASING doc .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17624 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iee1b483cce963d80172d992e739a80fda463f59a Gerrit-Change-Number: 17624 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 23 Jun 2021 02:11:12 + Gerrit-HasComments: No
[kudu-CR] [docs] Add 1.15.0 release notes to previous release notes
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17621 ) Change subject: [docs] Add 1.15.0 release notes to previous release notes .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17621 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15b021650714de55129374005f070e744e5745ff Gerrit-Change-Number: 17621 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 23 Jun 2021 02:10:30 + Gerrit-HasComments: No
[kudu-CR] [examples] Bump up Kudu version to 1.15.0 for examples
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17622 ) Change subject: [examples] Bump up Kudu version to 1.15.0 for examples .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17622 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6bc35c2ba261c89b0f31dad9d216b2dd1fb058aa Gerrit-Change-Number: 17622 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 23 Jun 2021 02:10:00 + Gerrit-HasComments: No
[kudu-CR] KUDU-3297 fix RPC negotiations with cyrus-sasl-gssapi-2.1.27-5 and newer
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17619 ) Change subject: KUDU-3297 fix RPC negotiations with cyrus-sasl-gssapi-2.1.27-5 and newer .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17619 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia655356798c753d5a223933cc09a0731018e10af Gerrit-Change-Number: 17619 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Kurt Deschler Gerrit-Comment-Date: Tue, 22 Jun 2021 19:48:31 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [site] Update website for 1.15.0 release
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17616 ) Change subject: [site] Update website for 1.15.0 release .. Patch Set 1: Verified+1 Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17616 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: I25a3998a7d03689fe7fc0c536f7531d7b0331378 Gerrit-Change-Number: 17616 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Comment-Date: Tue, 22 Jun 2021 16:12:57 + Gerrit-HasComments: No
[kudu-CR](gh-pages) [blog] Add a blog post for 1.15.0 release
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17617 ) Change subject: [blog] Add a blog post for 1.15.0 release .. Patch Set 1: Verified+1 Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/17617 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: gh-pages Gerrit-MessageType: comment Gerrit-Change-Id: Ia3ffdcc10ad03ef010774ea38ecf0338f1a6b9a7 Gerrit-Change-Number: 17617 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Comment-Date: Tue, 22 Jun 2021 16:12:10 + Gerrit-HasComments: No
[kudu-CR] [metrics] KUDU-3269: Add UUID of the server into the metrics output
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/17607 )
Change subject: [metrics] KUDU-3269: Add UUID of the server into the metrics
output
..
Patch Set 2:
(7 comments)
http://gerrit.cloudera.org:8080/#/c/17607/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17607/2//COMMIT_MSG@16
PS2, Line 16: "hostname": "MacBook-Pro.local",
Can you update this message to indicate that you also added the hostname?
Is it worth adding the IP too?
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/master/master-test.cc
File src/kudu/master/master-test.cc:
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/master/master-test.cc@1387
PS2, Line 1387: ){
Add a space before the brace
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/master/master-test.cc@1396
PS2, Line 1396: ASSERT_TRUE(GetHostname(_hostname).ok());
ASSERT_OK(GetHostname(_hostname));
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/server/server_base.cc
File src/kudu/server/server_base.cc:
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/server/server_base.cc@854
PS2, Line 854: e{
Add a space before brace
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/server/server_base.cc@855
PS2, Line 855: metric_entity_->SetAttribute("hostname", "Could not be
determined" );
I think I would prefer an empty string given the presence of a value could be
interpreted to be the hostname.
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/tserver/tablet_server-test.cc
File src/kudu/tserver/tablet_server-test.cc:
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/tserver/tablet_server-test.cc@3819
PS2, Line 3819: ){
Add a space before braces
http://gerrit.cloudera.org:8080/#/c/17607/2/src/kudu/tserver/tablet_server-test.cc@3828
PS2, Line 3828: ASSERT_TRUE(GetHostname(_hostname).ok());
ASSERT_OK(GetHostname(_hostname));
--
To view, visit http://gerrit.cloudera.org:8080/17607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d6d1e96f8067a7b1593da4f9d0e1931d3001016
Gerrit-Change-Number: 17607
Gerrit-PatchSet: 2
Gerrit-Owner: Abhishek Chennaka
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 22 Jun 2021 13:49:57 +
Gerrit-HasComments: Yes
[kudu-CR] [clock] change clock source selection for 'auto'
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17582 ) Change subject: [clock] change clock source selection for 'auto' .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/17582/3/src/kudu/clock/hybrid_clock.cc File src/kudu/clock/hybrid_clock.cc: http://gerrit.cloudera.org:8080/#/c/17582/3/src/kudu/clock/hybrid_clock.cc@476 PS3, Line 476: // Switch to the built-in NTP client unless the set of reference servers In the case the built-in NTP client can't reach the configured ntp servers, should we continue falling back to the system ntp? Is that something we can easily detect? -- To view, visit http://gerrit.cloudera.org:8080/17582 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I72cb54c488b2aa2c73acd6e6e5f6e50dd5811175 Gerrit-Change-Number: 17582 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 21 Jun 2021 19:15:04 + Gerrit-HasComments: Yes
[kudu-CR] [tools] tooling to set/clear table comments
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17600 ) Change subject: [tools] tooling to set/clear table comments .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/17600/1/src/kudu/tools/kudu-tool-test.cc File src/kudu/tools/kudu-tool-test.cc: http://gerrit.cloudera.org:8080/#/c/17600/1/src/kudu/tools/kudu-tool-test.cc@1180 PS1, Line 1180: "clear_comment.*Clear the comment for a table", Would it make sense to add column versions as well? -- To view, visit http://gerrit.cloudera.org:8080/17600 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0b8ea0ce05c642651eeadc1cb0cc654c5302921c Gerrit-Change-Number: 17600 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 17 Jun 2021 22:07:25 + Gerrit-HasComments: Yes
[kudu-CR] [docker] Add the license and notice file to the images
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17594 ) Change subject: [docker] Add the license and notice file to the images .. [docker] Add the license and notice file to the images This patch adds the LICENSE.txt and the NOTICE.txt files to all of the Docker images. Change-Id: Icafa66c176ff96b56d66fcaf0c557aa13cd1d177 Reviewed-on: http://gerrit.cloudera.org:8080/17594 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin Reviewed-by: Bankim Bhavsar --- M docker/Dockerfile 1 file changed, 10 insertions(+), 0 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved Bankim Bhavsar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icafa66c176ff96b56d66fcaf0c557aa13cd1d177 Gerrit-Change-Number: 17594 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Add the license and notice file to the images
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17594 Change subject: [docker] Add the license and notice file to the images .. [docker] Add the license and notice file to the images This patch adds the LICENSE.txt and the NOTICE.txt files to all of the Docker images. Change-Id: Icafa66c176ff96b56d66fcaf0c557aa13cd1d177 --- M docker/Dockerfile 1 file changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/17594/1 -- To view, visit http://gerrit.cloudera.org:8080/17594 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icafa66c176ff96b56d66fcaf0c557aa13cd1d177 Gerrit-Change-Number: 17594 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [docker] Add option to force the latest tag
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17574 ) Change subject: [docker] Add option to force the latest tag .. [docker] Add option to force the latest tag This patch adds a `--force-latest` that allows users to force using the simple `latest` tag even when the default base OS is not used. This is useful because the latest tag is often used to retrieve the last relevant build and is commonly used when a user just wants to test or use their latest build. For example if a user wants to test the quickstart with CentOS 7 they can build with `--force-latest` and run normally as opposed to manually editing the quickstart.yml file. This is also useful for CI/CD pipelines that always want to publish the last build image as `latest`. Change-Id: I13ff38683d3f923577f8728fdf30a4b3e09bc943 Reviewed-on: http://gerrit.cloudera.org:8080/17574 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M docker/docker-build.py 1 file changed, 5 insertions(+), 2 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17574 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I13ff38683d3f923577f8728fdf30a4b3e09bc943 Gerrit-Change-Number: 17574 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Add option to force the latest tag
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17574 ) Change subject: [docker] Add option to force the latest tag .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/17574/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17574/1//COMMIT_MSG@17 PS1, Line 17: editing > typo Done http://gerrit.cloudera.org:8080/#/c/17574/1/docker/docker-build.py File docker/docker-build.py: http://gerrit.cloudera.org:8080/#/c/17574/1/docker/docker-build.py@118 PS1, Line 118: > nit: off-by-one indent Done http://gerrit.cloudera.org:8080/#/c/17574/1/docker/docker-build.py@118 PS1, Line 118: as > Nit: space after "base" Done http://gerrit.cloudera.org:8080/#/c/17574/1/docker/docker-build.py@121 PS1, Line 121: If passed, skips adding a tag using `-latest` along with the ' : 'versioned tag > nit: is it worth mentioning that this overrides the --force-latest tag abov Done -- To view, visit http://gerrit.cloudera.org:8080/17574 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I13ff38683d3f923577f8728fdf30a4b3e09bc943 Gerrit-Change-Number: 17574 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 14 Jun 2021 15:56:14 + Gerrit-HasComments: Yes
[kudu-CR] [docker] Add option to force the latest tag
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17574 to look at the new patch set (#2). Change subject: [docker] Add option to force the latest tag .. [docker] Add option to force the latest tag This patch adds a `--force-latest` that allows users to force using the simple `latest` tag even when the default base OS is not used. This is useful because the latest tag is often used to retrieve the last relevant build and is commonly used when a user just wants to test or use their latest build. For example if a user wants to test the quickstart with CentOS 7 they can build with `--force-latest` and run normally as opposed to manually editing the quickstart.yml file. This is also useful for CI/CD pipelines that always want to publish the last build image as `latest`. Change-Id: I13ff38683d3f923577f8728fdf30a4b3e09bc943 --- M docker/docker-build.py 1 file changed, 5 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/74/17574/2 -- To view, visit http://gerrit.cloudera.org:8080/17574 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I13ff38683d3f923577f8728fdf30a4b3e09bc943 Gerrit-Change-Number: 17574 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] WIP [clock] change clock source selection for 'auto'
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17582 ) Change subject: WIP [clock] change clock source selection for 'auto' .. Patch Set 2: (1 comment) The overall change/approach makes sense to me. http://gerrit.cloudera.org:8080/#/c/17582/2/src/kudu/clock/hybrid_clock.h File src/kudu/clock/hybrid_clock.h: http://gerrit.cloudera.org:8080/#/c/17582/2/src/kudu/clock/hybrid_clock.h@206 PS2, Line 206: // The 'builtin_ntp_servers' is used in case of TimeSource::BUILTIN_NTP_SYNC. this line is no longer needed. -- To view, visit http://gerrit.cloudera.org:8080/17582 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I72cb54c488b2aa2c73acd6e6e5f6e50dd5811175 Gerrit-Change-Number: 17582 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Fri, 11 Jun 2021 12:56:27 + Gerrit-HasComments: Yes
[kudu-CR] [consensus] KUDU-2302: don't crash if new leader can't resolve peer
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17585 ) Change subject: [consensus] KUDU-2302: don't crash if new leader can't resolve peer .. Patch Set 2: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/17585/2/src/kudu/consensus/peer_manager.h File src/kudu/consensus/peer_manager.h: http://gerrit.cloudera.org:8080/#/c/17585/2/src/kudu/consensus/peer_manager.h@60 PS2, Line 60: // TODO(awong): those peers should be retried later. What happens currently when the peers are not retried? -- To view, visit http://gerrit.cloudera.org:8080/17585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22d870ecc526fa47b97f6856c3b023bc1ec029c7 Gerrit-Change-Number: 17585 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 11 Jun 2021 12:55:42 + Gerrit-HasComments: Yes
[kudu-CR] [consensus] KUDU-2302: don't crash if new leader can't resolve peer
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17585 ) Change subject: [consensus] KUDU-2302: don't crash if new leader can't resolve peer .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22d870ecc526fa47b97f6856c3b023bc1ec029c7 Gerrit-Change-Number: 17585 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 11 Jun 2021 12:51:27 + Gerrit-HasComments: No
[kudu-CR] [consensus] KUDU-2302: don't crash if new leader can't resolve peer
Grant Henke has removed a vote on this change. Change subject: [consensus] KUDU-2302: don't crash if new leader can't resolve peer .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/17585 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I22d870ecc526fa47b97f6856c3b023bc1ec029c7 Gerrit-Change-Number: 17585 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [util] flag to specify hosts to fail DNS resolution
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17584 ) Change subject: [util] flag to specify hosts to fail DNS resolution .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17584 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I25f2d3073fa0fa0a430afacf7c8cd1402e6bd6ab Gerrit-Change-Number: 17584 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Fri, 11 Jun 2021 12:51:12 + Gerrit-HasComments: No
[kudu-CR] [docker] Add option to force the latest tag
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17574 Change subject: [docker] Add option to force the latest tag .. [docker] Add option to force the latest tag This patch adds a `--force-latest` that allows users to force using the simple `latest` tag even when the default base OS is not used. This is useful because the latest tag is often used to retrieve the last relevant build and is commonly used when a user just wants to test or use their latest build. For example if a user wants to test the quickstart with CentOS 7 they can build with `--force-latest` and run normally as opposed to manually edditing the quickstart.yml file. This is also useful for CI/CD pipelines that always want to publish the last build image as `latest`. Change-Id: I13ff38683d3f923577f8728fdf30a4b3e09bc943 --- M docker/docker-build.py 1 file changed, 4 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/74/17574/1 -- To view, visit http://gerrit.cloudera.org:8080/17574 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I13ff38683d3f923577f8728fdf30a4b3e09bc943 Gerrit-Change-Number: 17574 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR](branch-1.15.x) [docker] Add support for publishing UBI based runtime images
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17573 Change subject: [docker] Add support for publishing UBI based runtime images .. [docker] Add support for publishing UBI based runtime images This patch adds a `--use-ubi-runtime` option to the docker build. When that option is passed for Centos 7 and 8 builds, the Kudu runtime images (`kudu` & `kudu-python`) will be built using a UBI base instead of the CentOS base. If the `--use-ubi-runtime` flag is passed for any unsupported OS, a clear error message will be printed. We do not build the development images using the UBI base because the package repositories in the UBI do not contain all the packages necessary to build Kudu and support development. Additionally the benefits of UBI are primarily for runtime/deployment usage and continuing to use CentOS for the development can simplify the build. This patch also required some small changes to how we generated the published image labels given the UBI images use a slightly different format than was previously handled. More can be read about RedHat UBI here: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image https://developers.redhat.com/articles/ubi-faq I tested building images with `--use-ubi-runtime` on both CentOS 7 and 8. Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Reviewed-on: http://gerrit.cloudera.org:8080/17564 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke (cherry picked from commit 76cb0dd808aaef548ef80682e13a00711e7dd6a4) --- M docker/Dockerfile M docker/bootstrap-runtime-env.sh M docker/docker-build.py 3 files changed, 56 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/73/17573/1 -- To view, visit http://gerrit.cloudera.org:8080/17573 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17573 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17564 ) Change subject: [docker] Add support for publishing UBI based runtime images .. [docker] Add support for publishing UBI based runtime images This patch adds a `--use-ubi-runtime` option to the docker build. When that option is passed for Centos 7 and 8 builds, the Kudu runtime images (`kudu` & `kudu-python`) will be built using a UBI base instead of the CentOS base. If the `--use-ubi-runtime` flag is passed for any unsupported OS, a clear error message will be printed. We do not build the development images using the UBI base because the package repositories in the UBI do not contain all the packages necessary to build Kudu and support development. Additionally the benefits of UBI are primarily for runtime/deployment usage and continuing to use CentOS for the development can simplify the build. This patch also required some small changes to how we generated the published image labels given the UBI images use a slightly different format than was previously handled. More can be read about RedHat UBI here: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image https://developers.redhat.com/articles/ubi-faq I tested building images with `--use-ubi-runtime` on both CentOS 7 and 8. Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Reviewed-on: http://gerrit.cloudera.org:8080/17564 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke --- M docker/Dockerfile M docker/bootstrap-runtime-env.sh M docker/docker-build.py 3 files changed, 56 insertions(+), 17 deletions(-) Approvals: Kudu Jenkins: Verified Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17564 ) Change subject: [docker] Add support for publishing UBI based runtime images .. Patch Set 4: Code-Review+2 Carrying the +2 through the commit message changes. -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 10 Jun 2021 01:43:01 + Gerrit-HasComments: No
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17564 ) Change subject: [docker] Add support for publishing UBI based runtime images .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/17564/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17564/3//COMMIT_MSG@18 PS3, Line 18: neccessary > necessary Done http://gerrit.cloudera.org:8080/#/c/17564/3//COMMIT_MSG@27 PS3, Line 27: here > The page below also contains some useful info and references: Done -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 10 Jun 2021 01:42:37 + Gerrit-HasComments: Yes
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17564 to look at the new patch set (#4). Change subject: [docker] Add support for publishing UBI based runtime images .. [docker] Add support for publishing UBI based runtime images This patch adds a `--use-ubi-runtime` option to the docker build. When that option is passed for Centos 7 and 8 builds, the Kudu runtime images (`kudu` & `kudu-python`) will be built using a UBI base instead of the CentOS base. If the `--use-ubi-runtime` flag is passed for any unsupported OS, a clear error message will be printed. We do not build the development images using the UBI base because the package repositories in the UBI do not contain all the packages necessary to build Kudu and support development. Additionally the benefits of UBI are primarily for runtime/deployment usage and continuing to use CentOS for the development can simplify the build. This patch also required some small changes to how we generated the published image labels given the UBI images use a slightly different format than was previously handled. More can be read about RedHat UBI here: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image https://developers.redhat.com/articles/ubi-faq I tested building images with `--use-ubi-runtime` on both CentOS 7 and 8. Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 --- M docker/Dockerfile M docker/bootstrap-runtime-env.sh M docker/docker-build.py 3 files changed, 56 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17564/4 -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/17564 )
Change subject: [docker] Add support for publishing UBI based runtime images
..
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17564/1/docker/docker-build.py
File docker/docker-build.py:
http://gerrit.cloudera.org:8080/#/c/17564/1/docker/docker-build.py@346
PS1, Line 346: if opts.use_ubi:
: try:
: runtime_base = UBI_MAP[base]
: except KeyError:
: print('ERROR: --use-ubi is only valid for the following
bases: %s' % UBI_MAP.keys())
: sys.exit(1)
: print('Runtime images will use UBI image %s...' %
runtime_base)
> A map check in a for loop is not expensive, and it could save a lengthy hea
Done
--
To view, visit http://gerrit.cloudera.org:8080/17564
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07
Gerrit-Change-Number: 17564
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 09 Jun 2021 22:00:35 +
Gerrit-HasComments: Yes
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Hello Kudu Jenkins, Andrew Wong, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17564 to look at the new patch set (#3). Change subject: [docker] Add support for publishing UBI based runtime images .. [docker] Add support for publishing UBI based runtime images This patch adds a `--use-ubi-runtime` option to the docker build. When that option is passed for Centos 7 and 8 builds, the Kudu runtime images (`kudu` & `kudu-python`) will be built using a UBI base instead of the CentOS base. If the `--use-ubi-runtime` flag is passed for any unsupported OS, a clear error message will be printed. We do not build the development images using the UBI base because the package repositories in the UBI do not contain all the packages neccessary to build Kudu and support development. Additionally the benefits of UBI are primarily for runtime/deployment usage and continuing to use CentOS for the development can simplify the build. This patch also required some small changes to how we generated the published image labels given the UBI images use a slightly different format than was previously handled. More can be read about RedHat UBI here: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image I tested building images with `--use-ubi-runtime` on both CentOS 7 and 8. Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 --- M docker/Dockerfile M docker/bootstrap-runtime-env.sh M docker/docker-build.py 3 files changed, 56 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17564/3 -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17564 ) Change subject: [docker] Add support for publishing UBI based runtime images .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/17564/1/docker/docker-build.py File docker/docker-build.py: http://gerrit.cloudera.org:8080/#/c/17564/1/docker/docker-build.py@78 PS1, Line 78: : # Targets that use the dev base image. > Missed this? Or maybe it already exist somewhere, given it's not really rel oh, sorry. I missed this. The definitions of the runtime and dev images are included in a comment above thier definition in the dockerfile. -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 09 Jun 2021 17:57:40 + Gerrit-HasComments: Yes
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Hello Kudu Jenkins, Andrew Wong, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17564 to look at the new patch set (#2). Change subject: [docker] Add support for publishing UBI based runtime images .. [docker] Add support for publishing UBI based runtime images This patch adds a `--use-ubi-runtime` option to the docker build. When that option is passed for Centos 7 and 8 builds, the Kudu runtime images (`kudu` & `kudu-python`) will be built using a UBI base instead of the CentOS base. If the `--use-ubi-runtime` flag is passed for any unsupported OS, a clear error message will be printed. We do not build the development images using the UBI base because the package repositories in the UBI do not contain all the packages neccessary to build Kudu and support development. Additionally the benefits of UBI are primarily for runtime/deployment usage and continuing to use CentOS for the development can simplify the build. This patch also required some small changes to how we generated the published image labels given the UBI images use a slightly different format than was previously handled. More can be read about RedHat UBI here: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image I tested building images with `--use-ubi-runtime` on both CentOS 7 and 8. Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 --- M docker/Dockerfile M docker/bootstrap-runtime-env.sh M docker/docker-build.py 3 files changed, 53 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17564/2 -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/17564 )
Change subject: [docker] Add support for publishing UBI based runtime images
..
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/17564/1/docker/Dockerfile
File docker/Dockerfile:
http://gerrit.cloudera.org:8080/#/c/17564/1/docker/Dockerfile@30
PS1, Line 30: s
> nit: drop the apostrophe
Done
http://gerrit.cloudera.org:8080/#/c/17564/1/docker/docker-build.py
File docker/docker-build.py:
http://gerrit.cloudera.org:8080/#/c/17564/1/docker/docker-build.py@100
PS1, Line 100: --use-ubi
> baking*
I think UBI is clearly understood as a Red Hat thing. I can add the "runtime"
suffix.
http://gerrit.cloudera.org:8080/#/c/17564/1/docker/docker-build.py@346
PS1, Line 346: if opts.use_ubi_runtime:
: try:
: runtime_base = UBI_MAP[base]
: except KeyError:
: print('ERROR: --use-ubi-runtime is only valid for the
following bases: %s' % UBI_MAP.keys())
: sys.exit(1)
: print('Runtime images will use UBI image %s...' %
runtime_base)
> To fail a bit faster, maybe move this into its own loop so we exit before d
I am not sure it's really worth it. This serves as both a validation and the
way to set the runtime_base for each base. I would end up doing it twice.
--
To view, visit http://gerrit.cloudera.org:8080/17564
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07
Gerrit-Change-Number: 17564
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 09 Jun 2021 14:50:39 +
Gerrit-HasComments: Yes
[kudu-CR] [docker] Add support for publishing UBI based runtime images
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17564 Change subject: [docker] Add support for publishing UBI based runtime images .. [docker] Add support for publishing UBI based runtime images This patch adds a `--use-ubi` option to the docker build. When that option is passed for Centos 7 and 8 builds, the Kudu runtime images (`kudu` & `kudu-python`) will be built using a UBI base instead of the CentOS base. If the `--use-ubi` flag is passed for any unsupported OS, a clear error message will be printed. We do not build the development images using the UBI base because the package repositories in the UBI do not contain all the packages neccessary to build Kudu and support development. Additionally the benefits of UBI are primarily for runtime/deployment usage and continuing to use CentOS for the development can simplify the build. This patch also required some small changes to how we generated the published image labels given the UBI images use a slightly different format than was previously handled. More can be read about RedHat UBI here: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image I tested building images with `--use-ubi` on both CentOS 7 and 8. Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 --- M docker/Dockerfile M docker/bootstrap-runtime-env.sh M docker/docker-build.py 3 files changed, 53 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/64/17564/1 -- To view, visit http://gerrit.cloudera.org:8080/17564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7be121c4ef8422d8aa7556aa163254b43c226b07 Gerrit-Change-Number: 17564 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-3291: properly disambiguate between deltas of a row with the same timestamp
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17547 ) Change subject: KUDU-3291: properly disambiguate between deltas of a row with the same timestamp .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17547 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iccfc518999d36679f85ed901ba65cf7b4894cd55 Gerrit-Change-Number: 17547 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 08 Jun 2021 14:45:14 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [hms] Avoid blocking table alterations that are not relevant to Kudu
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17537 ) Change subject: [hms] Avoid blocking table alterations that are not relevant to Kudu .. [hms] Avoid blocking table alterations that are not relevant to Kudu This patch adjust the KuduMetastorePlugin to prevent blocking table alterations that are not relevant to Kudu metadata which needs to be synchronized. Additionally it prevents calling the Kudu master to check if HMS sync is enabled when the metadata changes are not relevant to Kudu’s HMS synchronization. Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Reviewed-on: http://gerrit.cloudera.org:8080/17531 Tested-by: Kudu Jenkins Reviewed-by: Andrew Wong Reviewed-by: Alexey Serbin (cherry picked from commit 3667428d9071cbcf6825b07983529eddffc86dd5) Reviewed-on: http://gerrit.cloudera.org:8080/17537 Reviewed-by: Bankim Bhavsar --- M java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java 2 files changed, 184 insertions(+), 8 deletions(-) Approvals: Andrew Wong: Looks good to me, approved Kudu Jenkins: Verified Bankim Bhavsar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17537 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Gerrit-Change-Number: 17537 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [hms] Avoid blocking table alterations that are not relevant to Kudu
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17537 Change subject: [hms] Avoid blocking table alterations that are not relevant to Kudu .. [hms] Avoid blocking table alterations that are not relevant to Kudu This patch adjust the KuduMetastorePlugin to prevent blocking table alterations that are not relevant to Kudu metadata which needs to be synchronized. Additionally it prevents calling the Kudu master to check if HMS sync is enabled when the metadata changes are not relevant to Kudu’s HMS synchronization. Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Reviewed-on: http://gerrit.cloudera.org:8080/17531 Tested-by: Kudu Jenkins Reviewed-by: Andrew Wong Reviewed-by: Alexey Serbin (cherry picked from commit 3667428d9071cbcf6825b07983529eddffc86dd5) --- M java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java 2 files changed, 184 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/37/17537/1 -- To view, visit http://gerrit.cloudera.org:8080/17537 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Gerrit-Change-Number: 17537 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [hms] Avoid blocking table alterations that are not relevant to Kudu
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17531 ) Change subject: [hms] Avoid blocking table alterations that are not relevant to Kudu .. [hms] Avoid blocking table alterations that are not relevant to Kudu This patch adjust the KuduMetastorePlugin to prevent blocking table alterations that are not relevant to Kudu metadata which needs to be synchronized. Additionally it prevents calling the Kudu master to check if HMS sync is enabled when the metadata changes are not relevant to Kudu’s HMS synchronization. Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Reviewed-on: http://gerrit.cloudera.org:8080/17531 Tested-by: Kudu Jenkins Reviewed-by: Andrew Wong Reviewed-by: Alexey Serbin --- M java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java 2 files changed, 184 insertions(+), 8 deletions(-) Approvals: Kudu Jenkins: Verified Andrew Wong: Looks good to me, approved Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17531 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Gerrit-Change-Number: 17531 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [hms] Avoid blocking table alterations that are not relevant to Kudu
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/17531 )
Change subject: [hms] Avoid blocking table alterations that are not relevant to
Kudu
..
Patch Set 3:
(18 comments)
http://gerrit.cloudera.org:8080/#/c/17531/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17531/3//COMMIT_MSG@12
PS3, Line 12: metdata
> typo: metadata
Done
http://gerrit.cloudera.org:8080/#/c/17531/3//COMMIT_MSG@13
PS3, Line 13: syncronization
> synchronization
Done
http://gerrit.cloudera.org:8080/#/c/17531/3//COMMIT_MSG@13
PS3, Line 13: syncronization
> typo: synchronization
Done
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java
File
java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java:
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java@238
PS3, Line 238: a
> nit: an ?
Done
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java@424
PS3, Line 424: to the relevant metadata
> Is it possible that storage handler changes between 'before' and 'after' ta
yes, which is why I check for it below
(hive_metastoreConstants.META_TABLE_STORAGE).
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java@431
PS3, Line 431: kuduMetadataUnchanged
> Does it make sense to add a consistency check into the body of kuduMetadata
It feels a bit verbose. It's not technically required for the method to work
and we do already validate that in its usage.
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java@431
PS3, Line 431: public
> Is it really necessary to declare this method as public given the presence
I can make it package protected.
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java@433
PS3, Line 433: before.getParameters()
> nit: maybe pull before.getParameters() and after.getParameters() out into a
Done
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java@433
PS3, Line 433: if
(!Objects.equals(before.getParameters().get(hive_metastoreConstants.META_TABLE_STORAGE),
:
after.getParameters().get(hive_metastoreConstants.META_TABLE_STORAGE)) ||
:
!Objects.equals(before.getParameters().get(KUDU_MASTER_ADDRS_KEY),
: after.getParameters().get(KUDU_MASTER_ADDRS_KEY)) ||
:
!Objects.equals(before.getParameters().get(KUDU_TABLE_ID_KEY),
: after.getParameters().get(KUDU_TABLE_ID_KEY)) ||
:
!Objects.equals(before.getParameters().get(KUDU_TABLE_NAME_KEY),
: after.getParameters().get(KUDU_TABLE_NAME_KEY)) ||
:
!Objects.equals(before.getParameters().get(KUDU_CLUSTER_ID_KEY),
: after.getParameters().get(KUDU_CLUSTER_ID_KEY))) {
> nit: to reduce the clutter, maybe introduce 'paramsBefore' and 'paramsAfter
Done
http://gerrit.cloudera.org:8080/#/c/17531/3/java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java@432
PS3, Line 432: // If any of the Kudu table properties have changed, return
false.
: if
(!Objects.equals(before.getParameters().get(hive_metastoreConstants.META_TABLE_STORAGE),
:
after.getParameters().get(hive_metastoreConstants.META_TABLE_STORAGE)) ||
:
!Objects.equals(before.getParameters().get(KUDU_MASTER_ADDRS_KEY),
: after.getParameters().get(KUDU_MASTER_ADDRS_KEY)) ||
:
!Objects.equals(before.getParameters().get(KUDU_TABLE_ID_KEY),
: after.getParameters().get(KUDU_TABLE_ID_KEY)) ||
:
!Objects.equals(before.getParameters().get(KUDU_TABLE_NAME_KEY),
: after.getParameters().get(KUDU_TABLE_NAME_KEY)) ||
:
!Objects.equals(before.getParameters().get(KUDU_CLUSTER_ID_KEY),
: after.getParameters().get(KUDU_CLUSTER_ID_KEY))) {
: return false;
: }
> I'm a little surprised to see some fields here that don't get synchronized
These don't technically get "synced" with Kudu but they would change the
perceived synchronization state (whether or not a table is considered
synchronized & in-sync by Kudu) so we should check if HMS sync is enabled to
ensure that is allowed.
[kudu-CR] [hms] Avoid blocking table alterations that are not relevant to Kudu
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17531 to look at the new patch set (#4). Change subject: [hms] Avoid blocking table alterations that are not relevant to Kudu .. [hms] Avoid blocking table alterations that are not relevant to Kudu This patch adjust the KuduMetastorePlugin to prevent blocking table alterations that are not relevant to Kudu metadata which needs to be synchronized. Additionally it prevents calling the Kudu master to check if HMS sync is enabled when the metadata changes are not relevant to Kudu’s HMS synchronization. Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db --- M java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java 2 files changed, 184 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/17531/4 -- To view, visit http://gerrit.cloudera.org:8080/17531 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Gerrit-Change-Number: 17531 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [java] KUDU-3267 Improve logging on writes to non-existent partitions
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17535 Change subject: [java] KUDU-3267 Improve logging on writes to non-existent partitions .. [java] KUDU-3267 Improve logging on writes to non-existent partitions This patch helps to log a more meaningful error message when a non-existent range partition is written to by the Kudu Java client. Due to different behavior of the client in different flush modes, the existing messages returned by the client are different when different flush modes are used. Effort has been made to bring consistency to the error messages returned in all the flush modes as well. For example in the default AUTO_FLUSH_BACKGROUND mode Existing log message snippet: Not found: ([0x00018005BDAA4BF52400, 0x00028005BCC2F4C85400)) New log message snippet (in any FLUSH mode): Not Found: accessed range partition ([0x8064, 0x80C8)) does not exist in table: TestKuduSession Thanks to Grant Henke for the help in writing this patch. Change-Id: Ia24582de6b060e908f5ecbc46e2638b95cd567b3 Reviewed-on: http://gerrit.cloudera.org:8080/17518 Tested-by: Alexey Serbin Reviewed-by: Alexey Serbin (cherry picked from commit cae3ec8df5037f560defe5b121f197f27c7065f1) --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java M java/kudu-client/src/main/java/org/apache/kudu/client/NonCoveredRangeException.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduPartitioner.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduSession.java 4 files changed, 59 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/35/17535/1 -- To view, visit http://gerrit.cloudera.org:8080/17535 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: Ia24582de6b060e908f5ecbc46e2638b95cd567b3 Gerrit-Change-Number: 17535 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Abhishek Chennaka
[kudu-CR](branch-1.15.x) [consensus] --evict failed followers is a runtime flag
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17534 Change subject: [consensus] --evict_failed_followers is a runtime flag .. [consensus] --evict_failed_followers is a runtime flag Current implementation of Raft consensus treats --evict_failed_followers as a runtime flag. This patch adds corresponding tag for the flag. Change-Id: Ia7ff0c4097df72c3a2d43ee166e29af5f67b5687 Reviewed-on: http://gerrit.cloudera.org:8080/17522 Tested-by: Kudu Jenkins Reviewed-by: Bankim Bhavsar (cherry picked from commit 0a8c84a53b4bbe916f9afca3280eb863840ece2a) --- M src/kudu/consensus/raft_consensus.cc 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/34/17534/1 -- To view, visit http://gerrit.cloudera.org:8080/17534 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: Ia7ff0c4097df72c3a2d43ee166e29af5f67b5687 Gerrit-Change-Number: 17534 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [hms] Avoid blocking table alterations that are not relevant to Kudu
Hello Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17531 to look at the new patch set (#3). Change subject: [hms] Avoid blocking table alterations that are not relevant to Kudu .. [hms] Avoid blocking table alterations that are not relevant to Kudu This patch adjust the KuduMetastorePlugin to prevent blocking table alterations that are not relevant to Kudu metadata which needs to be synchronized. Additionally it prevents calling the Kudu master to check if HMS sync is enabled when the metdata changes are not relevant to Kudu’s HMS syncronization. Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db --- M java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java 2 files changed, 181 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/17531/3 -- To view, visit http://gerrit.cloudera.org:8080/17531 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Gerrit-Change-Number: 17531 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2612 allow system user to read list of table replicas
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17529 ) Change subject: KUDU-2612 allow system user to read list of table replicas .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17529 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I26f06af17e5ee85522e2ef867d41cf0f3ddbe5d5 Gerrit-Change-Number: 17529 Gerrit-PatchSet: 3 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 02 Jun 2021 17:49:00 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) KUDU-2612 allow system user to read list of table replicas
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17532 ) Change subject: KUDU-2612 allow system user to read list of table replicas .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17532 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I26f06af17e5ee85522e2ef867d41cf0f3ddbe5d5 Gerrit-Change-Number: 17532 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 02 Jun 2021 17:47:27 + Gerrit-HasComments: No
[kudu-CR] [hms] Avoid blocking table alterations that are not relevant to Kudu
Hello Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17531 to look at the new patch set (#2). Change subject: [hms] Avoid blocking table alterations that are not relevant to Kudu .. [hms] Avoid blocking table alterations that are not relevant to Kudu This patch adjust the KuduMetastorePlugin to prevent blocking table alterations that are not relevant to Kudu metadata which needs to be synchronized. Additionally it prevents calling the Kudu master to check if HMS sync is enabled when the metdata changes are not relevant to Kudu’s HMS syncronization. Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db --- M java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java 2 files changed, 182 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/17531/2 -- To view, visit http://gerrit.cloudera.org:8080/17531 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Gerrit-Change-Number: 17531 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [hms] Avoid blocking table alterations that are not relevant to Kudu
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17531 Change subject: [hms] Avoid blocking table alterations that are not relevant to Kudu .. [hms] Avoid blocking table alterations that are not relevant to Kudu This patch adjust the KuduMetastorePlugin to prevent blocking table alterations that are not relevant to Kudu metadata which needs to be synchronized. Additionally it prevents calling the Kudu master to check if HMS sync is enabled when the metdata changes are not relevant to Kudu’s HMS syncronization. Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db --- M java/kudu-hive/src/main/java/org/apache/kudu/hive/metastore/KuduMetastorePlugin.java M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java 2 files changed, 182 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/17531/1 -- To view, visit http://gerrit.cloudera.org:8080/17531 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2517d891ab7168164700bb3ae642c49bde54b9db Gerrit-Change-Number: 17531 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR](branch-1.15.x) [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17513 ) Change subject: [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job .. [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this patch users can change the existing database name of the table or add a new database name to the table i.e. the prefix to the '.' in the full tablename as well as remove impala prefix for the tables which are being restored. 1.--newDatabaseName : Use this option to specify the new database name for the restored table. This will overwrite any existing database and if there is no existing database, a new database will be added to the table name. Will not affect "impala::" prefix. E.g: Adding database name "newDB" to the tables impala::default.test, impala::test, default.test, test will result in the table names impala::newDB.test, impala::newDB.test, newDB.test, newDB.test . This will not affect the existing/source tables. 2.--removeImpalaPrefix : If enabled, this option will remove the “impala::” prefix, if present from the restored table names. This is advisable if tables are backed up in Kudu clusters without HMS integration and being restored to Kudu clusters with HMS integration. Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Reviewed-on: http://gerrit.cloudera.org:8080/17388 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke (cherry picked from commit bd37d601d36bcf51c595baad52b5d24b5e20d684) Reviewed-on: http://gerrit.cloudera.org:8080/17513 Tested-by: Grant Henke Reviewed-by: Bankim Bhavsar --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestOptions.scala 4 files changed, 126 insertions(+), 4 deletions(-) Approvals: Grant Henke: Verified Bankim Bhavsar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17513 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17513 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17511 ) Change subject: [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls .. Patch Set 1: The failure is unrelated to this change. -- To view, visit http://gerrit.cloudera.org:8080/17511 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: Id4ff9cd8943c780d4fe8bf6afb2ef540c07a5c8d Gerrit-Change-Number: 17511 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 26 May 2021 16:23:59 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17511 ) Change subject: [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17511 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: Id4ff9cd8943c780d4fe8bf6afb2ef540c07a5c8d Gerrit-Change-Number: 17511 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 26 May 2021 16:23:45 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls
Grant Henke has removed a vote on this change. Change subject: [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/17511 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: deleteVote Gerrit-Change-Id: Id4ff9cd8943c780d4fe8bf6afb2ef540c07a5c8d Gerrit-Change-Number: 17511 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [consensus] a small optimisation on log prefix
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17512 ) Change subject: [consensus] a small optimisation on log prefix .. [consensus] a small optimisation on log prefix This patch introduces a small optimisation into the implementation of the RaftConsensus::LogPrefixThreadSafe() method: since the prefix it produces doesn't change, it's possible to construct the string once in the constructor and then return a constant reference. I also updated PendingRound class to store a reference to the prefix as well since instances of the PendingRound class have shorter live span than RaftConsensus objects which create PendingRound objects. Change-Id: I6a5082fe07449ef24827696638ddd8e925a08264 Reviewed-on: http://gerrit.cloudera.org:8080/17489 Tested-by: Alexey Serbin Reviewed-by: Andrew Wong (cherry picked from commit 8ffa18196dcd3917ae624f9dc9679040e65967cb) Reviewed-on: http://gerrit.cloudera.org:8080/17512 Tested-by: Kudu Jenkins Reviewed-by: Bankim Bhavsar --- M src/kudu/consensus/pending_rounds.cc M src/kudu/consensus/pending_rounds.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h 4 files changed, 16 insertions(+), 17 deletions(-) Approvals: Kudu Jenkins: Verified Bankim Bhavsar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17512 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I6a5082fe07449ef24827696638ddd8e925a08264 Gerrit-Change-Number: 17512 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17513 ) Change subject: [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/17513 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17513 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 26 May 2021 16:22:36 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job
Grant Henke has removed a vote on this change. Change subject: [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/17513 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: deleteVote Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17513 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17514 ) Change subject: [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig .. [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig When looking at one of recent pre-commit test failure [1][2], I found that AdminCliTest.TestUnsafeChangeConfigLeaderWithPendingConfig failed but it was hard to tell what was the actual status code returned: src/kudu/tools/kudu-admin-test.cc:881: Failure Value of: s.IsTimedOut() Actual: false Expected: true I reran the scenario multiple times with TSAN-enabled binaries, but was not able to reproduce the test failure yet. Anyways, I added more diagnostics about the actual status code returned and cleaned up the code of the test scenario a bit. Hopefully, next time it fails it will be clearer what's going on. [1] http://jenkins.kudu.apache.org/job/kudu-gerrit/23918/BUILD_TYPE=TSAN [2] http://dist-test.cloudera.org/job?job_id=jenkins-slave.1621893605.3746155 Change-Id: Id2bf354764f3ddb371de19910c5f879c1a06c78b Reviewed-on: http://gerrit.cloudera.org:8080/17504 Tested-by: Kudu Jenkins Reviewed-by: Mahesh Reddy Reviewed-by: Andrew Wong Reviewed-by: Grant Henke (cherry picked from commit a0bda0989d97fac182f409ecaef3357a1fb5329f) Reviewed-on: http://gerrit.cloudera.org:8080/17514 Reviewed-by: Alexey Serbin --- M src/kudu/tools/kudu-admin-test.cc 1 file changed, 7 insertions(+), 13 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/17514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: Id2bf354764f3ddb371de19910c5f879c1a06c78b Gerrit-Change-Number: 17514 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17514 Change subject: [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig .. [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig When looking at one of recent pre-commit test failure [1][2], I found that AdminCliTest.TestUnsafeChangeConfigLeaderWithPendingConfig failed but it was hard to tell what was the actual status code returned: src/kudu/tools/kudu-admin-test.cc:881: Failure Value of: s.IsTimedOut() Actual: false Expected: true I reran the scenario multiple times with TSAN-enabled binaries, but was not able to reproduce the test failure yet. Anyways, I added more diagnostics about the actual status code returned and cleaned up the code of the test scenario a bit. Hopefully, next time it fails it will be clearer what's going on. [1] http://jenkins.kudu.apache.org/job/kudu-gerrit/23918/BUILD_TYPE=TSAN [2] http://dist-test.cloudera.org/job?job_id=jenkins-slave.1621893605.3746155 Change-Id: Id2bf354764f3ddb371de19910c5f879c1a06c78b Reviewed-on: http://gerrit.cloudera.org:8080/17504 Tested-by: Kudu Jenkins Reviewed-by: Mahesh Reddy Reviewed-by: Andrew Wong Reviewed-by: Grant Henke (cherry picked from commit a0bda0989d97fac182f409ecaef3357a1fb5329f) --- M src/kudu/tools/kudu-admin-test.cc 1 file changed, 7 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/14/17514/1 -- To view, visit http://gerrit.cloudera.org:8080/17514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: Id2bf354764f3ddb371de19910c5f879c1a06c78b Gerrit-Change-Number: 17514 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17504 ) Change subject: [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig .. [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig When looking at one of recent pre-commit test failure [1][2], I found that AdminCliTest.TestUnsafeChangeConfigLeaderWithPendingConfig failed but it was hard to tell what was the actual status code returned: src/kudu/tools/kudu-admin-test.cc:881: Failure Value of: s.IsTimedOut() Actual: false Expected: true I reran the scenario multiple times with TSAN-enabled binaries, but was not able to reproduce the test failure yet. Anyways, I added more diagnostics about the actual status code returned and cleaned up the code of the test scenario a bit. Hopefully, next time it fails it will be clearer what's going on. [1] http://jenkins.kudu.apache.org/job/kudu-gerrit/23918/BUILD_TYPE=TSAN [2] http://dist-test.cloudera.org/job?job_id=jenkins-slave.1621893605.3746155 Change-Id: Id2bf354764f3ddb371de19910c5f879c1a06c78b Reviewed-on: http://gerrit.cloudera.org:8080/17504 Tested-by: Kudu Jenkins Reviewed-by: Mahesh Reddy Reviewed-by: Andrew Wong Reviewed-by: Grant Henke --- M src/kudu/tools/kudu-admin-test.cc 1 file changed, 7 insertions(+), 13 deletions(-) Approvals: Kudu Jenkins: Verified Mahesh Reddy: Looks good to me, but someone else must approve Andrew Wong: Looks good to me, approved Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id2bf354764f3ddb371de19910c5f879c1a06c78b Gerrit-Change-Number: 17504 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy
[kudu-CR](branch-1.15.x) [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17513 Change subject: [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job .. [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this patch users can change the existing database name of the table or add a new database name to the table i.e. the prefix to the '.' in the full tablename as well as remove impala prefix for the tables which are being restored. 1.--newDatabaseName : Use this option to specify the new database name for the restored table. This will overwrite any existing database and if there is no existing database, a new database will be added to the table name. Will not affect "impala::" prefix. E.g: Adding database name "newDB" to the tables impala::default.test, impala::test, default.test, test will result in the table names impala::newDB.test, impala::newDB.test, newDB.test, newDB.test . This will not affect the existing/source tables. 2.--removeImpalaPrefix : If enabled, this option will remove the “impala::” prefix, if present from the restored table names. This is advisable if tables are backed up in Kudu clusters without HMS integration and being restored to Kudu clusters with HMS integration. Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Reviewed-on: http://gerrit.cloudera.org:8080/17388 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke (cherry picked from commit bd37d601d36bcf51c595baad52b5d24b5e20d684) --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestOptions.scala 4 files changed, 126 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/17513/1 -- To view, visit http://gerrit.cloudera.org:8080/17513 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17513 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Abhishek Chennaka
[kudu-CR] [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17388 ) Change subject: [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job .. [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this patch users can change the existing database name of the table or add a new database name to the table i.e. the prefix to the '.' in the full tablename as well as remove impala prefix for the tables which are being restored. 1.--newDatabaseName : Use this option to specify the new database name for the restored table. This will overwrite any existing database and if there is no existing database, a new database will be added to the table name. Will not affect "impala::" prefix. E.g: Adding database name "newDB" to the tables impala::default.test, impala::test, default.test, test will result in the table names impala::newDB.test, impala::newDB.test, newDB.test, newDB.test . This will not affect the existing/source tables. 2.--removeImpalaPrefix : If enabled, this option will remove the “impala::” prefix, if present from the restored table names. This is advisable if tables are backed up in Kudu clusters without HMS integration and being restored to Kudu clusters with HMS integration. Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Reviewed-on: http://gerrit.cloudera.org:8080/17388 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestOptions.scala 4 files changed, 126 insertions(+), 4 deletions(-) Approvals: Kudu Jenkins: Verified Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17388 Gerrit-PatchSet: 10 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [consensus] a small optimisation on log prefix
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17512 Change subject: [consensus] a small optimisation on log prefix .. [consensus] a small optimisation on log prefix This patch introduces a small optimisation into the implementation of the RaftConsensus::LogPrefixThreadSafe() method: since the prefix it produces doesn't change, it's possible to construct the string once in the constructor and then return a constant reference. I also updated PendingRound class to store a reference to the prefix as well since instances of the PendingRound class have shorter live span than RaftConsensus objects which create PendingRound objects. Change-Id: I6a5082fe07449ef24827696638ddd8e925a08264 Reviewed-on: http://gerrit.cloudera.org:8080/17489 Tested-by: Alexey Serbin Reviewed-by: Andrew Wong (cherry picked from commit 8ffa18196dcd3917ae624f9dc9679040e65967cb) --- M src/kudu/consensus/pending_rounds.cc M src/kudu/consensus/pending_rounds.h M src/kudu/consensus/raft_consensus.cc M src/kudu/consensus/raft_consensus.h 4 files changed, 16 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/12/17512/1 -- To view, visit http://gerrit.cloudera.org:8080/17512 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I6a5082fe07449ef24827696638ddd8e925a08264 Gerrit-Change-Number: 17512 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR](branch-1.15.x) [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17511 Change subject: [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls .. [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls Prior to this patch, the TestRecoverFromOpIdOverflow scenario of the TsRecoveryITestDeathTest hasn't actually run for TSAN builds, but always reported success. A proper way of dealing with such conditions is to either skip a scenario or don't compile it at all as we do for others: * TestCFileBothCacheMemoryTypes.TestReadWriteLargeStrings * ThreadPoolTest.TestDeadlocks * DebugUtilTest.TestUnwindWhileUnsafe Change-Id: Id4ff9cd8943c780d4fe8bf6afb2ef540c07a5c8d Reviewed-on: http://gerrit.cloudera.org:8080/17491 Tested-by: Alexey Serbin Reviewed-by: Andrew Wong (cherry picked from commit 725c8b728a4c9fd582114aa529d7f8fba186bc9d) --- M src/kudu/integration-tests/ts_recovery-itest.cc 1 file changed, 6 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/11/17511/1 -- To view, visit http://gerrit.cloudera.org:8080/17511 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: Id4ff9cd8943c780d4fe8bf6afb2ef540c07a5c8d Gerrit-Change-Number: 17511 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17504 ) Change subject: [test] minor clean up on TestUnsafeChangeConfigLeaderWithPendingConfig .. Patch Set 1: Code-Review+2 I am not against logging in tests in general as I do often find it a useful way to track/debug what's happening. That said if this info has proven unhelpful I am okay with removing it. -- To view, visit http://gerrit.cloudera.org:8080/17504 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id2bf354764f3ddb371de19910c5f879c1a06c78b Gerrit-Change-Number: 17504 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy Gerrit-Comment-Date: Wed, 26 May 2021 02:20:56 + Gerrit-HasComments: No
[kudu-CR] [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17388 ) Change subject: [backup] KUDU-3183 Add --newDatabaseName and --removeImpalaPrefix options to restore job .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17388 Gerrit-PatchSet: 9 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 26 May 2021 01:28:13 + Gerrit-HasComments: No
[kudu-CR] KUDU-1884: set sasl protocol name for the TxnSystemClient
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17480 ) Change subject: KUDU-1884: set sasl_protocol_name for the TxnSystemClient .. Patch Set 7: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf Gerrit-Change-Number: 17480 Gerrit-PatchSet: 7 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 25 May 2021 17:52:15 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) KUDU-2612: disable TxnSystemClient initialization by default
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17506 Change subject: KUDU-2612: disable TxnSystemClient initialization by default .. KUDU-2612: disable TxnSystemClient initialization by default Currently the TxnSystemClient gets initialized on tablet servers by default, retrying every second until successful, and logging an error every minute if unable to. Since transactions support is currently experimental and disabled by default, so too should this initialization be. This patch hides the initialization with the existing --disable_txn_system_client_init flag (previously used for tests), adjusted to --enable_txn_system_client_init to match most of our feature-gating flags. Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b Reviewed-on: http://gerrit.cloudera.org:8080/17499 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin Reviewed-by: Grant Henke (cherry picked from commit 923245291fade95febc36265cfba8cc92ee457d5) --- M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTransaction.java M src/kudu/client/client-test.cc M src/kudu/integration-tests/client-negotiation-failover-itest.cc M src/kudu/integration-tests/location_assignment-itest.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/ts_tablet_manager-itest.cc M src/kudu/integration-tests/txn_commit-itest.cc M src/kudu/integration-tests/txn_status_manager-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/integration-tests/txn_write_ops-itest.cc M src/kudu/tools/kudu-txn-cli-test.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/transactions/txn_system_client.cc M src/kudu/tserver/tablet_service.cc 14 files changed, 144 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/06/17506/1 -- To view, visit http://gerrit.cloudera.org:8080/17506 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b Gerrit-Change-Number: 17506 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong
[kudu-CR] KUDU-1884: set sasl protocol name for the TxnSystemClient
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17480 ) Change subject: KUDU-1884: set sasl_protocol_name for the TxnSystemClient .. Patch Set 6: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf Gerrit-Change-Number: 17480 Gerrit-PatchSet: 6 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 25 May 2021 17:49:02 + Gerrit-HasComments: No
[kudu-CR] KUDU-1884: set sasl protocol name for the TxnSystemClient
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17480 ) Change subject: KUDU-1884: set sasl_protocol_name for the TxnSystemClient .. Patch Set 5: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf Gerrit-Change-Number: 17480 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 25 May 2021 16:04:24 + Gerrit-HasComments: No
[kudu-CR] KUDU-1884: set sasl protocol name for the TxnSystemClient
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17480 ) Change subject: KUDU-1884: set sasl_protocol_name for the TxnSystemClient .. Patch Set 5: Code-Review+1 You may need to rebase this on https://gerrit.cloudera.org/#/c/17499/ -- To view, visit http://gerrit.cloudera.org:8080/17480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf Gerrit-Change-Number: 17480 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 25 May 2021 15:51:38 + Gerrit-HasComments: No
[kudu-CR] KUDU-2612: disable TxnSystemClient initialization by default
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17499 ) Change subject: KUDU-2612: disable TxnSystemClient initialization by default .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b Gerrit-Change-Number: 17499 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Tue, 25 May 2021 15:48:21 + Gerrit-HasComments: No
[kudu-CR] [backup] KUDU-3183: Add --newDatabaseName and --removeImpalaPrefix options to restore job
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/17388 )
Change subject: [backup] KUDU-3183: Add --newDatabaseName and
--removeImpalaPrefix options to restore job
..
Patch Set 7: Code-Review+1
(7 comments)
http://gerrit.cloudera.org:8080/#/c/17388/7//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17388/7//COMMIT_MSG@24
PS7, Line 24: newDBtest
Shouldn't this be newDB.test? Otherwise it's just changing the table name.
http://gerrit.cloudera.org:8080/#/c/17388/7//COMMIT_MSG@24
PS7, Line 24: impala::newDB.default.test
Shouldn't this also be impala::newDB.test? If we create a table that is
impala::newDB.default.test then the table name changes to be "default.test"
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala:
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@48
PS7, Line 48: /**
nit: I had a hard time parsing some of this logic and tried simplifying it a
bit. Below is the code I wrote out. Feel free to use it if you think it's
clearer.
val ImpalaPrefix = "impala::"
/**
* Returns the table name in which the data will be restored considering the
flags removeImpalaPrefix,
* newDatabaseName and tableSuffix
*/
private def getRestoreTableName(fullTableName: String, options:
RestoreOptions): String = {
// Break the table down into prefix::databaseName.tableName
var prefix = ""
var databaseName = ""
var tableName = fullTableName
val hasImpalaPrefix = tableName.startsWith(ImpalaPrefix)
if (hasImpalaPrefix) {
prefix = ImpalaPrefix
tableName = tableName.substring(ImpalaPrefix.length)
}
val hasDatabase = tableName.contains(".")
if (hasDatabase) {
databaseName = tableName.substring(0, tableName.indexOf(".") + 1)
tableName = tableName.substring(tableName.indexOf(".") + 1)
}
// If the user does not want the Impala prefix, drop it
if (options.removeImpalaPrefix) {
prefix = ""
}
// If there is a databaseName specified by the user, use that
if (options.newDatabaseName.nonEmpty) {
databaseName = options.newDatabaseName.concat(".")
}
s"${prefix}${databaseName}${tableName}${options.tableSuffix}"
}
This matches closely to the code added in restoreAndValidateTable. I didn't run
it so there could be some mistakes.
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala
File java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala:
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala@238
PS7, Line 238: "If set, replaces the existing database name and if
there is no existing database name, a new database name " +
nit: Move up to the text( line like the rest of the options.
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala@240
PS7, Line 240: E.g.
replace with "example"
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala
File
java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala:
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala@594
PS7, Line 594: def testRemoveImpalaPrefixFlag() {
Nit: testTableNameChangeFlags given this includes testing removeImpalaPrefix
and newDatabaseName.
http://gerrit.cloudera.org:8080/#/c/17388/7/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala@830
PS7, Line 830: var onlyTableName = tableName
Can we just use KuduRestore.getRestoreTableName here given we manually validate
it in a different test above?
--
To view, visit http://gerrit.cloudera.org:8080/17388
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421
Gerrit-Change-Number: 17388
Gerrit-PatchSet: 7
Gerrit-Owner: Abhishek Chennaka
Gerrit-Reviewer: Abhishek Chennaka
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 25 May 2021 15:25:59 +
Gerrit-HasComments: Yes
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17495 ) Change subject: [kserver] output info on server-wide size limit .. [kserver] output info on server-wide size limit With this patch, Kudu servers output an INFO log line about the maximum number of threads in a server-wide thread pool. That's useful when assessing how many threads a Kudu server might legitimately spawn. Yes, those numbers can be calculated given the setting for RLIMIT_NPROC and values in /proc/sys/kernel/pid_max and /proc/sys/kernel/threads-max, but (a) those settings might change after the start of a server and (b) it's convenient to have a pre-computed number ready in the logs. Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Reviewed-on: http://gerrit.cloudera.org:8080/17474 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin (cherry picked from commit 48068c7cbf4066e3d2081960b50d466baf01543d) Reviewed-on: http://gerrit.cloudera.org:8080/17495 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin --- M src/kudu/kserver/kserver.cc 1 file changed, 6 insertions(+), 4 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [txns] Allow the TxnSystemClient to run in Authorized environments
Grant Henke has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/17498 )
Change subject: [txns] Allow the TxnSystemClient to run in Authorized
environments
..
[txns] Allow the TxnSystemClient to run in Authorized environments
When writing a test that runs the TxnSystemClient in an authorized
environment I saw issues where the TxnSystemClient could not
be initialized due to being unauthorized. It turns out that there
are many remote methods that the service user is not authorized
to call. This patch fixes the issue by updating the various required
calls to also allow the service user. I also enhanced a test to leverage
transactions in security-itest.cc
Here is a sample error I saw before this change:
W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt
to method kudu.master.MasterService.ConnectToMaster from {username='kudu',
principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639
W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize
TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the
cluster: Not authorized: unauthorized access to method: ConnectToMaster
Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0
Reviewed-on: http://gerrit.cloudera.org:8080/17490
Tested-by: Alexey Serbin
Reviewed-by: Grant Henke
(cherry picked from commit 35de43051b65c8eab0f348bfd7c8a0d6e49c8067)
Reviewed-on: http://gerrit.cloudera.org:8080/17498
Reviewed-by: Alexey Serbin
---
M src/kudu/client/client-test.cc
M src/kudu/integration-tests/security-itest.cc
M src/kudu/integration-tests/txn_status_table-itest.cc
M src/kudu/master/master.proto
4 files changed, 64 insertions(+), 20 deletions(-)
Approvals:
Alexey Serbin: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/17498
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.15.x
Gerrit-MessageType: merged
Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0
Gerrit-Change-Number: 17498
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [txns] Allow the TxnSystemClient to run in Authorized environments
Grant Henke has uploaded this change for review. (
http://gerrit.cloudera.org:8080/17498
Change subject: [txns] Allow the TxnSystemClient to run in Authorized
environments
..
[txns] Allow the TxnSystemClient to run in Authorized environments
When writing a test that runs the TxnSystemClient in an authorized
environment I saw issues where the TxnSystemClient could not
be initialized due to being unauthorized. It turns out that there
are many remote methods that the service user is not authorized
to call. This patch fixes the issue by updating the various required
calls to also allow the service user. I also enhanced a test to leverage
transactions in security-itest.cc
Here is a sample error I saw before this change:
W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt
to method kudu.master.MasterService.ConnectToMaster from {username='kudu',
principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639
W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize
TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the
cluster: Not authorized: unauthorized access to method: ConnectToMaster
Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0
Reviewed-on: http://gerrit.cloudera.org:8080/17490
Tested-by: Alexey Serbin
Reviewed-by: Grant Henke
(cherry picked from commit 35de43051b65c8eab0f348bfd7c8a0d6e49c8067)
---
M src/kudu/client/client-test.cc
M src/kudu/integration-tests/security-itest.cc
M src/kudu/integration-tests/txn_status_table-itest.cc
M src/kudu/master/master.proto
4 files changed, 64 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/17498/1
--
To view, visit http://gerrit.cloudera.org:8080/17498
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: branch-1.15.x
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0
Gerrit-Change-Number: 17498
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17495 ) Change subject: [kserver] output info on server-wide size limit .. Patch Set 1: yes, this is a clean backport which shouldn't introduce test issues. I reran and the run passed. -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 21:33:13 + Gerrit-HasComments: No
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 21:33:39 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [security] fix const-correctness of some ACL methods
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17497 ) Change subject: [security] fix const-correctness of some ACL methods .. [security] fix const-correctness of some ACL methods This patch fixes a few methods in simple_acl and server_base to be const-correct. This patch doesn't introduce any functional changes. Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Reviewed-on: http://gerrit.cloudera.org:8080/17496 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin (cherry picked from commit 68f0f1a1d1d9beaff60243b77b64f4b1b359702c) Reviewed-on: http://gerrit.cloudera.org:8080/17497 Tested-by: Kudu Jenkins --- M src/kudu/security/simple_acl.cc M src/kudu/security/simple_acl.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h 4 files changed, 9 insertions(+), 10 deletions(-) Approvals: Bankim Bhavsar: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/17497 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17497 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [java] a property to show output while running a test
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17494 ) Change subject: [java] a property to show output while running a test .. [java] a property to show output while running a test While working on non-trivial and relatively long running test scenarios, I found it's inconvenient that the test output isn't shown while a test scenario is running. This patch introduces a new command-line property 'showTestOutput' to direct gradle outputting stdout and stderr from the JVM on the console, so it's possible to do the standard stream redirection and other stream manipulation in a command line shell. An example of running a particular test scenario with the output redirected to the console: ./gradlew -DshowTestOutput cleanTest :kudu-client:test --test ... By default the property isn't set, so this patch preserves the original behavior w.r.t. the output of running Java tests. Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Reviewed-on: http://gerrit.cloudera.org:8080/17479 Tested-by: Kudu Jenkins Reviewed-by: Bankim Bhavsar Reviewed-by: Andrew Wong (cherry picked from commit 7cdc9f7c735efe99b2d39d61ac4034d41473522e) Reviewed-on: http://gerrit.cloudera.org:8080/17494 --- M java/gradle/tests.gradle 1 file changed, 4 insertions(+), 0 deletions(-) Approvals: Kudu Jenkins: Verified Bankim Bhavsar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17494 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Gerrit-Change-Number: 17494 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [security] fix const-correctness of some ACL methods
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17497 Change subject: [security] fix const-correctness of some ACL methods .. [security] fix const-correctness of some ACL methods This patch fixes a few methods in simple_acl and server_base to be const-correct. This patch doesn't introduce any functional changes. Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Reviewed-on: http://gerrit.cloudera.org:8080/17496 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin (cherry picked from commit 68f0f1a1d1d9beaff60243b77b64f4b1b359702c) --- M src/kudu/security/simple_acl.cc M src/kudu/security/simple_acl.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h 4 files changed, 9 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/97/17497/1 -- To view, visit http://gerrit.cloudera.org:8080/17497 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17497 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 3: Looks like I need to fix up a few tests that depended on the bad behavior -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 16:50:52 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17495 Change subject: [kserver] output info on server-wide size limit .. [kserver] output info on server-wide size limit With this patch, Kudu servers output an INFO log line about the maximum number of threads in a server-wide thread pool. That's useful when assessing how many threads a Kudu server might legitimately spawn. Yes, those numbers can be calculated given the setting for RLIMIT_NPROC and values in /proc/sys/kernel/pid_max and /proc/sys/kernel/threads-max, but (a) those settings might change after the start of a server and (b) it's convenient to have a pre-computed number ready in the logs. Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Reviewed-on: http://gerrit.cloudera.org:8080/17474 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin (cherry picked from commit 48068c7cbf4066e3d2081960b50d466baf01543d) --- M src/kudu/kserver/kserver.cc 1 file changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/17495/1 -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR](branch-1.15.x) [java] a property to show output while running a test
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17494 Change subject: [java] a property to show output while running a test .. [java] a property to show output while running a test While working on non-trivial and relatively long running test scenarios, I found it's inconvenient that the test output isn't shown while a test scenario is running. This patch introduces a new command-line property 'showTestOutput' to direct gradle outputting stdout and stderr from the JVM on the console, so it's possible to do the standard stream redirection and other stream manipulation in a command line shell. An example of running a particular test scenario with the output redirected to the console: ./gradlew -DshowTestOutput cleanTest :kudu-client:test --test ... By default the property isn't set, so this patch preserves the original behavior w.r.t. the output of running Java tests. Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Reviewed-on: http://gerrit.cloudera.org:8080/17479 Tested-by: Kudu Jenkins Reviewed-by: Bankim Bhavsar Reviewed-by: Andrew Wong (cherry picked from commit 7cdc9f7c735efe99b2d39d61ac4034d41473522e) --- M java/gradle/tests.gradle 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/17494/1 -- To view, visit http://gerrit.cloudera.org:8080/17494 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Gerrit-Change-Number: 17494 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Hello Tidy Bot, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17490
to look at the new patch set (#3).
Change subject: [txns] Allow the TxnSystemClient to run in Authorized
environments
..
[txns] Allow the TxnSystemClient to run in Authorized environments
When writing a test that runs the TxnSystemClient in an authorized
environment I saw issues where the TxnSystemClient could not
be initialized due to being unauthorized. It turns out that there
are many remote methods that the service user is not authorized
to call. This patch fixes the issue by updating the various required
calls to also allow the service user. I also enhanced a test to leverage
transactions in security-itest.cc
Here is a sample error I saw before this change:
W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt
to method kudu.master.MasterService.ConnectToMaster from {username='kudu',
principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639
W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize
TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the
cluster: Not authorized: unauthorized access to method: ConnectToMaster
Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0
---
M src/kudu/integration-tests/security-itest.cc
M src/kudu/master/master.proto
2 files changed, 28 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/17490/3
--
To view, visit http://gerrit.cloudera.org:8080/17490
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0
Gerrit-Change-Number: 17490
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [txns] Authorization error when running transactional workload in secure tests
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17490 to look at the new patch set (#2). Change subject: [txns] Authorization error when running transactional workload in secure tests .. [txns] Authorization error when running transactional workload in secure tests This is a small patch to show a test issue I have seen when trying to write a test for another patch. Pushing for visibility, will update the message once the issue is identified and a fix is applied. Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 --- M src/kudu/integration-tests/security-itest.cc M src/kudu/mini-cluster/external_mini_cluster.cc 2 files changed, 23 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/17490/2 -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR](branch-1.15.x) [java] utility to detect sanitizer type used for kudu CLI
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17492 ) Change subject: [java] utility to detect sanitizer type used for kudu CLI .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I08c978ecda7321038b97c5c885914c02122606c1 Gerrit-Change-Number: 17492 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sun, 23 May 2021 23:33:57 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) KUDU-3277 fix concurrent session flush issue in Java client
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17493 ) Change subject: KUDU-3277 fix concurrent session flush issue in Java client .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9 Gerrit-Change-Number: 17493 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sun, 23 May 2021 23:34:07 + Gerrit-HasComments: No
[kudu-CR] [java] utility to detect sanitizer type used for kudu CLI
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17487 ) Change subject: [java] utility to detect sanitizer type used for kudu CLI .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17487 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I08c978ecda7321038b97c5c885914c02122606c1 Gerrit-Change-Number: 17487 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sun, 23 May 2021 22:04:37 + Gerrit-HasComments: No
[kudu-CR] KUDU-3277 fix concurrent session flush issue in Java client
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17486 ) Change subject: KUDU-3277 fix concurrent session flush issue in Java client .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17486 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9 Gerrit-Change-Number: 17486 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sun, 23 May 2021 22:03:10 + Gerrit-HasComments: No
[kudu-CR] KUDU-3277 fix concurrent session flush issue in Java client
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17486 ) Change subject: KUDU-3277 fix concurrent session flush issue in Java client .. Patch Set 2: I can be convinced otherwise. My gut is that unless there is something that makes running under TSAN impossible/untenable, we should try to run the tests across all of the build types. Looking at the 4 examples you mention, that is why they 3/4 are disabled. TestReadWriteLargeStrings is the only one disabled due to running slowly in TSAN. I am okay with this approach if making the test run well under TSAN isn't feasible. -- To view, visit http://gerrit.cloudera.org:8080/17486 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9 Gerrit-Change-Number: 17486 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sun, 23 May 2021 22:03:05 + Gerrit-HasComments: No
[kudu-CR] [txns] Authorization error when running transactional workload in secure tests
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17490 Change subject: [txns] Authorization error when running transactional workload in secure tests .. [txns] Authorization error when running transactional workload in secure tests This is a small patch to show a test issue I have seen when trying to write a test for another patch. Pushing for visibility, will update the message once the issue is identified and a fix is applied. Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 --- M src/kudu/integration-tests/security-itest.cc 1 file changed, 22 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/17490/1 -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-3277 fix concurrent session flush issue in Java client
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17486 ) Change subject: KUDU-3277 fix concurrent session flush issue in Java client .. Patch Set 2: I am not sure I like the idea of skipping tests under certain test run types. Can we reduce the work the test is doing and still reliably reproduce so that the test doesn't timeout in tsan? Reducing numSessions and numRowsPerSession will likely help. -- To view, visit http://gerrit.cloudera.org:8080/17486 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9 Gerrit-Change-Number: 17486 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Sun, 23 May 2021 16:59:27 + Gerrit-HasComments: No
[kudu-CR] [java] KUDU-3287: Improve the Java client close behavior
Hello Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17488 to look at the new patch set (#2). Change subject: [java] KUDU-3287: Improve the Java client close behavior .. [java] KUDU-3287: Improve the Java client close behavior This patch improves the Java client close behavior to prevent resources that live for some time after close() has returned. It does this by blocking ReleaseResourcesCB on shutdownGracefully. It also ensures the internal ExecutorService is shutdown. Change-Id: I1efdb459881bb8e52db18dc33abcece63b8e8030 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/KuduTestHarness.java 4 files changed, 79 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/88/17488/2 -- To view, visit http://gerrit.cloudera.org:8080/17488 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1efdb459881bb8e52db18dc33abcece63b8e8030 Gerrit-Change-Number: 17488 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [java] KUDU-3287: Improve the Java client close behavior
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17488 Change subject: [java] KUDU-3287: Improve the Java client close behavior .. [java] KUDU-3287: Improve the Java client close behavior This patch improves the Java client close behavior to prevent resources that live for some time after close() has returned. It does this by blocking ReleaseResourcesCB on shutdownGracefully. It also ensures the internal ExecutorService is shutdown. Change-Id: I1efdb459881bb8e52db18dc33abcece63b8e8030 --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduClient.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestAsyncKuduClient.java M java/kudu-test-utils/src/main/java/org/apache/kudu/test/KuduTestHarness.java 4 files changed, 81 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/88/17488/1 -- To view, visit http://gerrit.cloudera.org:8080/17488 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I1efdb459881bb8e52db18dc33abcece63b8e8030 Gerrit-Change-Number: 17488 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-3277 fix concurrent session flush issue in Java client
Grant Henke has posted comments on this change. (
http://gerrit.cloudera.org:8080/17486 )
Change subject: KUDU-3277 fix concurrent session flush issue in Java client
..
Patch Set 1:
(3 comments)
Looks like the test failed in TSAN. The workload might be a little too intense
and the slower TSAN environment times out.
1) testConcurrentFlush(org.apache.kudu.client.TestKuduClient)
org.junit.runners.model.TestTimedOutException: test timed out after 10
milliseconds
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at
org.apache.kudu.client.TestKuduClient.testConcurrentFlush(TestKuduClient.java:1530)
http://gerrit.cloudera.org:8080/#/c/17486/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java
File
java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java:
http://gerrit.cloudera.org:8080/#/c/17486/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@894
PS1, Line 894: if (injectLatencyBufferFlushCb) {
: try {
: Thread.sleep(randomizer.nextInt(16));
: } catch (InterruptedException e) {
: Thread.currentThread().interrupt();
: }
: }
> I would be happy to have this only in 'debug' mode, but I didn't find a way
Java does't really have multiple binary types. This approach makes sense.
http://gerrit.cloudera.org:8080/#/c/17486/1/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java@1005
PS1, Line 1005: injectLatencyBufferFlushCb = injectLatency;
Maybe log a warning here so it's easy to tell from the logs that latency will
be injected just in case.
http://gerrit.cloudera.org:8080/#/c/17486/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java
File java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java:
http://gerrit.cloudera.org:8080/#/c/17486/1/java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java@1468
PS1, Line 1468: 501
Is this specific value particularly significant? Looks like it's just over half
the default of 1000.
--
To view, visit http://gerrit.cloudera.org:8080/17486
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9
Gerrit-Change-Number: 17486
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Grant Henke
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 22 May 2021 13:35:51 +
Gerrit-HasComments: Yes
[kudu-CR] WIP: KUDU-1884: Set sasl protocol name for the TxnSystemClient
Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17480 to look at the new patch set (#3). Change subject: WIP: KUDU-1884: Set sasl_protocol_name for the TxnSystemClient .. WIP: KUDU-1884: Set sasl_protocol_name for the TxnSystemClient In clusters witha custom principal we need to set the sasl_protocol_name on the embeded client within the TxnSystemClient. This logic matches that in server_base.cc for server messenger configuration. WIP: Because of testing challenges. TODO: Remove all the hard coded proto name calls Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf --- M src/kudu/integration-tests/auth_token_expire-itest.cc M src/kudu/integration-tests/client-negotiation-failover-itest.cc M src/kudu/integration-tests/master_authz-itest.cc M src/kudu/integration-tests/master_hms-itest.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/txn_commit-itest.cc M src/kudu/integration-tests/txn_participant-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/master/txn_manager.cc M src/kudu/tools/tool_action_txn.cc M src/kudu/transactions/txn_system_client.cc M src/kudu/transactions/txn_system_client.h 12 files changed, 64 insertions(+), 40 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/17480/3 -- To view, visit http://gerrit.cloudera.org:8080/17480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf Gerrit-Change-Number: 17480 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120)
