[kudu-CR] KUDU-1884: set sasl protocol name for the TxnSystemClient
Alexey Serbin has uploaded a new patch set (#5) to the change originally created by Grant Henke. ( http://gerrit.cloudera.org:8080/17480 ) Change subject: KUDU-1884: set sasl_protocol_name for the TxnSystemClient .. KUDU-1884: set sasl_protocol_name for the TxnSystemClient In clusters with a custom principal we need to set the sasl_protocol_name on the embedded client within the TxnSystemClient. This logic matches that in server_base.cc for server messenger configuration. Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf --- M src/kudu/integration-tests/auth_token_expire-itest.cc M src/kudu/integration-tests/client-negotiation-failover-itest.cc M src/kudu/integration-tests/master_authz-itest.cc M src/kudu/integration-tests/master_hms-itest.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/txn_commit-itest.cc M src/kudu/integration-tests/txn_participant-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/master/txn_manager.cc M src/kudu/mini-cluster/external_mini_cluster.h M src/kudu/tools/tool_action_txn.cc M src/kudu/transactions/txn_system_client.cc M src/kudu/transactions/txn_system_client.h 13 files changed, 141 insertions(+), 61 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/17480/5 -- To view, visit http://gerrit.cloudera.org:8080/17480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf Gerrit-Change-Number: 17480 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2612: disable TxnSystemClient initialization by default
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17499 ) Change subject: KUDU-2612: disable TxnSystemClient initialization by default .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/17499/1/src/kudu/transactions/txn_system_client.cc File src/kudu/transactions/txn_system_client.cc: http://gerrit.cloudera.org:8080/#/c/17499/1/src/kudu/transactions/txn_system_client.cc@58 PS1, Line 58: enable_txn_system_client_init, > If we are going this route and this flag starts to appear not only in tests Makes sense. Done -- To view, visit http://gerrit.cloudera.org:8080/17499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b Gerrit-Change-Number: 17499 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 25 May 2021 05:22:06 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-2612: disable TxnSystemClient initialization by default
Hello Alexey Serbin, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17499 to look at the new patch set (#2). Change subject: KUDU-2612: disable TxnSystemClient initialization by default .. KUDU-2612: disable TxnSystemClient initialization by default Currently the TxnSystemClient gets initialized on tablet servers by default, retrying every second until successful, and logging an error every minute if unable to. Since transactions support is currently experimental and disabled by default, so too should this initialization be. This patch hides the initialization with the existing --disable_txn_system_client_init flag (previously used for tests), adjusted to --enable_txn_system_client_init to match most of our feature-gating flags. Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b --- M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTransaction.java M src/kudu/client/client-test.cc M src/kudu/integration-tests/client-negotiation-failover-itest.cc M src/kudu/integration-tests/location_assignment-itest.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/ts_tablet_manager-itest.cc M src/kudu/integration-tests/txn_commit-itest.cc M src/kudu/integration-tests/txn_status_manager-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/integration-tests/txn_write_ops-itest.cc M src/kudu/tools/kudu-txn-cli-test.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/transactions/txn_system_client.cc M src/kudu/tserver/tablet_service.cc 14 files changed, 143 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/99/17499/2 -- To view, visit http://gerrit.cloudera.org:8080/17499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b Gerrit-Change-Number: 17499 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2612: disable TxnSystemClient initialization by default
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17499 ) Change subject: KUDU-2612: disable TxnSystemClient initialization by default .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/17499/1/src/kudu/transactions/txn_system_client.cc File src/kudu/transactions/txn_system_client.cc: http://gerrit.cloudera.org:8080/#/c/17499/1/src/kudu/transactions/txn_system_client.cc@58 PS1, Line 58: disable_txn_system_client_init If we are going this route and this flag starts to appear not only in tests, maybe it's a good point to rename it to be similar to the majority of the flags with 'enable/disable' semantics? What do you think about renaming this into 'txn_system_client_enable' or alike? I guess the main thing is to have 'enable_something' instead of 'disable_something' -- that's a bit easier to read and understand. -- To view, visit http://gerrit.cloudera.org:8080/17499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b Gerrit-Change-Number: 17499 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 25 May 2021 01:29:21 + Gerrit-HasComments: Yes
[kudu-CR] WIP [master] KUDU-2181 Fix duplicate master address and remove unsafe flag tag
Bankim Bhavsar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17500 Change subject: WIP [master] KUDU-2181 Fix duplicate master address and remove unsafe flag tag .. WIP [master] KUDU-2181 Fix duplicate master address and remove unsafe flag tag While testing add master on a physical cluster, observed that supplying only hostnames without port resulted in duplicates being supplied to bring up new master which in turn leads to failure in creating distributed Raft config on startup. For e.g. kudu master add hp1,hp2 h2 Reason being hp2 is compared as hp2:7051 whereas in the vector of strings "master_addresses", it contains hp1,hp2. This changes adds a new function ParseAddresses() in HostPort class that's a variant of existing ParseStrings() function and takes a vector of strings instead. This new function is used in the duplicate detection logic. This change also removes the unsafe tag from --master_addr_add_new_master as the feature is ready but keeps it hidden as it's only meant to be used by the add master orchestration tool. WIP because of missing test. With UNIQUE_LOOPBACK on Linux, if masters are assigned same default port then I can skip supplying port and simulate the issue. Without UNIQUE_LOOPBACK on Mac, I'll have to disable the test. Change-Id: Icf29730e3a6b225adb24ff161cac2ad777b46b81 --- M src/kudu/master/sys_catalog.cc M src/kudu/tools/tool_action_master.cc M src/kudu/util/net/net_util.cc M src/kudu/util/net/net_util.h 4 files changed, 27 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/00/17500/1 -- To view, visit http://gerrit.cloudera.org:8080/17500 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Icf29730e3a6b225adb24ff161cac2ad777b46b81 Gerrit-Change-Number: 17500 Gerrit-PatchSet: 1 Gerrit-Owner: Bankim Bhavsar
[kudu-CR] KUDU-2612: disable TxnSystemClient initialization by default
Andrew Wong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17499 Change subject: KUDU-2612: disable TxnSystemClient initialization by default .. KUDU-2612: disable TxnSystemClient initialization by default Currently the TxnSystemClient gets initialized on tablet servers by default, retrying every second until successful, and logging an error every minute if unable to. Since transactions support is currently experimental and disabled by default, so too should this initialization be. This patch hides the initialization with the existing --disable_txn_system_client_init flag (previously used for tests), and sets its default value to true. Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b --- M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTransaction.java M src/kudu/integration-tests/txn_commit-itest.cc M src/kudu/integration-tests/txn_status_manager-itest.cc M src/kudu/integration-tests/txn_write_ops-itest.cc M src/kudu/transactions/txn_status_manager-test.cc M src/kudu/transactions/txn_system_client.cc M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/tablet_service.cc 8 files changed, 86 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/99/17499/1 -- To view, visit http://gerrit.cloudera.org:8080/17499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7c020a66db484f88ae1cb7c15d860d503a3f8a3b Gerrit-Change-Number: 17499 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong
[kudu-CR] [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu sid
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17388 ) Change subject: [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this p .. Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/17388/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17388/6//COMMIT_MSG@7 PS6, Line 7: Add tablePrefix option to the Kudu restore job nit: this should be updated. Maybe: "[backup] KUDU-3183: add --newDatabaseName and --removeImpalaPrefix options to restore job" -- To view, visit http://gerrit.cloudera.org:8080/17388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17388 Gerrit-PatchSet: 6 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 22:30:50 + Gerrit-HasComments: Yes
[kudu-CR] [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu sid
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17388 ) Change subject: [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this p .. Patch Set 6: Code-Review+1 (1 comment) http://gerrit.cloudera.org:8080/#/c/17388/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17388/6//COMMIT_MSG@7 PS6, Line 7: nit: remove this space, add a newline -- To view, visit http://gerrit.cloudera.org:8080/17388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17388 Gerrit-PatchSet: 6 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 22:30:01 + Gerrit-HasComments: Yes
[kudu-CR] WIP: KUDU-1884: Set sasl protocol name for the TxnSystemClient
Alexey Serbin has uploaded a new patch set (#4) to the change originally created by Grant Henke. ( http://gerrit.cloudera.org:8080/17480 ) Change subject: WIP: KUDU-1884: Set sasl_protocol_name for the TxnSystemClient .. WIP: KUDU-1884: Set sasl_protocol_name for the TxnSystemClient In clusters witha custom principal we need to set the sasl_protocol_name on the embeded client within the TxnSystemClient. This logic matches that in server_base.cc for server messenger configuration. WIP: Because of testing challenges. TODO: Remove all the hard coded proto name calls Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf --- M src/kudu/integration-tests/auth_token_expire-itest.cc M src/kudu/integration-tests/client-negotiation-failover-itest.cc M src/kudu/integration-tests/master_authz-itest.cc M src/kudu/integration-tests/master_hms-itest.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/txn_commit-itest.cc M src/kudu/integration-tests/txn_participant-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/master/txn_manager.cc M src/kudu/tools/tool_action_txn.cc M src/kudu/transactions/txn_system_client.cc M src/kudu/transactions/txn_system_client.h 12 files changed, 47 insertions(+), 37 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/17480/4 -- To view, visit http://gerrit.cloudera.org:8080/17480 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic7b7c7fd154796215bd3a3ce748a2fd3155adebf Gerrit-Change-Number: 17480 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR](branch-1.15.x) [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17498 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 2: > Build Failed > > http://jenkins.kudu.apache.org/job/kudu-gerrit/23917/ : ABORTED Oops, I didn't realize Grant already back-ported and posted this fore review. -- To view, visit http://gerrit.cloudera.org:8080/17498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17498 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 22:01:23 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17495 ) Change subject: [kserver] output info on server-wide size limit .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 21:47:37 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17495 ) Change subject: [kserver] output info on server-wide size limit .. Patch Set 1: > > Patch Set 1: Verified-1 > > > > Build Failed > > > > http://jenkins.kudu.apache.org/job/kudu-gerrit/23911/ : FAILURE > > Are we sure the test failures are not related? Those test failures are not related, as far as I can see. -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 21:47:31 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has uploaded a new patch set (#2) to the change originally created by Grant Henke. ( http://gerrit.cloudera.org:8080/17498 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. [txns] Allow the TxnSystemClient to run in Authorized environments When writing a test that runs the TxnSystemClient in an authorized environment I saw issues where the TxnSystemClient could not be initialized due to being unauthorized. It turns out that there are many remote methods that the service user is not authorized to call. This patch fixes the issue by updating the various required calls to also allow the service user. I also enhanced a test to leverage transactions in security-itest.cc Here is a sample error I saw before this change: W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='kudu', principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639 W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the cluster: Not authorized: unauthorized access to method: ConnectToMaster Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Reviewed-on: http://gerrit.cloudera.org:8080/17490 Tested-by: Alexey Serbin Reviewed-by: Grant Henke (cherry picked from commit 35de43051b65c8eab0f348bfd7c8a0d6e49c8067) --- M src/kudu/client/client-test.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/master/master.proto 4 files changed, 64 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/17498/2 -- To view, visit http://gerrit.cloudera.org:8080/17498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17498 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [txns] Allow the TxnSystemClient to run in Authorized environments
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17498 Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. [txns] Allow the TxnSystemClient to run in Authorized environments When writing a test that runs the TxnSystemClient in an authorized environment I saw issues where the TxnSystemClient could not be initialized due to being unauthorized. It turns out that there are many remote methods that the service user is not authorized to call. This patch fixes the issue by updating the various required calls to also allow the service user. I also enhanced a test to leverage transactions in security-itest.cc Here is a sample error I saw before this change: W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='kudu', principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639 W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the cluster: Not authorized: unauthorized access to method: ConnectToMaster Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Reviewed-on: http://gerrit.cloudera.org:8080/17490 Tested-by: Alexey Serbin Reviewed-by: Grant Henke (cherry picked from commit 35de43051b65c8eab0f348bfd7c8a0d6e49c8067) --- M src/kudu/client/client-test.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/master/master.proto 4 files changed, 64 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/17498/1 -- To view, visit http://gerrit.cloudera.org:8080/17498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17498 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. [txns] Allow the TxnSystemClient to run in Authorized environments When writing a test that runs the TxnSystemClient in an authorized environment I saw issues where the TxnSystemClient could not be initialized due to being unauthorized. It turns out that there are many remote methods that the service user is not authorized to call. This patch fixes the issue by updating the various required calls to also allow the service user. I also enhanced a test to leverage transactions in security-itest.cc Here is a sample error I saw before this change: W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='kudu', principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639 W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the cluster: Not authorized: unauthorized access to method: ConnectToMaster Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Reviewed-on: http://gerrit.cloudera.org:8080/17490 Tested-by: Alexey Serbin Reviewed-by: Grant Henke --- M src/kudu/client/client-test.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/master/master.proto 4 files changed, 64 insertions(+), 20 deletions(-) Approvals: Alexey Serbin: Verified Grant Henke: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 5 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17495 ) Change subject: [kserver] output info on server-wide size limit .. Patch Set 1: yes, this is a clean backport which shouldn't introduce test issues. I reran and the run passed. -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 21:33:13 + Gerrit-HasComments: No
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 21:33:39 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [security] fix const-correctness of some ACL methods
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17497 ) Change subject: [security] fix const-correctness of some ACL methods .. [security] fix const-correctness of some ACL methods This patch fixes a few methods in simple_acl and server_base to be const-correct. This patch doesn't introduce any functional changes. Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Reviewed-on: http://gerrit.cloudera.org:8080/17496 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin (cherry picked from commit 68f0f1a1d1d9beaff60243b77b64f4b1b359702c) Reviewed-on: http://gerrit.cloudera.org:8080/17497 Tested-by: Kudu Jenkins --- M src/kudu/security/simple_acl.cc M src/kudu/security/simple_acl.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h 4 files changed, 9 insertions(+), 10 deletions(-) Approvals: Bankim Bhavsar: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/17497 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17497 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has removed a vote on this change. Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 4: Verified+1 unrelated test failure in DeleteTableITest.TestAutoTombstoneAfterCrashDuringTabletCopy -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 21:16:54 + Gerrit-HasComments: No
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has uploaded a new patch set (#4) to the change originally created by Grant Henke. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. [txns] Allow the TxnSystemClient to run in Authorized environments When writing a test that runs the TxnSystemClient in an authorized environment I saw issues where the TxnSystemClient could not be initialized due to being unauthorized. It turns out that there are many remote methods that the service user is not authorized to call. This patch fixes the issue by updating the various required calls to also allow the service user. I also enhanced a test to leverage transactions in security-itest.cc Here is a sample error I saw before this change: W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='kudu', principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639 W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the cluster: Not authorized: unauthorized access to method: ConnectToMaster Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 --- M src/kudu/client/client-test.cc M src/kudu/integration-tests/security-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/master/master.proto 4 files changed, 64 insertions(+), 20 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/17490/4 -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 4 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR](branch-1.15.x) [java] a property to show output while running a test
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17494 ) Change subject: [java] a property to show output while running a test .. [java] a property to show output while running a test While working on non-trivial and relatively long running test scenarios, I found it's inconvenient that the test output isn't shown while a test scenario is running. This patch introduces a new command-line property 'showTestOutput' to direct gradle outputting stdout and stderr from the JVM on the console, so it's possible to do the standard stream redirection and other stream manipulation in a command line shell. An example of running a particular test scenario with the output redirected to the console: ./gradlew -DshowTestOutput cleanTest :kudu-client:test --test ... By default the property isn't set, so this patch preserves the original behavior w.r.t. the output of running Java tests. Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Reviewed-on: http://gerrit.cloudera.org:8080/17479 Tested-by: Kudu Jenkins Reviewed-by: Bankim Bhavsar Reviewed-by: Andrew Wong (cherry picked from commit 7cdc9f7c735efe99b2d39d61ac4034d41473522e) Reviewed-on: http://gerrit.cloudera.org:8080/17494 --- M java/gradle/tests.gradle 1 file changed, 4 insertions(+), 0 deletions(-) Approvals: Kudu Jenkins: Verified Bankim Bhavsar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17494 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Gerrit-Change-Number: 17494 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [security] fix const-correctness of some ACL methods
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17497 ) Change subject: [security] fix const-correctness of some ACL methods .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17497 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17497 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 19:03:31 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17495 ) Change subject: [kserver] output info on server-wide size limit .. Patch Set 1: > Patch Set 1: Verified-1 > > Build Failed > > http://jenkins.kudu.apache.org/job/kudu-gerrit/23911/ : FAILURE Are we sure the test failures are not related? -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 19:03:18 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [java] a property to show output while running a test
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17494 ) Change subject: [java] a property to show output while running a test .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17494 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Gerrit-Change-Number: 17494 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 19:02:41 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [security] fix const-correctness of some ACL methods
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17497 Change subject: [security] fix const-correctness of some ACL methods .. [security] fix const-correctness of some ACL methods This patch fixes a few methods in simple_acl and server_base to be const-correct. This patch doesn't introduce any functional changes. Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Reviewed-on: http://gerrit.cloudera.org:8080/17496 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin (cherry picked from commit 68f0f1a1d1d9beaff60243b77b64f4b1b359702c) --- M src/kudu/security/simple_acl.cc M src/kudu/security/simple_acl.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h 4 files changed, 9 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/97/17497/1 -- To view, visit http://gerrit.cloudera.org:8080/17497 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17497 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [security] fix const-correctness of some ACL methods
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17496 ) Change subject: [security] fix const-correctness of some ACL methods .. [security] fix const-correctness of some ACL methods This patch fixes a few methods in simple_acl and server_base to be const-correct. This patch doesn't introduce any functional changes. Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Reviewed-on: http://gerrit.cloudera.org:8080/17496 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin --- M src/kudu/security/simple_acl.cc M src/kudu/security/simple_acl.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h 4 files changed, 9 insertions(+), 10 deletions(-) Approvals: Bankim Bhavsar: Looks good to me, approved Alexey Serbin: Verified -- To view, visit http://gerrit.cloudera.org:8080/17496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17496 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [security] fix const-correctness of some ACL methods
Alexey Serbin has removed a vote on this change. Change subject: [security] fix const-correctness of some ACL methods .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/17496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17496 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [security] fix const-correctness of some ACL methods
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17496 ) Change subject: [security] fix const-correctness of some ACL methods .. Patch Set 1: Verified+1 unrelated test failure: * ToolTest.TestReplaceTablet -- To view, visit http://gerrit.cloudera.org:8080/17496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17496 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 18:45:05 + Gerrit-HasComments: No
[kudu-CR] [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls
Alexey Serbin has removed a vote on this change. Change subject: [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/17491 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: Id4ff9cd8943c780d4fe8bf6afb2ef540c07a5c8d Gerrit-Change-Number: 17491 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17491 ) Change subject: [tests] TsRecoveryITestDeathTest isn't for for TSAN buidls .. Patch Set 1: Verified+1 unrelated test failures: * TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted * testTxnKeepaliveSwitchesToOtherTxnManager(org.apache.kudu.client.TestKuduTransaction) -- To view, visit http://gerrit.cloudera.org:8080/17491 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id4ff9cd8943c780d4fe8bf6afb2ef540c07a5c8d Gerrit-Change-Number: 17491 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 18:38:29 + Gerrit-HasComments: No
[kudu-CR] [security] fix const-correctness of some ACL methods
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17496 ) Change subject: [security] fix const-correctness of some ACL methods .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17496 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 18:18:38 + Gerrit-HasComments: No
[kudu-CR] [security] fix const-correctness of some ACL methods
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17496 Change subject: [security] fix const-correctness of some ACL methods .. [security] fix const-correctness of some ACL methods This patch fixes a few methods in simple_acl and server_base to be const-correct. This patch doesn't introduce any functional changes. Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d --- M src/kudu/security/simple_acl.cc M src/kudu/security/simple_acl.h M src/kudu/server/server_base.cc M src/kudu/server/server_base.h 4 files changed, 9 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/96/17496/1 -- To view, visit http://gerrit.cloudera.org:8080/17496 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I0052528865c877354d5fdfa2e069a6daf716336d Gerrit-Change-Number: 17496 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 3: > Thank you for fixing this. > > I guess we will also need to make sure TxnManager is able to create > new ranges and write into tables/tablets. I'll take a look at that > -- we can add corresponding permissions for methods in > TabletServerService in a separate changelist. I took a look at that and it seems TxnManager uses regular MasterService::AlterTable() RPC which already has "AuthorizeClientOrServiceUser" -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 16:58:41 + Gerrit-HasComments: No
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 3: Looks like I need to fix up a few tests that depended on the bad behavior -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 16:50:52 + Gerrit-HasComments: No
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 3: Code-Review+2 Thank you for fixing this. I guess we will also need to make sure TxnManager is able to create new ranges and write into tables/tablets. I'll take a look at that -- we can add corresponding permissions for methods in TabletServerService in a separate changelist. -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 16:44:38 + Gerrit-HasComments: No
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17490 ) Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. Patch Set 3: Code-Review+2 Nice find! -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Mon, 24 May 2021 16:42:26 + Gerrit-HasComments: No
[kudu-CR](branch-1.15.x) [kserver] output info on server-wide size limit
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17495 Change subject: [kserver] output info on server-wide size limit .. [kserver] output info on server-wide size limit With this patch, Kudu servers output an INFO log line about the maximum number of threads in a server-wide thread pool. That's useful when assessing how many threads a Kudu server might legitimately spawn. Yes, those numbers can be calculated given the setting for RLIMIT_NPROC and values in /proc/sys/kernel/pid_max and /proc/sys/kernel/threads-max, but (a) those settings might change after the start of a server and (b) it's convenient to have a pre-computed number ready in the logs. Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Reviewed-on: http://gerrit.cloudera.org:8080/17474 Reviewed-by: Bankim Bhavsar Tested-by: Alexey Serbin (cherry picked from commit 48068c7cbf4066e3d2081960b50d466baf01543d) --- M src/kudu/kserver/kserver.cc 1 file changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/17495/1 -- To view, visit http://gerrit.cloudera.org:8080/17495 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I62b4168fc153e693f0fcc3d6d1ecfc6be0e69b47 Gerrit-Change-Number: 17495 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR](branch-1.15.x) [java] a property to show output while running a test
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17494 Change subject: [java] a property to show output while running a test .. [java] a property to show output while running a test While working on non-trivial and relatively long running test scenarios, I found it's inconvenient that the test output isn't shown while a test scenario is running. This patch introduces a new command-line property 'showTestOutput' to direct gradle outputting stdout and stderr from the JVM on the console, so it's possible to do the standard stream redirection and other stream manipulation in a command line shell. An example of running a particular test scenario with the output redirected to the console: ./gradlew -DshowTestOutput cleanTest :kudu-client:test --test ... By default the property isn't set, so this patch preserves the original behavior w.r.t. the output of running Java tests. Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Reviewed-on: http://gerrit.cloudera.org:8080/17479 Tested-by: Kudu Jenkins Reviewed-by: Bankim Bhavsar Reviewed-by: Andrew Wong (cherry picked from commit 7cdc9f7c735efe99b2d39d61ac4034d41473522e) --- M java/gradle/tests.gradle 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/17494/1 -- To view, visit http://gerrit.cloudera.org:8080/17494 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: newchange Gerrit-Change-Id: I82590957004eef427a19f36a32e903012ea220d4 Gerrit-Change-Number: 17494 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] [txns] Allow the TxnSystemClient to run in Authorized environments
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17490 to look at the new patch set (#3). Change subject: [txns] Allow the TxnSystemClient to run in Authorized environments .. [txns] Allow the TxnSystemClient to run in Authorized environments When writing a test that runs the TxnSystemClient in an authorized environment I saw issues where the TxnSystemClient could not be initialized due to being unauthorized. It turns out that there are many remote methods that the service user is not authorized to call. This patch fixes the issue by updating the various required calls to also allow the service user. I also enhanced a test to leverage transactions in security-itest.cc Here is a sample error I saw before this change: W0524 08:57:12.618356 216932352 server_base.cc:694] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='kudu', principal='kudu/127.0@krbtest.com'} at 127.0.0.1:51639 W0524 08:57:12.618837 94556160 txn_system_client.cc:478] unable to initialize TxnSystemClient, will retry in 1.000s: Remote error: Could not connect to the cluster: Not authorized: unauthorized access to method: ConnectToMaster Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 --- M src/kudu/integration-tests/security-itest.cc M src/kudu/master/master.proto 2 files changed, 28 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/17490/3 -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR](branch-1.15.x) KUDU-3277 fix concurrent session flush issue in Java client
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17493 ) Change subject: KUDU-3277 fix concurrent session flush issue in Java client .. KUDU-3277 fix concurrent session flush issue in Java client This patch fixes the issue reported by KUDU-3277. The bug manifested itself in rare cases when a session running in the AUTO_FLUSH_BACKGROUND mode was either being flushed explicitly or implicitly (i.e. upon closing the session) while the AUTO_FLUSH_BACKGROUND session's logic was flushing its data buffers concurrently as well. This patch also adds a reproduction scenario for KUDU-3277. The newly introduced test scenario was reliably failing before the fix: * 'java.lang.AssertionError: This Deferred was already called' messages were encountered in the log multiple times with the stack exactly as described in KUDU-3277 * some flusher threads were unable to join since KuduSession.flush() would hang (i.e. would not return) Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9 Reviewed-on: http://gerrit.cloudera.org:8080/17486 Tested-by: Alexey Serbin Reviewed-by: Grant Henke (cherry picked from commit d1969e2623e30bfd62f2463f15e0bf02422d1d84) Reviewed-on: http://gerrit.cloudera.org:8080/17493 Reviewed-by: Bankim Bhavsar --- M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduClient.java 2 files changed, 170 insertions(+), 8 deletions(-) Approvals: Grant Henke: Looks good to me, approved Alexey Serbin: Verified Bankim Bhavsar: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/17493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9 Gerrit-Change-Number: 17493 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) [java] utility to detect sanitizer type used for kudu CLI
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17492 ) Change subject: [java] utility to detect sanitizer type used for kudu CLI .. [java] utility to detect sanitizer type used for kudu CLI This patch adds a new test utility to determine the type of the C++ sanitizer type used by the kudu CLI binary, if any. The new utility is added into the kudu-test-utils package. I found it's useful to find whether the mini-cluster binaries are built with TSAN/ASAN sanitizers, so it's possible to skip heavy test scenarios which are not stable when running against binaries built with ASAN/TSAN support. I didn't add any tests, but this functionality is going to be used in a follow-up patch. Change-Id: I08c978ecda7321038b97c5c885914c02122606c1 Reviewed-on: http://gerrit.cloudera.org:8080/17487 Tested-by: Alexey Serbin Reviewed-by: Grant Henke (cherry picked from commit 6a0752420fdf46b821038d2f789cc9742a2ac864) Reviewed-on: http://gerrit.cloudera.org:8080/17492 --- M java/kudu-test-utils/src/main/java/org/apache/kudu/test/cluster/KuduBinaryInfo.java 1 file changed, 80 insertions(+), 0 deletions(-) Approvals: Grant Henke: Looks good to me, approved Alexey Serbin: Verified -- To view, visit http://gerrit.cloudera.org:8080/17492 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: merged Gerrit-Change-Id: I08c978ecda7321038b97c5c885914c02122606c1 Gerrit-Change-Number: 17492 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [txns] Authorization error when running transactional workload in secure tests
Hello Tidy Bot, Kudu Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17490 to look at the new patch set (#2). Change subject: [txns] Authorization error when running transactional workload in secure tests .. [txns] Authorization error when running transactional workload in secure tests This is a small patch to show a test issue I have seen when trying to write a test for another patch. Pushing for visibility, will update the message once the issue is identified and a fix is applied. Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 --- M src/kudu/integration-tests/security-itest.cc M src/kudu/mini-cluster/external_mini_cluster.cc 2 files changed, 23 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/90/17490/2 -- To view, visit http://gerrit.cloudera.org:8080/17490 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4622586fc27b5b67005bf023e4fbaebaf5454ad0 Gerrit-Change-Number: 17490 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu sid
Abhishek Chennaka has posted comments on this change. ( http://gerrit.cloudera.org:8080/17388 ) Change subject: [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this p .. Patch Set 6: (8 comments) http://gerrit.cloudera.org:8080/#/c/17388/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17388/5//COMMIT_MSG@8 PS5, Line 8: Wh > nit: drop Done http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala File java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala: http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@57 PS5, Line 57: val impalaPrefixLen = if (fullTableName.startsWith("impala::")) 8 else 0 : val indexOfPeriod = fullTableName.indexOf('.') > nit: these can be vals since they're const Done http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@59 PS5, Line 59: indexOfPeriod != -1) { > nit: can use != Done http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@66 PS5, Line 66: newDatabaseName.isEmpty) { > nit: prefer isEmpty(), since it's a cheaper operation. Same below. Done http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@73 PS5, Line 73: impalaPrefixLen != 0 && (!options.re > nit: prefer impalaPrefixLen != 0, since it's a cheaper operation. Done http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala@72 PS5, Line 72: // Build the table name which is being restored and return : if (impalaPrefixLen != 0 && (!options.removeImpalaPrefix)) { : return s"impala::${databaseName}${onlyTableName}${options.tableSuffix}" : } : return s"${databas > nit: rather than storing a variable, how about something like: Done http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala File java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala: http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala@839 PS5, Line 839: if (!options.newDatabaseName.isEmpty()) { : databaseName = options.newDatabaseName : } : if (!(databaseName.isEmpty())) { : databaseName = databaseName.concat(".") : } > nit: same here Done http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestOptions.scala File java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestOptions.scala: http://gerrit.cloudera.org:8080/#/c/17388/5/java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestOptions.scala@57 PS5, Line 57: K > nit: capitalize Kudu Done -- To view, visit http://gerrit.cloudera.org:8080/17388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17388 Gerrit-PatchSet: 6 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 14:14:07 + Gerrit-HasComments: Yes
[kudu-CR] [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu sid
Hello Alexey Serbin, Kudu Jenkins, Andrew Wong, Grant Henke, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17388 to look at the new patch set (#6). Change subject: [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this p .. [backup] KUDU-3183 Add tablePrefix option to the Kudu restore job While Kudu does not have a notion of database, usually the full table name is stored as . on kudu side. (NOTE: database name is optional). Using the options in this patch users can change the existing database name of the table or add a new database name to the table i.e. the prefix to the '.' in the full tablename as well as remove impala prefix for the tables which are being restored. 1.--newDatabaseName : Use this option to specify the new database name for the restored table. This will overwrite any existing database and if there is no existing database, a new database will be added to the table name. Will not affect "impala::" prefix. E.g: Adding database name "newDB" to the tables impala::default.test, impala::test, default.test, test will result in the table names impala::newDB.default.test, impala::newDB.test, newDB.test, newDBtest. This will not affect the existing/source tables. 2.--removeImpalaPrefix : If enabled, this option will remove the “impala::” prefix, if present from the restored table names. This is advisable if tables are backed up in Kudu clusters without HMS integration and being restored to Kudu clusters with HMS integration. Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 --- M java/kudu-backup/src/main/scala/org/apache/kudu/backup/KuduRestore.scala M java/kudu-backup/src/main/scala/org/apache/kudu/backup/Options.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestKuduBackup.scala M java/kudu-backup/src/test/scala/org/apache/kudu/backup/TestOptions.scala 4 files changed, 144 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/88/17388/6 -- To view, visit http://gerrit.cloudera.org:8080/17388 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I65adcc1b3de0a8e1ac5b7f50a2d3a7036aa69421 Gerrit-Change-Number: 17388 Gerrit-PatchSet: 6 Gerrit-Owner: Abhishek Chennaka Gerrit-Reviewer: Abhishek Chennaka Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](branch-1.15.x) KUDU-3277 fix concurrent session flush issue in Java client
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17493 ) Change subject: KUDU-3277 fix concurrent session flush issue in Java client .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/17493 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.15.x Gerrit-MessageType: comment Gerrit-Change-Id: If6aaccc06abf1a2673620ab7c649f51f91999ad9 Gerrit-Change-Number: 17493 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Mon, 24 May 2021 13:55:51 + Gerrit-HasComments: No