[kudu-CR] KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory
Hello Kudu Jenkins, Andrew Wong, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16991 to look at the new patch set (#2). Change subject: KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory .. KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory This patch fixes a flakiness in the TestCompletedOpsHistory scenario. The flakiness is a test-only issue which became apparent as a result of the recent changes introduced into the MaintenanceManager with 9e4664d44 changelist. In essence, with finer granularity of locking in the scoped cleanup of the MaintenanceManager::LaunchOp() method, the test thread calling MaintenanceManager::GetMaintenanceManagerStatusDump() has a slight chance of acquiring 'completed_ops_lock_' ahead of the thread executing the code in the LaunchOp()'s scoped cleanup. This patch wraps the related code into ASSERT_EVENTUALLY to resolve test-only race condition mentioned above. I verified that this patch fixes the issue by running the test scenario multiple times under dist-test (RELEASE build). Before: 2 out of 256 runs failed http://dist-test.cloudera.org//job?job_id=aserbin.1611806979.74192 After : 0 out of 256 runs failed http://dist-test.cloudera.org//job?job_id=aserbin.1611809676.95320 Change-Id: I760287b3ed4d50e32d2f9257e5390fdf8fa8f288 --- M src/kudu/util/maintenance_manager-test.cc 1 file changed, 21 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/91/16991/2 -- To view, visit http://gerrit.cloudera.org:8080/16991 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I760287b3ed4d50e32d2f9257e5390fdf8fa8f288 Gerrit-Change-Number: 16991 Gerrit-PatchSet: 2 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2612: loosen restrictions for BEGIN COMMIT ops
Andrew Wong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16992 Change subject: KUDU-2612: loosen restrictions for BEGIN_COMMIT ops .. KUDU-2612: loosen restrictions for BEGIN_COMMIT ops This patch updates the state validations for the BEGIN_COMMIT. Rather than only requiring an in-flight transaction to be kOpen (the common case) or kCommitInProgress (if re-attempting a BEGIN_COMMIT call), it's more robust to allow the call to be made even after the transaction has already been finalized. This will allow a to-be-merged implementation of a commit task to be retried with much less fuss. Here's some pseudo-code for such a commit task: commit task: 1. persist COMMIT_IN_PROGRESS record on TxnStatusManager 2. foreach participant as p: BEGIN_COMMIT(p) 3. commit_ts = max timestamp used across BEGIN_COMMIT ops 4. foreach participant as p: FINALIZE_COMMIT(p, commit_ts) 5. persist COMMITTED record on TxnStatusManager If the commit task is interrupted (e.g. by some crash) between steps 3 and 5, we may be left with some participants with fully finalized commits. In such cases, all other participants _must_ also finalize, and they must finalize with the same timestamp. To ensure this, it must be possible to re-run a commit task. However, re-running it without this patch may lead to issues because the BEGIN_COMMIT ops would yield errors, complaining about an illegal state on participants that were finalized. This patch allows for a BEGIN_COMMIT op to succeed and return immediately if a FINALIZE_COMMIT op has already completed. If so, the finalized commit timestamp is sent back, allowing for the above commit task to be repeatable. Change-Id: Ifa4c5314190c84648c1b1edea7aab776b4882f97 --- M src/kudu/integration-tests/txn_participant-itest.cc M src/kudu/tablet/tablet_metadata.cc M src/kudu/tablet/tablet_metadata.h M src/kudu/tablet/txn_participant-test.cc M src/kudu/tablet/txn_participant.h 5 files changed, 119 insertions(+), 19 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/92/16992/1 -- To view, visit http://gerrit.cloudera.org:8080/16992 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ifa4c5314190c84648c1b1edea7aab776b4882f97 Gerrit-Change-Number: 16992 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong
[kudu-CR] KUDU-2612: add a TxnSystemClient to the tservers
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16974 ) Change subject: KUDU-2612: add a TxnSystemClient to the tservers .. Patch Set 5: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc File src/kudu/transactions/txn_system_client.cc: http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@414 PS4, Line 414: } > Yeah, it might be worth making this robust, though I'm going to hold off an yep, this makes sense to me. -- To view, visit http://gerrit.cloudera.org:8080/16974 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I33b5a2bb5c56ae4bb4b42069af5813e2780fc4bc Gerrit-Change-Number: 16974 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 28 Jan 2021 07:49:55 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/16991 ) Change subject: KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory .. Patch Set 1: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/16991/1/src/kudu/util/maintenance_manager-test.cc File src/kudu/util/maintenance_manager-test.cc: http://gerrit.cloudera.org:8080/#/c/16991/1/src/kudu/util/maintenance_manager-test.cc@587 PS1, Line 587: information nit: information. -- To view, visit http://gerrit.cloudera.org:8080/16991 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I760287b3ed4d50e32d2f9257e5390fdf8fa8f288 Gerrit-Change-Number: 16991 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 28 Jan 2021 07:18:19 + Gerrit-HasComments: Yes
[kudu-CR] KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16991 ) Change subject: KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory .. Patch Set 1: Verified+1 Unrelated failure in RollingRestartArgs/RollingRestartITest.TestWorkloads/3 -- To view, visit http://gerrit.cloudera.org:8080/16991 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I760287b3ed4d50e32d2f9257e5390fdf8fa8f288 Gerrit-Change-Number: 16991 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 28 Jan 2021 07:02:10 + Gerrit-HasComments: No
[kudu-CR] KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory
Alexey Serbin has removed a vote on this change. Change subject: KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16991 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I760287b3ed4d50e32d2f9257e5390fdf8fa8f288 Gerrit-Change-Number: 16991 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16991 Change subject: KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory .. KUDU-3237 fix MaintenanceManagerTest.TestCompletedOpsHistory This patch fixes a flakiness in the TestCompletedOpsHistory scenario. The flakiness is a test-only issue which became apparent as result of the recent changes introduced into the MaintenanceManager with 9e4664d44 changelist. In essence, with the finer granularity of locking in the scoped cleanup of the MaintenanceManager::LaunchOp() method, the test thread calling MaintenanceManager::GetMaintenanceManagerStatusDump() have non-zero chances of acquiring 'completed_ops_lock_' ahead of the thread executing the code in the LaunchOp()'s scoped cleanup. This patch wraps the related code into ASSERT_EVENTUALLY to resolve test-only race condition mentioned above. I verified this fix addresses the issue running the affected test scenario multiple times under dist-test for RELEASE build. Before: 2 out of 256 runs failed http://dist-test.cloudera.org//job?job_id=aserbin.1611806979.74192 After : 0 out of 256 runs failed http://dist-test.cloudera.org//job?job_id=aserbin.1611809676.95320 Change-Id: I760287b3ed4d50e32d2f9257e5390fdf8fa8f288 --- M src/kudu/util/maintenance_manager-test.cc 1 file changed, 20 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/91/16991/1 -- To view, visit http://gerrit.cloudera.org:8080/16991 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I760287b3ed4d50e32d2f9257e5390fdf8fa8f288 Gerrit-Change-Number: 16991 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin
[kudu-CR] KUDU-2612 keep-alive txn heartbeating for Java client
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16967 ) Change subject: KUDU-2612 keep-alive txn heartbeating for Java client .. KUDU-2612 keep-alive txn heartbeating for Java client This patch adds keep-alive txn heartbeating into the Kudu Java client. The txn keepalive heartbeating is performed automatically by the client, and no API is exposed to send keep-alive messages for a transaction. The txn keepalive heartbeating continues until the original auto-closeable transaction handle (i.e. the handle created by KuduClient.newTransaction()) goes out of scope or the KuduTransaction.close() method called explicitly. Overall, the keepalive heartbeating behavior in the Kudu Java client mirrors the behavior of its C++ counterpart. This patch also contains a couple of test scenarios to cover the newly introduced functionality. Change-Id: I6326a5452223d8173b2da004bb5f3ab0c1e6ae4e Reviewed-on: http://gerrit.cloudera.org:8080/16967 Tested-by: Alexey Serbin Reviewed-by: Alexey Serbin --- M java/kudu-client/src/main/java/org/apache/kudu/client/BeginTransactionRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/KeepTransactionAliveRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/KeepTransactionAliveResponse.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduTransaction.java M java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTransaction.java 5 files changed, 497 insertions(+), 9 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6326a5452223d8173b2da004bb5f3ab0c1e6ae4e Gerrit-Change-Number: 16967 Gerrit-PatchSet: 6 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2612 keep-alive txn heartbeating for Java client
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16967 ) Change subject: KUDU-2612 keep-alive txn heartbeating for Java client .. Patch Set 5: Code-Review+2 Carrying over Andrew's +2 from PS3 and Hao's +2 from PS4. -- To view, visit http://gerrit.cloudera.org:8080/16967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6326a5452223d8173b2da004bb5f3ab0c1e6ae4e Gerrit-Change-Number: 16967 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 28 Jan 2021 04:59:23 + Gerrit-HasComments: No
[kudu-CR] KUDU-2612 keep-alive txn heartbeating for Java client
Alexey Serbin has removed a vote on this change. Change subject: KUDU-2612 keep-alive txn heartbeating for Java client .. Removed Verified-1 by Kudu Jenkins (120) -- To view, visit http://gerrit.cloudera.org:8080/16967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: deleteVote Gerrit-Change-Id: I6326a5452223d8173b2da004bb5f3ab0c1e6ae4e Gerrit-Change-Number: 16967 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2612 keep-alive txn heartbeating for Java client
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16967 ) Change subject: KUDU-2612 keep-alive txn heartbeating for Java client .. Patch Set 5: Verified+1 unrelated flake in AuthzProviders/TSAuthzITest.TestReadsAndWrites/Ranger test -- To view, visit http://gerrit.cloudera.org:8080/16967 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6326a5452223d8173b2da004bb5f3ab0c1e6ae4e Gerrit-Change-Number: 16967 Gerrit-PatchSet: 5 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 28 Jan 2021 04:58:46 + Gerrit-HasComments: No
[kudu-CR] KUDU-2612 Java client transaction implementation
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16929 ) Change subject: KUDU-2612 Java client transaction implementation .. KUDU-2612 Java client transaction implementation This patch implements the functionality behind the transaction-related API calls in the Java Kudu client (i.e. issuing RPC calls to TxnManager, retrying in case of transient errors, etc.). Corresponding tests have been added as well. Some of the newly introduced tests are disabled and should be re-enabled once the transaction commit orchestration is implemented. Change-Id: Ie0236875e7264877c3f7a13da4a5a3da6423786b Reviewed-on: http://gerrit.cloudera.org:8080/16929 Tested-by: Alexey Serbin Reviewed-by: Hao Hao --- A java/kudu-client/src/main/java/org/apache/kudu/client/AbortTransactionRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/AbortTransactionResponse.java M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java A java/kudu-client/src/main/java/org/apache/kudu/client/BeginTransactionRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/BeginTransactionResponse.java A java/kudu-client/src/main/java/org/apache/kudu/client/CommitTransactionRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/CommitTransactionResponse.java A java/kudu-client/src/main/java/org/apache/kudu/client/GetTransactionStateRequest.java A java/kudu-client/src/main/java/org/apache/kudu/client/GetTransactionStateResponse.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduRpc.java M java/kudu-client/src/main/java/org/apache/kudu/client/KuduTransaction.java M java/kudu-client/src/main/java/org/apache/kudu/client/RpcProxy.java M java/kudu-client/src/main/java/org/apache/kudu/client/Status.java A java/kudu-client/src/test/java/org/apache/kudu/client/TestKuduTransaction.java M src/kudu/transactions/txn_status_manager.cc 15 files changed, 1,200 insertions(+), 16 deletions(-) Approvals: Alexey Serbin: Verified Hao Hao: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16929 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie0236875e7264877c3f7a13da4a5a3da6423786b Gerrit-Change-Number: 16929 Gerrit-PatchSet: 10 Gerrit-Owner: Alexey Serbin Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3230: Fix the issue of hardcode sasl proto name
Hello Alexey Serbin, Kudu Jenkins, Bankim Bhavsar, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16956 to look at the new patch set (#17). Change subject: KUDU-3230: Fix the issue of hardcode sasl_proto_name .. KUDU-3230: Fix the issue of hardcode sasl_proto_name 'kudu' is used as sasl service name which is also used as kerberos principal name when Kerberos is enabled, as a result, users can only use 'kudu' than other principals. It is not flexible. It is better allow to customize sasl_krb5_principal_name by specifying other arguments for kudu-master and kudu-tserver. Change-Id: Ia1c8f9d9f772d000d9a588e4e9d6028711a62915 --- M src/kudu/integration-tests/security-itest.cc M src/kudu/rpc/messenger.cc M src/kudu/rpc/negotiation-test.cc 3 files changed, 46 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/16956/17 -- To view, visit http://gerrit.cloudera.org:8080/16956 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia1c8f9d9f772d000d9a588e4e9d6028711a62915 Gerrit-Change-Number: 16956 Gerrit-PatchSet: 17 Gerrit-Owner: Hongjiang Zhang Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-2612: add a TxnSystemClient to the tservers
Andrew Wong has posted comments on this change. (
http://gerrit.cloudera.org:8080/16974 )
Change subject: KUDU-2612: add a TxnSystemClient to the tservers
..
Patch Set 5:
(11 comments)
http://gerrit.cloudera.org:8080/#/c/16974/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/16974/4//COMMIT_MSG@23
PS4, Line 23: connecting to
> nit: connecting to master?
Done
http://gerrit.cloudera.org:8080/#/c/16974/4//COMMIT_MSG@33
PS4, Line 33: I left a TODO to refactor for code reuse
> SGTM -- I guess we can address that later on when the rest of the related p
Ack
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/rpc/sasl_common.cc
File src/kudu/rpc/sasl_common.cc:
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/rpc/sasl_common.cc@a280
PS4, Line 280:
> Why this is disabled? Sorry,I think you briefly mentioned offline, but I di
Servers initialize SASL in ServerBase::Init(), which calls the Once function
with the keytab. Clients, when building their messengers, also call SASL, but
don't use a keytab and instead rely on importing credentials from tokens.
There's a mismatch when both of these happen in the same process, since SASL
initialization sets some global state at L70.
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc
File src/kudu/transactions/txn_system_client.cc:
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@62
PS4, Line 62: disable_txn_system_client_init
> I guess a flag tagged as 'unsafe' is automatically marked as 'hidden': http
Right, 'unsafe' flags are automatically 'hidden'.
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@383
PS4, Line 383: ile (!shutting_down_) {
: static const MonoDelta kRetryInterval =
MonoDelta::FromSeconds(1);
: if (PREDICT_FALSE(FLAGS_disable_txn_system_client_init))
{
: KLOG_EVERY_N_SECS(WARNING, 60) <<
: Substitute("initialization of TxnSystemClient
disabled, will retry in $0",
: kRetryInterval.ToString());
:
> As we are retrying, do you expect FLAGS_disable_txn_system_client_init to c
I don't, but it can.
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@404
PS4, Line 404:
> nit: does it make sense to set it to be equal to the connection negotiation
Done
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@390
PS4, Line 390: continue;
: }
: // HACK: if the master addresses are all totally
unreachable,
: // KuduClientBuilder::Build() will hang, attempting
fruitlessly to
: // retry, in the below call to Create(). So first, make
sure we can at
: // least reach the masters; if not, try again.
: // TODO(awong): there's still a small window between
these pings and
: // client creation. If this ends up being a problem, we
may need to
: // come to a more robust solution, e.g. adding a timeout
to Create().
: DnsResolver dns_resolver;
: Status s;
: for (const auto& hp : master_addrs) {
: vector addrs;
: s = dns_resolver.ResolveAddresses(hp,
).AndThen([&] {
: unique_ptr proxy(
: new MasterServiceProxy(messenger, addrs[0],
hp.host()));
: PingRequestPB req;
: PingResponsePB resp;
: RpcController rpc;
:
rpc.set_timeout(MonoDelta::FromMilliseconds(FLAGS_rpc_negotiation_timeout_ms));
:
> I guess the sanitize check is sufficient enough, even later when we actuall
I'm not sure I understand the question. Are you asking what happens if, between
the Ping() and Create() calls, the masters become unreachable?
Yeah, it might be worth making this robust, though I'm going to hold off and
write a TODO, since having all masters become unavailable at once seems indeed
unlikely.
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@414
PS4, Line 414: }
> nit: what if masters become unavailable right after the check above but bef
Yeah, it might be worth making this robust, though I'm going to hold off and
write a TODO, since having all masters become unavailable at once seems indeed
unlikely.
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@435
PS4, Line 435: shutdown c
> Do we want to check(dcheck) txn_client_ is not null here?
Done
http://gerrit.cloudera.org:8080/#/c/16974/4/src/kudu/transactions/txn_system_client.cc@443
PS4, Line 443:
[kudu-CR] KUDU-2612: add a TxnSystemClient to the tservers
Hello Alexey Serbin, Kudu Jenkins, Hao Hao, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16974 to look at the new patch set (#5). Change subject: KUDU-2612: add a TxnSystemClient to the tservers .. KUDU-2612: add a TxnSystemClient to the tservers This patch adds a TxnSystemClient that gets initialized asynchronously, attempting to connect to the masters in the background in a similar fashion to the Heartbeater threads. There is some intricacy in the initialization of the client to note. Namely, if trying to connect to a set of masters while none of the masters can be reached, the KuduClientBuilder will attempt to retry connecting to each master repeatedly. This is problematic, as several tserver tests do not spin up masters. So, taking a page out of the Heartbeater book, the initialization will first ping each master. As long as at least one of them can connect, the TxnSystemClient will then proceed to attempt to connect to the cluster. This patch focuses only on adding the client initialization logic to the tservers, ensuring that even without connecting to a cluster, tservers can successfully bootstrap. This client will be used in later patches to communicate between the TxnStatusManager and transaction participants. Also note the similarities between TxnManager initialization, which happens on masters in a single-threaded threadpool. I considered reusing the implementation for tservers, but opted not to given the TxnManager initialization fairly well embedded in master code and has some initialization pieces not needed by TxnStatusManagers and participants (e.g. tservers aren't in charge of creating the transaction status table). I left a TODO to refactor for code reuse. Change-Id: I33b5a2bb5c56ae4bb4b42069af5813e2780fc4bc --- M src/kudu/client/client-test.cc M src/kudu/integration-tests/location_assignment-itest.cc M src/kudu/integration-tests/txn_status_table-itest.cc M src/kudu/rpc/sasl_common.cc M src/kudu/transactions/CMakeLists.txt M src/kudu/transactions/txn_system_client.cc M src/kudu/transactions/txn_system_client.h M src/kudu/tserver/tablet_server.cc M src/kudu/tserver/tablet_server.h 9 files changed, 214 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/74/16974/5 -- To view, visit http://gerrit.cloudera.org:8080/16974 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I33b5a2bb5c56ae4bb4b42069af5813e2780fc4bc Gerrit-Change-Number: 16974 Gerrit-PatchSet: 5 Gerrit-Owner: Andrew Wong Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR](gh-pages) Update website for 1.14.0 release
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16990 Change subject: Update website for 1.14.0 release .. Update website for 1.14.0 release Change-Id: Ia97cc8456a8ad0a81bb9732c7576641f41e165aa --- M apidocs M cpp-client-api M docs A releases/1.14.0/apidocs/allclasses-frame.html A releases/1.14.0/apidocs/allclasses-noframe.html A releases/1.14.0/apidocs/constant-values.html A releases/1.14.0/apidocs/deprecated-list.html A releases/1.14.0/apidocs/help-doc.html A releases/1.14.0/apidocs/index-all.html A releases/1.14.0/apidocs/index.html A releases/1.14.0/apidocs/org/apache/kudu/ColumnSchema.ColumnSchemaBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/ColumnSchema.CompressionAlgorithm.html A releases/1.14.0/apidocs/org/apache/kudu/ColumnSchema.Encoding.html A releases/1.14.0/apidocs/org/apache/kudu/ColumnSchema.html A releases/1.14.0/apidocs/org/apache/kudu/ColumnTypeAttributes.ColumnTypeAttributesBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/ColumnTypeAttributes.html A releases/1.14.0/apidocs/org/apache/kudu/Schema.html A releases/1.14.0/apidocs/org/apache/kudu/Type.html A releases/1.14.0/apidocs/org/apache/kudu/client/AbstractKuduScannerBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/client/AlterTableOptions.html A releases/1.14.0/apidocs/org/apache/kudu/client/AlterTableResponse.html A releases/1.14.0/apidocs/org/apache/kudu/client/AsyncKuduClient.AsyncKuduClientBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/client/AsyncKuduClient.html A releases/1.14.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.AsyncKuduScannerBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.ReadMode.html A releases/1.14.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.RowDataFormat.html A releases/1.14.0/apidocs/org/apache/kudu/client/AsyncKuduScanner.html A releases/1.14.0/apidocs/org/apache/kudu/client/AsyncKuduSession.html A releases/1.14.0/apidocs/org/apache/kudu/client/ColumnRangePredicate.html A releases/1.14.0/apidocs/org/apache/kudu/client/CreateTableOptions.html A releases/1.14.0/apidocs/org/apache/kudu/client/Delete.html A releases/1.14.0/apidocs/org/apache/kudu/client/DeleteIgnore.html A releases/1.14.0/apidocs/org/apache/kudu/client/DeleteTableResponse.html A releases/1.14.0/apidocs/org/apache/kudu/client/ExternalConsistencyMode.html A releases/1.14.0/apidocs/org/apache/kudu/client/HasFailedRpcException.html A releases/1.14.0/apidocs/org/apache/kudu/client/Insert.html A releases/1.14.0/apidocs/org/apache/kudu/client/InsertIgnore.html A releases/1.14.0/apidocs/org/apache/kudu/client/IsAlterTableDoneResponse.html A releases/1.14.0/apidocs/org/apache/kudu/client/IsCreateTableDoneResponse.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduClient.KuduClientBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduClient.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduException.OriginalException.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduException.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduPartitioner.KuduPartitionerBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduPartitioner.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduPredicate.ComparisonOp.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduPredicate.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduScanToken.KuduScanTokenBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduScanToken.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduScanner.KuduScannerBuilder.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduScanner.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduScannerIterator.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduSession.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduTable.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduTableStatistics.html A releases/1.14.0/apidocs/org/apache/kudu/client/KuduTransaction.html A releases/1.14.0/apidocs/org/apache/kudu/client/ListTablesResponse.html A releases/1.14.0/apidocs/org/apache/kudu/client/ListTabletServersResponse.html A releases/1.14.0/apidocs/org/apache/kudu/client/LocatedTablet.Replica.html A releases/1.14.0/apidocs/org/apache/kudu/client/LocatedTablet.html A releases/1.14.0/apidocs/org/apache/kudu/client/Operation.html A releases/1.14.0/apidocs/org/apache/kudu/client/OperationResponse.html A releases/1.14.0/apidocs/org/apache/kudu/client/PartialRow.html A releases/1.14.0/apidocs/org/apache/kudu/client/PleaseThrottleException.html A releases/1.14.0/apidocs/org/apache/kudu/client/RangePartitionBound.html A releases/1.14.0/apidocs/org/apache/kudu/client/ReplicaSelection.html A releases/1.14.0/apidocs/org/apache/kudu/client/ResourceMetrics.html A releases/1.14.0/apidocs/org/apache/kudu/client/RowError.html A
[kudu-CR] [docker] Update pip install
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16984 ) Change subject: [docker] Update pip install .. Patch Set 1: Code-Review+1 (3 comments) http://gerrit.cloudera.org:8080/#/c/16984/1/docker/bootstrap-python-env.sh File docker/bootstrap-python-env.sh: http://gerrit.cloudera.org:8080/#/c/16984/1/docker/bootstrap-python-env.sh@37 PS1, Line 37: "7" Would it make sense to fail if below 2.7 in an obvious way? I imagine it'll fail now complaining about a syntax error trying to use the general get-pip.py http://gerrit.cloudera.org:8080/#/c/16984/1/docker/bootstrap-python-env.sh@38 PS1, Line 38:c nit: maybe mention that pypa's generic script isn't supported with 2.7 so we need to get a specific one. http://gerrit.cloudera.org:8080/#/c/16984/1/docker/bootstrap-python-env.sh@40 PS1, Line 40: Python 2. nit: 2 and 3, as before? -- To view, visit http://gerrit.cloudera.org:8080/16984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9a22e5084f8a5973d26372372f2ead4c46dcda9 Gerrit-Change-Number: 16984 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 27 Jan 2021 19:20:33 + Gerrit-HasComments: Yes
[kudu-CR](branch-1.14.x) KUDU-3239: [build] Disable errorprone on Java 11
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16986 ) Change subject: KUDU-3239: [build] Disable errorprone on Java 11 .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.14.x Gerrit-MessageType: comment Gerrit-Change-Id: Ic13f771222e67ac754f464f8015caf0a4901831d Gerrit-Change-Number: 16986 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 27 Jan 2021 18:48:26 + Gerrit-HasComments: No
[kudu-CR] KUDU-3230: Fix the issue of hardcode sasl proto name
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16956 ) Change subject: KUDU-3230: Fix the issue of hardcode sasl_proto_name .. Patch Set 16: (2 comments) http://gerrit.cloudera.org:8080/#/c/16956/16/src/kudu/integration-tests/security-itest.cc File src/kudu/integration-tests/security-itest.cc: http://gerrit.cloudera.org:8080/#/c/16956/16/src/kudu/integration-tests/security-itest.cc@227 PS16, Line 227: Test customized SASL protocol name. It would be nice to outline the sequence of the actions and the expected outcome. I.e., are Kudu server components supposed to run with the customized Kerberos credentials? What Kerberos credentials the client is going to use to access the cluster? Basically, outline the essence of the scenario here pointing to the important preconditions and outline the expected outcome. In the code below, you could add comments to outline some particular details, as needed. http://gerrit.cloudera.org:8080/#/c/16956/16/src/kudu/integration-tests/security-itest.cc@230 PS16, Line 230: FLAGS_sasl_krb5_principal_name I'm not sure I understand what's the purpose of setting this flag right here: is it just to have custom principal name for the client or the servers are supposed to use the custom SPN as well? If the latter, the proper way to do so is adding custom flags for kudu-master process run as a part of the ExternalMiniCluster. An example can be found in the SaslPlainFallback test scenario. -- To view, visit http://gerrit.cloudera.org:8080/16956 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia1c8f9d9f772d000d9a588e4e9d6028711a62915 Gerrit-Change-Number: 16956 Gerrit-PatchSet: 16 Gerrit-Owner: Hongjiang Zhang Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 27 Jan 2021 18:42:20 + Gerrit-HasComments: Yes
[kudu-CR] [docker] Update pip install
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16984 ) Change subject: [docker] Update pip install .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/16984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9a22e5084f8a5973d26372372f2ead4c46dcda9 Gerrit-Change-Number: 16984 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Andrew Wong Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 27 Jan 2021 18:37:43 + Gerrit-HasComments: No
[kudu-CR](branch-1.14.x) KUDU-3239: [build] Disable errorprone on Java 11
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16986 ) Change subject: KUDU-3239: [build] Disable errorprone on Java 11 .. Patch Set 1: Code-Review+2 Thanks! -- To view, visit http://gerrit.cloudera.org:8080/16986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.14.x Gerrit-MessageType: comment Gerrit-Change-Id: Ic13f771222e67ac754f464f8015caf0a4901831d Gerrit-Change-Number: 16986 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 27 Jan 2021 18:08:36 + Gerrit-HasComments: No
[kudu-CR](branch-1.14.x) KUDU-3239: [build] Disable errorprone on Java 11
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16986 Change subject: KUDU-3239: [build] Disable errorprone on Java 11 .. KUDU-3239: [build] Disable errorprone on Java 11 It turns out that errorprone is not working with Java 11. This patch disables it until we can upgrade/update the errorprone plugin to work on with Java 11. Change-Id: Ic13f771222e67ac754f464f8015caf0a4901831d Reviewed-on: http://gerrit.cloudera.org:8080/16983 Reviewed-by: Bankim Bhavsar Reviewed-by: Hao Hao Reviewed-by: Alexey Serbin Tested-by: Kudu Jenkins (cherry picked from commit 922ab9e7d46f683a5e04b4e36b1af9b95e96e025) --- M java/gradle/quality.gradle 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/16986/1 -- To view, visit http://gerrit.cloudera.org:8080/16986 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.14.x Gerrit-MessageType: newchange Gerrit-Change-Id: Ic13f771222e67ac754f464f8015caf0a4901831d Gerrit-Change-Number: 16986 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[kudu-CR] KUDU-3239: [build] Disable errorprone on Java 11
Bankim Bhavsar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16983 ) Change subject: KUDU-3239: [build] Disable errorprone on Java 11 .. Patch Set 3: Reminder to cherry pick this change to 1.14 branch. -- To view, visit http://gerrit.cloudera.org:8080/16983 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic13f771222e67ac754f464f8015caf0a4901831d Gerrit-Change-Number: 16983 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Alexey Serbin Gerrit-Reviewer: Attila Bukor Gerrit-Reviewer: Bankim Bhavsar Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Wed, 27 Jan 2021 18:06:29 + Gerrit-HasComments: No
[kudu-CR] KUDU-3230: Fix the issue of hardcode sasl proto name
Bankim Bhavsar has posted comments on this change. (
http://gerrit.cloudera.org:8080/16956 )
Change subject: KUDU-3230: Fix the issue of hardcode sasl_proto_name
..
Patch Set 16:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/16956/16/src/kudu/integration-tests/security-itest.cc
File src/kudu/integration-tests/security-itest.cc:
http://gerrit.cloudera.org:8080/#/c/16956/16/src/kudu/integration-tests/security-itest.cc@240
PS16, Line 240: //string klist;
: //ASSERT_OK(cluster_->kdc()->Klist());
: //ASSERT_STR_CONTAINS(klist, spn);
Could you remove this commented out code?
http://gerrit.cloudera.org:8080/#/c/16956/16/src/kudu/rpc/messenger.cc
File src/kudu/rpc/messenger.cc:
http://gerrit.cloudera.org:8080/#/c/16956/16/src/kudu/rpc/messenger.cc@64
PS16, Line 64: return v.find_first_of("\t\n ") == std::string::npos;
More comprehensive to use isspace() check as it covers other whitespace
characters as well.
https://en.cppreference.com/w/cpp/string/byte/isspace
--
To view, visit http://gerrit.cloudera.org:8080/16956
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia1c8f9d9f772d000d9a588e4e9d6028711a62915
Gerrit-Change-Number: 16956
Gerrit-PatchSet: 16
Gerrit-Owner: Hongjiang Zhang
Gerrit-Reviewer: Alexey Serbin
Gerrit-Reviewer: Bankim Bhavsar
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 27 Jan 2021 18:00:01 +
Gerrit-HasComments: Yes
[kudu-CR] [docker] Update pip install
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16984 Change subject: [docker] Update pip install .. [docker] Update pip install It looks like Python has adjusted the get-pip.py URL to no longer support Python 2.7. This patch updates the Docker scripts to use the 2.7 specific URL and drop the old 2.6 speicific url now that no supported images use Python 2.6. Change-Id: Ia9a22e5084f8a5973d26372372f2ead4c46dcda9 --- M docker/bootstrap-dev-env.sh M docker/bootstrap-python-env.sh 2 files changed, 5 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/16984/1 -- To view, visit http://gerrit.cloudera.org:8080/16984 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia9a22e5084f8a5973d26372372f2ead4c46dcda9 Gerrit-Change-Number: 16984 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
