Hello Will Berkeley, Kudu Jenkins, Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11449
to look at the new patch set (#2).
Change subject: KUDU-2580 [c++ client] authn token reacquisition fix
......................................................................
KUDU-2580 [c++ client] authn token reacquisition fix
This patch updates the authn token reacquisition logic to handle
the following scenario:
1. Client is running against a multi-master cluster.
2. Client successfully authenticates and gets an authn token by calling
ConnectToCluster().
3. Client keeps the connection to leader master open, but follower
masters close connections to the client due to inactivity.
4. After the authn token expires, a change in the master leadership
happens.
5. Client tries to open a table, making a request to the former leader
master. The former leader returns NOT_THE_LEADER error.
The original authn token reacquisition logic was straightforwardly
retrying RPCs in case of error responses with code
FATAL_INVALID_AUTHENTICATION_TOKEN only. The scenario described
above was not handled properly, so the opening table operation would
fail with the following error:
Timed out: GetTableSchema timed out after deadline expired
Under the hood, the following would happen prior to this patch in
the above scenario when calling the GetTableSchema():
* Having the connection to the former leader master still open,
but authn token expired and master re-election selected another
master as leader, the GetTableSchema() RPC call would hit the
code of the last 'if ()' closure within the 'for ()' retry cycle
in the KuduClient::Data::SyncLeaderMasterRpc() method,
with error code MasterErrorPB::NOT_THE_LEADER.
* The ConnectToMaster() attempt would return an error with the
FATAL_INVALID_AUTHENTICATION_TOKEN error code, and the control
flow will continue with the retries, but the state of the client
stays the same (i.e. the leader master proxy is not updated).
* That will happen again and again, until the GetTableSchema() RPC
call times out.
This patch also enables the ClientReacquiresAuthnToken scenario
of MultiMasterIdleConnectionsITest since it passes with this fix.
Change-Id: I4477d0f2bb36ee5ef580b585cae189a634d5002f
---
M src/kudu/client/client-internal.cc
M src/kudu/client/client-internal.h
M src/kudu/client/master_rpc.cc
M src/kudu/integration-tests/authn_token_expire-itest.cc
4 files changed, 64 insertions(+), 33 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/49/11449/2
--
To view, visit http://gerrit.cloudera.org:8080/11449
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4477d0f2bb36ee5ef580b585cae189a634d5002f
Gerrit-Change-Number: 11449
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <[email protected]>
