[
https://issues.apache.org/jira/browse/HADOOP-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17124241#comment-17124241
]
Hudson commented on HADOOP-16828:
---------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18319 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/18319/])
HADOOP-16828. Zookeeper Delegation Token Manager fetch sequence number (xyao:
rev 6288e15118fab65a9a1452898e639313c6996769)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java
> Zookeeper Delegation Token Manager fetch sequence number by batch
> -----------------------------------------------------------------
>
> Key: HADOOP-16828
> URL: https://issues.apache.org/jira/browse/HADOOP-16828
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Fengnan Li
> Assignee: Fengnan Li
> Priority: Major
> Fix For: 3.4.0
>
> Attachments: HADOOP-16828.001.patch, HADOOP-16828.002.patch, Screen
> Shot 2020-01-25 at 2.25.06 PM.png, Screen Shot 2020-01-25 at 2.25.16 PM.png,
> Screen Shot 2020-01-25 at 2.25.24 PM.png
>
>
> Currently in ZKDelegationTokenSecretManager.java the seq number is
> incremented by 1 each time there is a request for creating new token. This
> will need to send traffic to Zookeeper server. With multiple managers
> running, there is data contention going on. Also, since the current logic of
> incrementing is using tryAndSet which is optimistic concurrency control
> without locking. This data contention is having performance degradation when
> the secret manager are under volume of traffic.
> The change here is to fetching this seq number by batch instead of 1, which
> will reduce the traffic sent to ZK and make many operations inside ZK secret
> manager's memory.
> After putting this into production we saw huge improvement to the RPC
> processing latency of get delegationtoken calls. Also, since ZK takes less
> traffic in this way. Other write calls, like renew and cancel delegation
> tokens are benefiting from this change.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
