[
https://issues.apache.org/jira/browse/HADOOP-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell updated HADOOP-16828:
---------------------------------------
Fix Version/s: 3.3.1
> Zookeeper Delegation Token Manager fetch sequence number by batch
> -----------------------------------------------------------------
>
> Key: HADOOP-16828
> URL: https://issues.apache.org/jira/browse/HADOOP-16828
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Fengnan Li
> Assignee: Fengnan Li
> Priority: Major
> Fix For: 3.3.1, 3.4.0
>
> Attachments: HADOOP-16828.001.patch, HADOOP-16828.002.patch, Screen
> Shot 2020-01-25 at 2.25.06 PM.png, Screen Shot 2020-01-25 at 2.25.16 PM.png,
> Screen Shot 2020-01-25 at 2.25.24 PM.png
>
>
> Currently in ZKDelegationTokenSecretManager.java the seq number is
> incremented by 1 each time there is a request for creating new token. This
> will need to send traffic to Zookeeper server. With multiple managers
> running, there is data contention going on. Also, since the current logic of
> incrementing is using tryAndSet which is optimistic concurrency control
> without locking. This data contention is having performance degradation when
> the secret manager are under volume of traffic.
> The change here is to fetching this seq number by batch instead of 1, which
> will reduce the traffic sent to ZK and make many operations inside ZK secret
> manager's memory.
> After putting this into production we saw huge improvement to the RPC
> processing latency of get delegationtoken calls. Also, since ZK takes less
> traffic in this way. Other write calls, like renew and cancel delegation
> tokens are benefiting from this change.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
