[
https://issues.apache.org/jira/browse/HADOOP-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17692316#comment-17692316
]
ASF GitHub Bot commented on HADOOP-18535:
-----------------------------------------
omalley closed pull request #5325: HADOOP-18535. Implement token storage
solution based on MySQL
URL: https://github.com/apache/hadoop/pull/5325
> Implement token storage solution based on MySQL
> -----------------------------------------------
>
> Key: HADOOP-18535
> URL: https://issues.apache.org/jira/browse/HADOOP-18535
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Hector Sandoval Chaverri
> Assignee: Hector Sandoval Chaverri
> Priority: Major
> Labels: pull-request-available
>
> Hadoop RBF supports custom implementations of secret managers. At the moment,
> the only available implementation is ZKDelegationTokenSecretManagerImpl,
> which stores tokens and delegation keys in Zookeeper.
> During our investigation, we found that the performance of routers is limited
> by the writes to the Zookeeper token store, which impacts requests for token
> creation, renewal and cancellation. An alternative secret manager
> implementation has been created, based on MySQL, to handle a higher number of
> writes.
> We measured the throughput of each token operation (create/renew/cancel) on
> different setups and obtained the following results:
> # Sending requests directly to Namenode (no RBF):
> Token creations: 290 reqs per sec
> Token renewals: 86 reqs per sec
> Token cancellations: 97 reqs per sec
> # Sending requests to routers using Zookeeper based secret manager:
> Token creations: 31 reqs per sec
> Token renewals: 29 reqs per sec
> Token cancellations: 40 reqs per sec
> # Sending requests to routers using SQL based secret manager:
> Token creations: 241 reqs per sec
> Token renewals: 103 reqs per sec
> Token cancellations: 114 reqs per sec
> We noticed a significant improvement when using a SQL secret manager,
> comparable to the throughput offered by Namenodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
