[
https://issues.apache.org/jira/browse/HADOOP-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-16130:
-------------------------------------
Description:
At the last Hadoop Contributors Meetup, [~daryn] shared another KMS throughput
bottleneck is ZooKeeper -- KMS uses ZK to store delegation tokens. ZK would be
brought to a halt when expired delegation tokens are purged. That sounds
critical especially given that in most deployments KMS share the same ZK quorum
as HDFS, it would cause NameNode failover.
The current KMS benchmark does not support delegation token operations
(addDelegationTokens, cancelDelegationToken, renewDelegationToken) so it's hard
to understand how bad it is, and hard to quantify the improvement of a fix.
File this jira to support those operations before we move on to the fix for the
ZK issue.
was:
At the last Hadoop Contributors Meetup, [~daryn] mentioned another KMS
throughput bottleneck is ZooKeeper -- KMS uses ZK to store delegation tokens.
ZK would be brought to a halt when expired delegation tokens are purged. That
sounds critical especially given that in most deployments KMS share the same ZK
quorum as HDFS, it would cause NameNode failover.
The current KMS benchmark does not support delegation token operations
(addDelegationTokens, cancelDelegationToken, renewDelegationToken) so it's hard
to understand how bad it is, and hard to quantify the improvement of a fix.
File this jira to support those operations before we move on to the fix for the
ZK issue.
> Support delegation token operations in KMS Benchmark
> ----------------------------------------------------
>
> Key: HADOOP-16130
> URL: https://issues.apache.org/jira/browse/HADOOP-16130
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 3.3.0
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> At the last Hadoop Contributors Meetup, [~daryn] shared another KMS
> throughput bottleneck is ZooKeeper -- KMS uses ZK to store delegation tokens.
> ZK would be brought to a halt when expired delegation tokens are purged. That
> sounds critical especially given that in most deployments KMS share the same
> ZK quorum as HDFS, it would cause NameNode failover.
> The current KMS benchmark does not support delegation token operations
> (addDelegationTokens, cancelDelegationToken, renewDelegationToken) so it's
> hard to understand how bad it is, and hard to quantify the improvement of a
> fix.
> File this jira to support those operations before we move on to the fix for
> the ZK issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
