Piotr Wikieł created HIVE-13075:
-----------------------------------
Summary: Metastore shuts down when no delegation token is found in
ZooKeeper
Key: HIVE-13075
URL: https://issues.apache.org/jira/browse/HIVE-13075
Project: Hive
Issue Type: Bug
Reporter: Piotr Wikieł
Priority: Minor
{{ZooKeeperTokenStore}} looks [as
follows|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java#L397]:
{code:java}
@Override
public DelegationTokenInformation getToken(DelegationTokenIdentifier
tokenIdentifier) {
byte[] tokenBytes = zkGetData(getTokenPath(tokenIdentifier));
try {
return
HiveDelegationTokenSupport.decodeDelegationTokenInformation(tokenBytes);
} catch (Exception ex) {
throw new TokenStoreException("Failed to decode token", ex);
}
}
{code}
which is slightly different from [DBTokenStore
implementation|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/DBTokenStore.java#L85]
that is protected against {{tokenBytes==null}} because nullable {{tokenBytes}}
causes NPE to be thrown in
[HiveDelegationTokenSupport#decodeDelegationTokenInformation|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java#L51]
Furthermore, NPE thrown here causes
[TokenStoreDelegationTokenSecretManager.ExpiredTokenRemover|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java#L333]
to catch it and exits MetaStore.
{{null}} from
{{[zkGetData()|https://github.com/apache/hive/blob/branch-1.2/shims/common/src/main/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java#L284]}}
is possible during ZooKeeper failure or (and that was our case) when another
metastore instance removes tokens during {{ExpiredTokenRemover}} run. There
were two solutions of this problem:
* distributed lock in ZooKeeper acquired during one metastore instance's
ExpiredTokenRemover run,
* simple null check
I think null check is sufficient if it is in {{DBTokenStore}}.
Patch will be attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
