[
https://issues.apache.org/jira/browse/HDFS-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321866#comment-17321866
]
zhangxiping edited comment on HDFS-15756 at 4/15/21, 6:10 AM:
--------------------------------------------------------------
-_- like :
{code:java}
//代码占位符 AbstractDelegationTokenSecretManager
public synchronized long renewToken(Token<TokenIdent> token,
String renewer) throws InvalidToken, IOException {
ByteArrayInputStream buf = new ByteArrayInputStream(token.getIdentifier());
DataInputStream in = new DataInputStream(buf);
TokenIdent id = createIdentifier();
id.readFields(in);
LOG.info("Token renewal for identifier: " + formatTokenId(id)
+ "; total currentTokens " + currentTokens.size());
...
...
DelegationTokenInformation tokenInfo = getTokenInfo(id);
if (tokenInfo == null && id.getIssueDate() + 60000 > now) {
throw new StandbyException("Renewal request for unknown token "
+ formatTokenId(id)+",will Failover to other server and retry");
}
if (tokenInfo == null) {
throw new InvalidToken("Renewal request for unknown token "
+ formatTokenId(id));
}
updateToken(id, info);
return renewTime;
}
{code}
was (Author: zhangxiping):
-_- like :
{code:java}
//代码占位符 AbstractDelegationTokenSecretManager
public synchronized long renewToken(Token<TokenIdent> token,
String renewer) throws InvalidToken, IOException {
ByteArrayInputStream buf = new ByteArrayInputStream(token.getIdentifier());
DataInputStream in = new DataInputStream(buf);
TokenIdent id = createIdentifier();
id.readFields(in);
LOG.info("Token renewal for identifier: " + formatTokenId(id)
+ "; total currentTokens " + currentTokens.size());
long now = Time.now();
...
...
if (getTokenInfo(id) == null && id.getIssueDate() + 60000 > now) {
throw new StandbyException("Renewal request for unknown token "
+ formatTokenId(id)+",will Failover to other server and retry");
}
// if (getTokenInfo(id) == null) {
// throw new InvalidToken("Renewal request for unknown token "
// + formatTokenId(id));
// }
updateToken(id, info);
return renewTime;
}
{code}
> RBF: Cannot get updated delegation token from zookeeper
> -------------------------------------------------------
>
> Key: HDFS-15756
> URL: https://issues.apache.org/jira/browse/HDFS-15756
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
> Affects Versions: 3.0.0
> Reporter: hbprotoss
> Priority: Major
>
> Affected version: all version with rbf
> When RBF work with spark 2.4 client mode, there will be a chance that token
> is missing across different nodes in RBF cluster. The root cause is that
> spark renew the token(via resource manager) immediately after got one, as
> zookeeper don't have a strong consistency guarantee after an update in
> cluster, zookeeper client may read a stale value in some followers not synced
> with other nodes.
>
> We apply a patch in spark, but it is still the problem of RBF. Is it possible
> for RBF to replace the delegation token store using some other
> datasource(redis for example)?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
