[
https://issues.apache.org/jira/browse/HDFS-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937288#comment-16937288
]
Konstantin Shvachko edited comment on HDFS-14509 at 9/25/19 1:12 AM:
---------------------------------------------------------------------
We should assume that protobuf for {{BlockTokenIdentifier}} is disabled, since
DN 2.x cannot read proto.
Then I see two ways to fix this
# NN 3.x does not include storage types into block token until the upgrade is
finalized.
This will require changes on branch-3.x only.
# DN 2.x uses all raw bytes received from NN (including storages) for
calculating the password. Deserialization of these bytes on DN 2.x will still
omit storage fields.
This should be uniformly changed on all branches up to 2.x.
Looks like [~brahmareddy]'s patch goes in the direction of approach #1. While
[~John Smith]'s PR assumes approach #2.
Both should work, but it seems that #2 is more general. Since it will work if
we ever decide to add more fields into the token. While with #1 we will have to
explicitly exclude any new fields during upgrade.
was (Author: shv):
We should assume that protobuf for {{BlockTokenIdentifier}} is disabled, since
DN 2.x cannot read proto.
Then I see two ways to fix this
# NN 3.x does not include storage types into block token until the upgrade is
finalized.
This will require changes on branch-3.x only.
# DN 2.x uses all raw bytes received from NN (including storages) for
calculating the password. Desalinization of these bytes will still omit storage
fields.
This should be uniformly changed on all branches up to 2.x.
Looks like [~brahmareddy]'s patch goes in the direction of approach #1. While
[~John Smith]'s PR assumes approach #2.
Both should work, but it seems that #2 is more general. Since it will work if
we ever decide to add more fields into the token. While with #1 we will have to
explicitly exclude any new fields during upgrade.
> DN throws InvalidToken due to inequality of password when upgrade NN 2.x to
> 3.x
> -------------------------------------------------------------------------------
>
> Key: HDFS-14509
> URL: https://issues.apache.org/jira/browse/HDFS-14509
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Yuxuan Wang
> Priority: Blocker
> Attachments: HDFS-14509-001.patch
>
>
> According to the doc, if we want to upgrade cluster from 2.x to 3.x, we need
> upgrade NN first. And there will be a intermediate state that NN is 3.x and
> DN is 2.x. At that moment, if a client reads (or writes) a block, it will get
> a block token from NN and then deliver the token to DN who can verify the
> token. But the verification in the code now is :
> {code:title=BlockTokenSecretManager.java|borderStyle=solid}
> public void checkAccess(...)
> {
> ...
> id.readFields(new DataInputStream(new
> ByteArrayInputStream(token.getIdentifier())));
> ...
> if (!Arrays.equals(retrievePassword(id), token.getPassword())) {
> throw new InvalidToken("Block token with " + id.toString()
> + " doesn't have the correct token password");
> }
> }
> {code}
> And {{retrievePassword(id)}} is:
> {code}
> public byte[] retrievePassword(BlockTokenIdentifier identifier)
> {
> ...
> return createPassword(identifier.getBytes(), key.getKey());
> }
> {code}
> So, if NN's identifier add new fields, DN will lose the fields and compute
> wrong password.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
