[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11956: --- Release Note: Hadoop 2.x clients do not pass the storage ID or target storage IDs when writing a block. For backwards compatibility, the DataNode will not require the presence of these fields. This means older clients are unable to write to a particular storage as chosen by the NameNode (e.g. HDFS-9806). (was: Introduce dfs.block.access.token.storageid.enable which will be false by default. When it's turned on, the BlockTokenSecretManager.checkAccess will consider the storage ID when verifying the request. This allows for backwards compatibility all the way back to 2.6.x.) > Fix BlockToken compatibility with Hadoop 2.x clients > > > Key: HDFS-11956 > URL: https://issues.apache.org/jira/browse/HDFS-11956 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Ewan Higgs >Priority: Blocker > Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch, > HDFS-11956.003.patch, HDFS-11956.004.patch > > > Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. > When talking to a 3.0.0-alpha4 DN with security on: > {noformat} > 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: > Block token verification failed: op=WRITE_BLOCK, > remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs > [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with > StorageIDs [] > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.004.patch Attaching version of the patch that doesn't use a config switch. > Fix BlockToken compatibility with Hadoop 2.x clients > > > Key: HDFS-11956 > URL: https://issues.apache.org/jira/browse/HDFS-11956 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Ewan Higgs >Priority: Blocker > Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch, > HDFS-11956.003.patch, HDFS-11956.004.patch > > > Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. > When talking to a 3.0.0-alpha4 DN with security on: > {noformat} > 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: > Block token verification failed: op=WRITE_BLOCK, > remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs > [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with > StorageIDs [] > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.003.patch Attaching patch with value for hdfs-default.xml and some checkstyle fixes. > Fix BlockToken compatibility with Hadoop 2.x clients > > > Key: HDFS-11956 > URL: https://issues.apache.org/jira/browse/HDFS-11956 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Ewan Higgs >Priority: Blocker > Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch, > HDFS-11956.003.patch > > > Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. > When talking to a 3.0.0-alpha4 DN with security on: > {noformat} > 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: > Block token verification failed: op=WRITE_BLOCK, > remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs > [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with > StorageIDs [] > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.002.patch Attaching updated patch with a unit test. In the test, {{strictSM}} {{BlockTokenSecretManager}} will fail when the passed storageIds are wrong; but {{permissiveSM}} will allow it. {{strictSM}} corresponds to having the config value enabled while {{permissiveSM}} corresponds to it being disabled for legacy clients. > Fix BlockToken compatibility with Hadoop 2.x clients > > > Key: HDFS-11956 > URL: https://issues.apache.org/jira/browse/HDFS-11956 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Ewan Higgs >Priority: Blocker > Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch > > > Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. > When talking to a 3.0.0-alpha4 DN with security on: > {noformat} > 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: > Block token verification failed: op=WRITE_BLOCK, > remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs > [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with > StorageIDs [] > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Assignee: Ewan Higgs (was: Chris Douglas) Release Note: Introduce dfs.block.access.token.storageid.enable which will be false by default. When it's turned on, the BlockTokenSecretManager.checkAccess will consider the storage ID when verifying the request. This allows for backwards compatibility all the way back to 2.6.x. Status: Patch Available (was: Open) Introduce dfs.block.access.token.storageid.enable which will be false by default. When it's turned on, the BlockTokenSecretManager.checkAccess will consider the storage ID when verifying the request. This allows for backwards compatibility all the way back to 2.6.x. > Fix BlockToken compatibility with Hadoop 2.x clients > > > Key: HDFS-11956 > URL: https://issues.apache.org/jira/browse/HDFS-11956 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Ewan Higgs >Priority: Blocker > Attachments: HDFS-11956.001.patch > > > Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. > When talking to a 3.0.0-alpha4 DN with security on: > {noformat} > 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: > Block token verification failed: op=WRITE_BLOCK, > remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs > [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with > StorageIDs [] > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
[ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11956: -- Attachment: HDFS-11956.001.patch Attaching a patch that introduces {{dfs.block.access.token.storageid.enable}} which will be false by default. When it's turned on, the {{BlockTokenSecretManager.checkAccess}} will consider the storage ID when verifying the request. > Fix BlockToken compatibility with Hadoop 2.x clients > > > Key: HDFS-11956 > URL: https://issues.apache.org/jira/browse/HDFS-11956 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.0.0-alpha4 >Reporter: Andrew Wang >Assignee: Chris Douglas >Priority: Blocker > Attachments: HDFS-11956.001.patch > > > Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. > When talking to a 3.0.0-alpha4 DN with security on: > {noformat} > 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: > Block token verification failed: op=WRITE_BLOCK, > remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs > [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with > StorageIDs [] > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org