[
https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041207#comment-16041207
]
Andrew Wang commented on HDFS-9807:
-----------------------------------
I believe this (or HDFS-6708) broke backwards compatibility with older clients.
I'm running a 2.6.0-ish client against a 3.0.0-alpha4-ish DN, and am seeing
this in the DN log:
{noformat}
2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:
Block token verification failed: op=WRITE_BLOCK,
remoteAddress=/172.28.208.200:53900, message=Block token with StorageIDs
[DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with StorageIDs
[]
{noformat}
The client then blacklists each DN when it gets this error, until it runs out
of DNs. Dunno if this retry behavior is fixed in newer clients;
InvalidBlockTokenExceptions are kind of fatal.
What is the expected behavior here?
> Add an optional StorageID to writes
> -----------------------------------
>
> Key: HDFS-9807
> URL: https://issues.apache.org/jira/browse/HDFS-9807
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 3.0.0-alpha2
> Reporter: Chris Douglas
> Assignee: Ewan Higgs
> Fix For: 3.0.0-alpha4
>
> Attachments: HDFS-9807.001.patch, HDFS-9807.002.patch,
> HDFS-9807.003.patch, HDFS-9807.004.patch, HDFS-9807.005.patch,
> HDFS-9807.006.patch, HDFS-9807.007.patch, HDFS-9807.008.patch,
> HDFS-9807.009.patch, HDFS-9807.010.patch
>
>
> The {{BlockPlacementPolicy}} considers specific storages, but when the
> replica is written the DN {{VolumeChoosingPolicy}} is unaware of any
> preference or constraints from other policies affecting placement. This
> limits heterogeneity to the declared storage types, which are treated as
> fungible within the target DN. It should be possible to influence or
> constrain the DN policy to select a particular storage.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
