[
https://issues.apache.org/jira/browse/HDFS-13350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419497#comment-16419497
]
Lei (Eddy) Xu commented on HDFS-13350:
--------------------------------------
The legacy negative block ID and EC block IDs are safe in the following cases:
* NN bootstraps from fsimage / edit logs, because NN first checks INode type
then allocate different block type for block IDs in fsimage. So for legacy
block ID, the INode type is normal replication file, and NN does not check
block ID value for such file.
* NN assigns new block IDs for EC file, {{SequentialBlockGroupIdGenerator}}
will check the existence of block Ids in block map, and skip any existed
negative IDs. So it is also safe here.
* HDFS-7994 addressed several cases in {{BlockManager}}.
Other than these, {{isStripedBlockID()}} are mostly used in
{{CorruptReplicasMap}} and {{InvalidateBlocks}}.
Upload the patch to address the remained usages.
> Negative legacy block ID will confuse Erasure Coding to be considered as
> striped block
> --------------------------------------------------------------------------------------
>
> Key: HDFS-13350
> URL: https://issues.apache.org/jira/browse/HDFS-13350
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
> Affects Versions: 3.0.1
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Priority: Major
> Attachments: HDFS-13350.00.patch
>
>
> HDFS-4645 has changed HDFS block ID from randomly generated to sequential
> positive IDs. And later on, HDFS EC was built on the assumption that normal
> 3x replica block IDs are positive, so EC re-use negative IDs as striped
> blocks.
> However, there are legacy block IDs can be negative in the system, we should
> not use hardcode method to check whether a block is stripe or not:
> {code}
> public static boolean isStripedBlockID(long id) {
> return BlockType.fromBlockId(id) == STRIPED;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
