[
https://issues.apache.org/jira/browse/HDFS-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132863#comment-15132863
]
Jing Zhao commented on HDFS-9754:
---------------------------------
Thanks for the review, [~vinayrpet] and [~szetszwo]!
bq. INodeFile#collectBlocksBeyondMax(..) and related changes are not exactly
related to this Jira
This change can help us get rid of {{BlocksMapUpdateInfo#removeDeleteBlock}}.
Without the change when doing truncation on a snapshotted file, the current
code first collects blocks that will be truncated, then use the latest snapshot
to check if any block should be retained. In this way we cannot assign {{bcId}}
to {{INVALID_INODE_ID}} when adding block into {{BlocksMapUpdateInfo}}.
bq. Condition A and Condition B are different
Thanks for the analysis, Nicholas. So looks like when we add a block into the
under-replicated queue, we always make sure the block is completed. Then later
if the block is no longer in complete/committed state, it must have been
appended or truncated. Then this block should be the last block of the file,
and the block and the corresponding file should now be tracked by file lease.
Thus looks like we then do not need to schedule recovery for this block.
In the meanwhile, condition A has not considered the case where the file is
appended on a new block (i.e., the variable length block support). So I think
we should check the block's state instead of the file's state. Does this make
sense to you, Nicholas?
> Avoid unnecessary getBlockCollection calls in BlockManager
> ----------------------------------------------------------
>
> Key: HDFS-9754
> URL: https://issues.apache.org/jira/browse/HDFS-9754
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-9754.000.patch
>
>
> Currently BlockManager calls {{Namesystem#getBlockCollection}} in order to:
> 1. check if the block has already been abandoned
> 2. identify the storage policy of the block
> 3. meta save
> For #1 we can use BlockInfo's internal state instead of checking if the
> corresponding file still exists.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
