[
https://issues.apache.org/jira/browse/HDFS-12880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273612#comment-16273612
]
Konstantin Shvachko commented on HDFS-12880:
--------------------------------------------
File delete is implemented as a two phase action, with each phase protected by
a separate lock segment. In the first phase NN collects blocks for deletion,
invalidates them, and deletes the file INode. Then it releases the lock. Later
it acquires the lock again and in the second phase removes collected blocks
from BlocksMap.
In a brief moment between the two phases BlocksMap contains abandoned blocks
that do not belong to any files.
We can fix it by redistributing actions between the two phases:
# Collect blocks and INodes to be deleted. Also remove the target INode from
its parent {{removeChild()}}
# Delete collected blocks from BlocksMap and invalidate them, then delete
collected INodes from INodeMap.
This should prevent tools like Fsck and ReplicationMonitor from accessing
abandoned blocks.
> Disallow abandoned blocks in the BlocksMap
> ------------------------------------------
>
> Key: HDFS-12880
> URL: https://issues.apache.org/jira/browse/HDFS-12880
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 2.7.4
> Reporter: Konstantin Shvachko
>
> BlocksMap used to contain only valid blocks, that is belonging to a file. The
> issue is intended to restore this invariant. This was discussed in details
> while fixing HDFS-12638
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
