[jira] [Commented] (HDFS-140) When a file is deleted, its blocks remain in the blocksmap till the next block report from Datanode

[Uma Maheswara Rao G (Commented) (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HDFS-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125663#comment-13125663
 ] 

Uma Maheswara Rao G commented on HDFS-140:
------------------------------------------

Hi Todd,

 Thanks a lot for taking a look!

I have verified this in trunk codebase. this issue has been addressed already.

{code}
 public void removeBlock(Block block) {
    block.setNumBytes(BlockCommand.NO_ACK);
    addToInvalidates(block);
    corruptReplicas.removeFromCorruptReplicasMap(block);
    blocksMap.removeBlock(block);
  }
{code}

FsNameSystem#removePathAndBlock will invoke this API. This is making sure that, 
block has been removed from blocksMap.
It looks to me that 0.21 onwards, this issue has been fixed. But i did not get 
exact JIRA specific to this problem.

But I am using 20.2 version. When i profiled, i found that blockMap elements 
are growing. Observation is that, after blockreports, that has been come down.

So,By looking at the code, current versions in 20X( *20.205,20.206* ) also has 
this problem.

Thanks
Uma


                
> When a file is deleted, its blocks remain in the blocksmap till the next 
> block report from Datanode
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-140
>                 URL: https://issues.apache.org/jira/browse/HDFS-140
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.20.1
>            Reporter: dhruba borthakur
>            Assignee: Uma Maheswara Rao G
>             Fix For: 0.20.205.0
>
>         Attachments: HDFS-140.20security205.patch
>
>
> When a file is deleted, the namenode sends out block deletions messages to 
> the appropriate datanodes. However, the namenode does not delete these blocks 
> from the blocksmap. Instead, the processing of the next block report from the 
> datanode causes these blocks to get removed from the blocksmap.
> If we desire to make block report processing less frequent, this issue needs 
> to be addressed. Also, this introduces indeterministic behaviout to a a few 
> unit tests. Another factor to consider is to ensure that duplicate block 
> detection is not compromised.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira