[
https://issues.apache.org/jira/browse/HDFS-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinay updated HDFS-5177:
------------------------
Attachment: HDFS-5177.patch
Attaching the patch.
Please review
> blocksScheduled count should be decremented for abandoned blocks
> -----------------------------------------------------------------
>
> Key: HDFS-5177
> URL: https://issues.apache.org/jira/browse/HDFS-5177
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0, 2.1.0-beta
> Reporter: Vinay
> Attachments: HDFS-5177.patch
>
>
> DatanodeDescriptor#incBlocksScheduled() will be called for all datanodes of
> the block on each allocation. But same should be decremented for abandoned
> blocks.
> When one of the datanodes is down and same is allocated for the block along
> with other live datanodes, then this block will be abandoned, but the
> scheduled count on other datanodes will consider live datanodes as loaded,
> but in reality these datanodes may not be loaded.
> Anyway this scheduled count will be rolled every 20 mins.
> Problem will come if the rate of creation of files is more. Due to increase
> in the scheduled count, there might be chances of missing local datanode to
> write to. and some times writes also can fail in small clusters.
> So we need to decrement the unnecessary count on abandon block call.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
