[
https://issues.apache.org/jira/browse/HDFS-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated HDFS-5177:
--------------------------------
Attachment: HDFS-5177-04.patch
Updated the patch.
No change in protocol in latest patch. (Previous patches changed the
clientprotocol).
Please review.
> blocksScheduled count should be decremented for abandoned blocks
> -----------------------------------------------------------------
>
> Key: HDFS-5177
> URL: https://issues.apache.org/jira/browse/HDFS-5177
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0, 2.1.0-beta
> Reporter: Vinayakumar B
> Assignee: Vinayakumar B
> Attachments: HDFS-5177-04.patch, HDFS-5177.patch, HDFS-5177.patch,
> HDFS-5177.patch
>
>
> DatanodeDescriptor#incBlocksScheduled() will be called for all datanodes of
> the block on each allocation. But same should be decremented for abandoned
> blocks.
> When one of the datanodes is down and same is allocated for the block along
> with other live datanodes, then this block will be abandoned, but the
> scheduled count on other datanodes will consider live datanodes as loaded,
> but in reality these datanodes may not be loaded.
> Anyway this scheduled count will be rolled every 20 mins.
> Problem will come if the rate of creation of files is more. Due to increase
> in the scheduled count, there might be chances of missing local datanode to
> write to. and some times writes also can fail in small clusters.
> So we need to decrement the unnecessary count on abandon block call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
