[
https://issues.apache.org/jira/browse/HDFS-9789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated HDFS-9789:
----------------------------
Status: Patch Available (was: Open)
> Correctly update DataNode's scheduled block size when writing small EC file
> ---------------------------------------------------------------------------
>
> Key: HDFS-9789
> URL: https://issues.apache.org/jira/browse/HDFS-9789
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-9789.000.patch
>
>
> Currently DatanodeDescriptor's scheduled block size is increased when
> allocating a new block. It is then decreased when NN receives IBR from the DN.
> When writing an erasure-coded file whose size is less a stripe, we can have
> less than 6 internal data blocks. Then we never decrease the scheduled block
> size for Datanodes that are actually not used for storing any internal block.
> When writing large amount of small EC files within a short period, this may
> cause NN to wrongly exclude DN for writing due to high scheduled block size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
