[
https://issues.apache.org/jira/browse/HDFS-6898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109898#comment-14109898
]
Colin Patrick McCabe commented on HDFS-6898:
--------------------------------------------
Nice idea. I will wait for the revised version of this patch.
bq. A related question is whether DN should pre-allocate disk space for
received RBW blocks. If the underlying local FS supports fallocate, this will
improve the disk layout of the block.
We certainly don't want to call {{fallocate}}, since that will result in
hundreds of megs of unnecessary disk I/O. We could call {{ftruncate}}, but
this would get confusing if we didn't end up writing the full block. Currently
I believe there are some places assuming that the block size "means
something".. i.e. that we don't just set all block sizes to 128 MB and use
sparse files. So it would be pretty complex to reverse that assumption.
I doubt there would be much performance gain since we're already doing big
sequential writes, a good case for ext4, etc.
> DN must reserve space for a full block when an RBW block is created
> -------------------------------------------------------------------
>
> Key: HDFS-6898
> URL: https://issues.apache.org/jira/browse/HDFS-6898
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 2.5.0
> Reporter: Gopal V
> Assignee: Arpit Agarwal
> Attachments: HDFS-6898.01.patch, HDFS-6898.03.patch
>
>
> DN will successfully create two RBW blocks on the same volume even if the
> free space is sufficient for just one full block.
> One or both block writers may subsequently get a DiskOutOfSpace exception.
> This can be avoided by allocating space up front.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
