[
https://issues.apache.org/jira/browse/HDFS-822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789964#action_12789964
]
dhruba borthakur commented on HDFS-822:
---------------------------------------
This looks like a bug that might be good to fix in 0.21 itself.
On a related note, suppose the last partial block of a file has three replicas
on three different volumes on three different datanodes. And let's suppose that
these volumes (disks) have no more free space on them whereas there is plenty
of free space on other datanodes and disks. Now, if an application re-opens the
file to append to it, the append will fail because there isn't any free space
on those devices (even though the cluster has tons of free space). Is it
possible to avoid this inconsistent behaviour?
> Appends to already-finalized blocks can rename across volumes
> -------------------------------------------------------------
>
> Key: HDFS-822
> URL: https://issues.apache.org/jira/browse/HDFS-822
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Todd Lipcon
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.21.0, 0.22.0
>
>
> This is a performance thing. As I understand the code in FSDataset.append, if
> the block is already finalized, it needs to move it into the RBW directory so
> it can go back into a "being written" state. This is done using
> volumes.getNextVolume without preference to the volume that the block
> currently exists on. It seems to me that this could cause a lot of slow
> cross-volume copies on applications that periodically
> append/close/append/close a file. Instead, getNextVolume could provide an
> alternate form that gives preference to a particular volume, so the rename
> stays on the same disk.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
