[
https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189216#comment-14189216
]
Konstantin Shvachko commented on HDFS-3107:
-------------------------------------------
Yes, good point, Nicholas.
During upgrade old blocks are hard linked and the hard links are stored in a
separate directory, which allows the blocks themselves be deleted while the
file system is updated. So when we roll back the deleted blocks are still
available via those hard links.
With truncate the same is applied to the blocks that were deleted. For the
blocks that are truncated to a smaller length we will need to do
copy-on-truncate recovery, same as we do for snapshots. That is if upgrade is
in progress NN will schedule copy-on-truncate wether snapshots are present or
not.
The bottom line is that implementing copy-on-truncate is needed both for
snapshots and upgrades. I'll make a note to update the design.
> HDFS truncate
> -------------
>
> Key: HDFS-3107
> URL: https://issues.apache.org/jira/browse/HDFS-3107
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, namenode
> Reporter: Lei Chang
> Assignee: Plamen Jeliazkov
> Attachments: HDFS-3107.008.patch, HDFS-3107.patch, HDFS-3107.patch,
> HDFS-3107.patch, HDFS-3107.patch, HDFS-3107.patch, HDFS-3107.patch,
> HDFS-3107.patch, HDFS_truncate.pdf, HDFS_truncate.pdf,
> HDFS_truncate_semantics_Mar15.pdf, HDFS_truncate_semantics_Mar21.pdf,
> editsStored, editsStored, editsStored.xml
>
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> Systems with transaction support often need to undo changes made to the
> underlying storage when a transaction is aborted. Currently HDFS does not
> support truncate (a standard Posix operation) which is a reverse operation of
> append, which makes upper layer applications use ugly workarounds (such as
> keeping track of the discarded byte range per file in a separate metadata
> store, and periodically running a vacuum process to rewrite compacted files)
> to overcome this limitation of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
