[
https://issues.apache.org/jira/browse/HDFS-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093483#comment-14093483
]
Colin Patrick McCabe commented on HDFS-6820:
--------------------------------------------
Hmm, good find. You can fix this problem by calling fsync on the directory
that contains both files after the first rename (but before the second).
Unfortunately, this requires JNI.
> Namenode fails to boot if the file system reorders rename operations
> --------------------------------------------------------------------
>
> Key: HDFS-6820
> URL: https://issues.apache.org/jira/browse/HDFS-6820
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Samer Al-Kiswany
> Priority: Minor
>
> After studying the steps HDFS name node takes to update the logs we found the
> following bug. The bug may not manifest in all current file system
> implementations, but it is possible in file systems that reorder metadata
> operations. e.g. btrfs
> Looking at the strace of HDFS name node we see the following when updating
> the image:
> create(fsimage.chk)
> append(fsimage.chk)
> fsync(fsimage.chk)
> create(fsimage.md5.tmp)
> append(fsimage.md5.tmp)
> fsync(fsimage.md5.tmp)
> rename(fsimage.md5, fsimage.md5.tmp)
> rename(fsimage, fsimage.chk)
> If the file system reorders the last two rename operations and the system
> crashes before the second rename is persisted on disk, the system may end up
> with a fsimage that does not have a corresponding md5 file. In this case HDFS
> namenode does not boot.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
