Samer Al-Kiswany created HDFS-6820:
--------------------------------------
Summary: Namenode fails to boot if the file system reorders rename
operations
Key: HDFS-6820
URL: https://issues.apache.org/jira/browse/HDFS-6820
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Samer Al-Kiswany
Priority: Minor
After studying the steps HDFS name node takes to update the logs we found the
following bug. The bug may not manifest in all current file system
implementations, but it is possible in file systems that reorder metadata
operations. e.g. btrfs
Looking at the strace of HDFS name node we see the following when updating the
image:
create(fsimage.chk)
append(fsimage.chk)
fsync(fsimage.chk)
create(fsimage.md5.tmp)
append(fsimage.md5.tmp)
fsync(fsimage.md5.tmp)
rename(fsimage.md5, fsimage.md5.tmp)
rename(fsimage, fsimage.chk)
If the file system reorders the last two rename operations and the system
crashes before the second rename is persisted on disk, the system may end up
with a fsimage that does not have a corresponding md5 file. In this case HDFS
namenode does not boot.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
