[
https://issues.apache.org/jira/browse/HDFS-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195092#comment-13195092
]
Todd Lipcon commented on HDFS-2759:
-----------------------------------
bq. The explanation, per the comment, is that syncing metadata is unnecessary
because of pre-allocation. I don't think that's reasonable, though, since
EditLogFileOutputStream#preallocate doesn't call sync itself, which means that
the file length might never get updated upon returning from
EditLogFileOutputStream#flushAndSync.
This isn't quite true - the difference between {{force(true)}} and
{{force(false)}} is that {{force(false)}} calls {{fdatasync()}}. The man page
says:
{quote}
fdatasync() is similar to fsync(), but does not flush modified metadata
*unless that metadata is needed in order to allow a subsequent data*
*retrieval to be correctly handled*. For example, changes to st_atime or
st_mtime (respectively, time of last access and time of last modifica‐
tion; see stat(2)) do not require flushing because they are not neces‐
sary for a subsequent data read to be handled correctly. *On the other*
*hand, a change to the file size* (st_size, as made by say ftruncate(2)),
*would require a metadata flush*.
{quote}
(emphasis mine)
So I think we're OK here.
> Pre-allocate HDFS edit log files after writing version number
> -------------------------------------------------------------
>
> Key: HDFS-2759
> URL: https://issues.apache.org/jira/browse/HDFS-2759
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, name-node
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HDFS-2759.patch, HDFS-2759.patch
>
>
> In HDFS-2709 it was discovered that there's a potential race wherein edits
> log files are pre-allocated before the version number is written into the
> header of the file. This can cause the NameNode to read an invalid HDFS
> layout version, and hence fail to read the edit log file. We should write the
> header, then pre-allocate the rest of the file after this point.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
