[
https://issues.apache.org/jira/browse/HDFS-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2759:
---------------------------------
Attachment: HDFS-2759.patch
bq. I remember you ran a benchmark at some point to check for edit log
throughput - if you have that around still would you mind re-running to make
sure this doesn't cause any unforeseen regression?
I just re-ran the benchmark, and saw no performance regressions. In fact,
pre-allocation ops and the slowest 0.01% of ops seem to have actually sped up a
little bit, since we can now return from the flushAndSync call without waiting
for the pre-allocation data to be synced to disk.
Here's a (very slightly) updated patch which is identical to the last, except
that it also fixes up an incorrect comment justifying why we don't need to
force the syncing of metadata when logging an op.
> Pre-allocate HDFS edit log files after writing version number
> -------------------------------------------------------------
>
> Key: HDFS-2759
> URL: https://issues.apache.org/jira/browse/HDFS-2759
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, name-node
> Affects Versions: 0.24.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: HDFS-2759.patch, HDFS-2759.patch, HDFS-2759.patch
>
>
> In HDFS-2709 it was discovered that there's a potential race wherein edits
> log files are pre-allocated before the version number is written into the
> header of the file. This can cause the NameNode to read an invalid HDFS
> layout version, and hence fail to read the edit log file. We should write the
> header, then pre-allocate the rest of the file after this point.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
