[
https://issues.apache.org/jira/browse/HDFS-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931234#comment-13931234
]
Todd Lipcon commented on HDFS-6038:
-----------------------------------
I didn't look at the code in detail, but the design approach of
length-prefixing the edits seems reasonable. My only feedback might be that it
would have been nicer to do that change in a JIRA labeled as such, and then
make the JN change separately. But I'm not against doing it all here -- just
worried that other contributors may want to review this patch as it's actually
making an edit log format change, not just a protocol change for the JNs.
I also noticed a spot or two where you are missing a finally {
IOUtils.closeStream(...); } -- might be worth checking for that before
committing.
In terms of testing, it might be nice to add a QJM test which writes "fake" ops
to a JournalNode -- ie ops with garbage data but a correct length and checksum.
Perhaps you could do this by changing QJMTestUtil.writeOp() to write a garbage
op?
> JournalNode hardcodes NameNodeLayoutVersion in the edit log file
> ----------------------------------------------------------------
>
> Key: HDFS-6038
> URL: https://issues.apache.org/jira/browse/HDFS-6038
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: journal-node, namenode
> Reporter: Haohui Mai
> Assignee: Jing Zhao
> Attachments: HDFS-6038.000.patch, HDFS-6038.001.patch,
> HDFS-6038.002.patch, HDFS-6038.003.patch, HDFS-6038.004.patch,
> HDFS-6038.005.patch, HDFS-6038.006.patch, HDFS-6038.007.patch, editsStored
>
>
> In HA setup, the JNs receive edit logs (blob) from the NN and write into edit
> log files. In order to write well-formed edit log files, the JNs prepend a
> header for each edit log file.
> The problem is that the JN hard-codes the version (i.e.,
> {{NameNodeLayoutVersion}} in the edit log, therefore it generates incorrect
> edit logs when the newer release bumps the {{NameNodeLayoutVersion}} during
> rolling upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
