[
https://issues.apache.org/jira/browse/HDFS-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458310#comment-13458310
]
Colin Patrick McCabe commented on HDFS-3954:
--------------------------------------------
The way the edit log is written out in binary format has changed over time.
For instance, since {{Feature.EDITS_CHECKSUM}} (version negative 28), we have
checksums; prior to that, we didn't. Similarly with {{Feature.STORED_TXIDS}},
etc. So I don't think it makes sense to write out the edit log in the latest
format, and slap the version number of an older format on it-- which is what
you seem to be suggesting in your JIRA description.
I don't think there is a perfect solution for this problem. If you want to
edit log in a particular format, the easiest thing to do is probably to keep
around a version of Hadoop that corresponds to that format and run oev from
there.
We could try to support writing old-style edit logs in the new code, but I
think that this would bitrot extremely quickly-- it's difficult enough to keep
bugs out of the latest and greatest edit log format, let alone every format
that's ever existed.
It might be reasonable to detect that we're writing something that is invalid
in the new format, and throw an exception, as opposed to silently writing out a
new-format log which has odd stuff in it. What problem in particular did you
encounter?
> OEV can generate a corrupt edits log
> ------------------------------------
>
> Key: HDFS-3954
> URL: https://issues.apache.org/jira/browse/HDFS-3954
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node, tools
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> The offline edits viewer can output in various formats such as xml and
> binary. Xml output correctly preserves the input's layout version. Binary
> output always writes the latest layout version into the header. Converting
> an older layout to xml and back may result in a corrupt edits log.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
