[
https://issues.apache.org/jira/browse/HDFS-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445179#comment-13445179
]
Colin Patrick McCabe commented on HDFS-3540:
--------------------------------------------
bq. Before HDFS-3521, there is a UNCHECKED_REGION_LENGTH in Recovery Mode. If a
stray OP_INVALID byte is within the unchecked region, it will cause silent data
loss.
Nicholas, you didn't address the main point of my comment, which is that after
HDFS-3521, if a stray OP_INVALID byte is found anywhere in the log, it will
cause silent data loss-- unless the sysadmin configures
{{dfs.namenode.edits.toleration.length}} to something other than the default.
Based on your earlier comments, I think we both agree that this should not be
the default. Let's fix this (independently of everything else were discussing
here.)
bq. You still do not know the corruption length since there may be padding at
the end. System admins won't know the padding length and so they won't be able
to know the corruption length.
The padding length is going to be a megabyte at most. Since the edit log files
are fairly large, you should have a good idea of what percentage through the
file you are. If you have an idea for improving the error messages of
{{FSEditLog.java}}, perhaps you should file a JIRA for that? It's not directly
relevant here, though, since all methods of manual recovery will face the same
issues.
bq. Before HDFS-3521, there is a UNCHECKED_REGION_LENGTH in Recovery Mode...
I want to emphasize one thing here: {{UNCHECKED_REGION_LENGTH}} is *not* part
of Recovery Mode. If you look at the history {{FSEditLog.java}}, you'll see
that change 1325075 (HDFS-3055) introduced Recovery mode, but not
{{UNCHECKED_REGION_LENGTH}}. This was introduced in HDFS-3479 (the backport of
HDFS_3335 to branch-1). Please see this comment, introduced by HDFS-3479:
{code}
+ /** The end of the edit log should contain only 0x00 or 0xff bytes.
+ * If it contains other bytes, the log itself may be corrupt.
+ * It is important to check this; if we don't, a stray OP_INVALID byte
+ * could make us stop reading the edit log halfway through, and we'd never
+ * know that we had lost data.
+ *
+ * We don't check the very last part of the edit log, in case the
+ * NameNode crashed while writing to the edit log.
+ */
{code}
I encourage anyone interested in this to check out the history of
{{FSEditLog.java}}. It's a very good guide and it will make understanding this
discussion much easier.
As I said before, I still think we should get rid of the unchecked region
altogether. But this has nothing to do with Recovery Mode, it has to do with
HDFS-3479.
> Further improvement on recovery mode and edit log toleration in branch-1
> ------------------------------------------------------------------------
>
> Key: HDFS-3540
> URL: https://issues.apache.org/jira/browse/HDFS-3540
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 1.2.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
>
> *Recovery Mode*: HDFS-3479 backported HDFS-3335 to branch-1. However, the
> recovery mode feature in branch-1 is dramatically different from the recovery
> mode in trunk since the edit log implementations in these two branch are
> different. For example, there is UNCHECKED_REGION_LENGTH in branch-1 but not
> in trunk.
> *Edit Log Toleration*: HDFS-3521 added this feature to branch-1 to remedy
> UNCHECKED_REGION_LENGTH and to tolerate edit log corruption.
> There are overlaps between these two features. We study potential further
> improvement in this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
