[
https://issues.apache.org/jira/browse/HDFS-5995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918763#comment-13918763
]
Colin Patrick McCabe commented on HDFS-5995:
--------------------------------------------
Thanks for the clarification, [~cnauroth].
bq. I believe Haohui's argument is that there is very little done in the way
of validating the actual data contained within an op before we try to apply it.
This goes beyond just checking the number of bytes in the op... Perhaps it's
good to try to prevent this one possible corruption case like Colin suggested,
even if we can't cover every possible kind of corruption
It's not about preventing all possible kinds of corruption. It's about
preventing the NN and JN from crashing when they see bad data.
Why doesn't the CRC check for each opcode catch this problem?
> TestFSEditLogLoader#testValidateEditLogWithCorruptBody gets OutOfMemoryError
> and dumps heap.
> --------------------------------------------------------------------------------------------
>
> Key: HDFS-5995
> URL: https://issues.apache.org/jira/browse/HDFS-5995
> Project: Hadoop HDFS
> Issue Type: Test
> Components: namenode, test
> Affects Versions: 3.0.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Priority: Minor
> Attachments: HDFS-5995.1.patch
>
>
> {{TestFSEditLogLoader#testValidateEditLogWithCorruptBody}} is experiencing
> {{OutOfMemoryError}} and dumping heap since the merge of HDFS-4685. This
> doesn't actually cause the test to fail, because it's a failure test that
> corrupts an edit log intentionally. Still, this might cause confusion if
> someone reviews the build logs and thinks this is a more serious problem.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
