[ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14198856#comment-14198856 ]
Stephen Yuan Jiang commented on HBASE-11625: -------------------------------------------- The fix in HBASE-5885 stated very clear about what the issue is (an Hadoop bug that I think still not fixed, at least not fixed fully - not sure whether there is a ticket opened for it): {code} diff --git c/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java w/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java index d6a4705..7e3a68c 100644 + // If this is the local file system hadoop has a bug where seeks + // do not go to the correct location if setVerifyChecksum(false) is called. + // This manifests itself in that incorrect data is read and HFileBlocks won't be able to read + // their header magic numbers. See HBASE-5885 + if (useHBaseChecksum && !(fs instanceof LocalFileSystem)) { {code} However, it looks like HBASE-11218 re-introduced the issue: {code} diff --git hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java index 8178787..f8cf7b3 100644 @@ -81,6 +81,13 @@ public class HFileSystem extends FilterFileSystem { this.useHBaseChecksum = useHBaseChecksum; fs.initialize(getDefaultUri(conf), conf); + + // disable checksum verification for local fileSystem, see HBASE-11218 + if (fs instanceof LocalFileSystem) { + fs.setWriteChecksum(false); + fs.setVerifyChecksum(false); + } {code} > Reading datablock throws "Invalid HFile block magic" and can not switch to > hdfs checksum > ----------------------------------------------------------------------------------------- > > Key: HBASE-11625 > URL: https://issues.apache.org/jira/browse/HBASE-11625 > Project: HBase > Issue Type: Bug > Components: HFile > Affects Versions: 0.94.21, 0.98.4, 0.98.5 > Reporter: qian wang > Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz > > > when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it > could happen file corruption but it only can switch to hdfs checksum > inputstream till validateBlockChecksum(). If the datablock's header corrupted > when b = new HFileBlock(),it throws the exception "Invalid HFile block magic" > and the rpc call fail -- This message was sent by Atlassian JIRA (v6.3.4#6332)