[
https://issues.apache.org/jira/browse/HBASE-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442917#comment-13442917
]
Hudson commented on HBASE-6586:
-------------------------------
Integrated in HBase-TRUNK #3287 (See
[https://builds.apache.org/job/HBase-TRUNK/3287/])
HBASE-6586 Quarantine Corrupted HFiles with hbck (Revision 1377941)
Result = SUCCESS
jmhsieh :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/DoNotRetryIOException.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HBaseIOException.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CorruptHFileException.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/HFileCorruptionChecker.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestFixedFileTrailer.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java
> Quarantine Corrupted HFiles with hbck
> -------------------------------------
>
> Key: HBASE-6586
> URL: https://issues.apache.org/jira/browse/HBASE-6586
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.92.1
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.92.2, 0.96.0, 0.94.3
>
> Attachments: 0001-hbase-6568-hbck-quarantine-v6.patch,
> hbase-6586-92-v3.patch, hbase-6586-92-v8a.patch, hbase-6586-92-v8b.patch,
> hbase-6586-92-v8.patch, hbase-6586-92-v9.patch, hbase-6586-94-v3.patch,
> hbase-6586-94-v9.patch, hbase-6586.patch, hbase-6586-trunk-v3.patch,
> hbase-6586-trunk-v4.patch, hbase-6586-trunk-v5.patch,
> hbase-6586-trunk-v6.patch, hbase-6586-trunk-v7.patch,
> hbase-6586-trunk-v8.patch, hbase-6586-trunk-v9.patch
>
>
> We've encountered a few upgrades from 0.90 hbases + 20.2/1.x hdfs to 0.92
> hbases + hdfs 2.x that get stuck. I haven't been able to duplicate the
> problem in my dev environment but we suspect this may be related to
> HDFS-3731. On the HBase side, it seems reasonable to quarantine what are
> most likely truncated hfiles, so that can could later be recovered.
> Here's an example of the exception we've encountered:
> {code}
> 2012-07-18 05:55:01,152 ERROR handler.OpenRegionHandler
> (OpenRegionHandler.java:openRegion(346)) - Failed open of
> region=user_mappings,080112102AA76EF98197605D341B9E6C5824D2BC|1001,1317824890618.eaed0e7abc6d27d28ff0e5a9b49c4c
> 0d.
> java.io.IOException: java.lang.IllegalArgumentException: Invalid HFile
> version: 842220600 (expected to be between 1 and 2)
> at
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:306)
>
> at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:371)
> at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:387)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1026)
>
> at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:485)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:566)
>
> at org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:286)
> at org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:223)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2534)
>
> at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:454)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3282)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3230)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:331)
> at
> org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:107)
> at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.IllegalArgumentException: Invalid HFile version:
> 842220600 (expected to be between 1 and 2)
> at org.apache.hadoop.hbase.io.hfile.HFile.checkFormatVersion(HFile.java:515)
> at
> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:303)
>
> ... 17 more
> {code}
> Specifically -- the FixedFileTrailer are incorrect, and seemingly missing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
