Any comment on this issue? The HFile checksum seems to be a recently added feature. I tried to delete the region causing the problem and insert the data again. But this time another region had compaction failure.
In my case, each row has one column family and each column is ~128kb. I use python thrift api to insert data, with mutation batch set to 100. The entire table has ~4 million rows and I have 3 region servers with 1 master. On Fri, Oct 10, 2014 at 9:55 AM, Steve Fan <[email protected]> wrote: > hbase.regionserver.checksum.verify = true > hbase.hstore.checksum.algorithm = CRC32 > > > On Thu, Oct 9, 2014 at 10:06 PM, Ted Yu <[email protected]> wrote: > >> What're the values for the following config ? >> >> hbase.regionserver.checksum.verify >> hbase.hstore.checksum.algorithm >> >> Cheers >> >> On Thu, Oct 9, 2014 at 6:29 AM, Steve Fan <[email protected]> wrote: >> >> > I'm getting compaction failure for a region after heavy writes. >> > >> > Running hbase hbck -details returns everything is OK. >> > >> > I'm running 0.98.1-cdh5.1.0 >> > >> > ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: >> Compaction >> > failed Request = >> > regionName=table,rowkey,1409072707535.b3481b3baef0fdc711b178caf6a6072a., >> > storeName=data, fileCount=3, fileSize=474.5 M (129.4 M, 214.8 M, 130.3 >> M), >> > priority=1, time=8587702075256007 >> > java.lang.IndexOutOfBoundsException >> > at java.nio.ByteBuffer.wrap(ByteBuffer.java:371) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFileBlock.getBufferReadOnly(HFileBlock.java:343) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.ChecksumUtil.validateBlockChecksum(ChecksumUtil.java:150) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.validateBlockChecksum(HFileBlock.java:1573) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1509) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1314) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:355) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:605) >> > at >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:719) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:136) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:108) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:507) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:217) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:76) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:109) >> > at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1080) >> > at >> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482) >> > at >> > >> > >> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:475) >> > at >> > >> > >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> > at >> > >> > >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> > at java.lang.Thread.run(Thread.java:745) >> > >> > >
