Dinh Duong Mai created HBASE-14018: -------------------------------------- Summary: RegionServer is aborted when flushing memstore. Key: HBASE-14018 URL: https://issues.apache.org/jira/browse/HBASE-14018 Project: HBase Issue Type: Bug Affects Versions: 1.0.1.1 Environment: CentOS x64 Server Reporter: Dinh Duong Mai
+ Pseudo-distributed Hadoop, ZK_HBASE_MANAGE = true (1 master, 1 regionserver). + Put data to OpenTSDB, 1000 records / s, for 2000 seconds. + RegionServer is aborted. RegionServer logs: 2015-07-03 16:43:27,320 INFO [MemStoreFlusher.1] regionserver.HRegion: Started memstore flush for tsdb,,1435894216453.35f5c254751fef111cdd3788f8465324., current region memstore size 128.12 MB 2015-07-03 16:43:27,955 FATAL [MemStoreFlusher.1] regionserver.HRegionServer: ABORTING region server node2.vmcluster,16040,1435897661557: Replay of WAL required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: tsdb,,1435894216453.35f5c254751fef111cdd3788f8465324. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2001) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1772) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1704) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:445) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:407) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$800(MemStoreFlusher.java:69) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:225) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArrayIndexOutOfBoundsException: -32743 at org.apache.hadoop.hbase.CellComparator.getMinimumMidpointArray(CellComparator.java:478) at org.apache.hadoop.hbase.CellComparator.getMidpoint(CellComparator.java:448) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishBlock(HFileWriterV2.java:165) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.checkBlockBoundary(HFileWriterV2.java:146) at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:263) at org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87) at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:932) at org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:121) at org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:71) at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:879) at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2128) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1955) ... 7 more 2015-07-03 16:43:27,956 FATAL [MemStoreFlusher.1] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint] -- This message was sent by Atlassian JIRA (v6.3.4#6332)