[
https://issues.apache.org/jira/browse/HBASE-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662505#action_12662505
]
Andrew Purtell commented on HBASE-1115:
---------------------------------------
I've seen the same thing after HBASE-1104. This is a broad symptom of
compaction failure I suspect.
> Regions failed compaction left in open and writable state but are not
> scannable
> -------------------------------------------------------------------------------
>
> Key: HBASE-1115
> URL: https://issues.apache.org/jira/browse/HBASE-1115
> Project: Hadoop HBase
> Issue Type: Bug
> Environment: apurtell cluster, HBase TRUNK on hadoop 0.18 branch
> Reporter: Andrew Purtell
> Priority: Critical
>
> After an instance of HBASE-1114, where compaction fails, the affected region
> is left in an open and writable state, but scanners fail construction. Later
> a manual reassignment via close_region brings the region all the way back up.
> Line numbers are for 0.19-on-hadoop-0.18 branch, sorry about that. I'm
> upgrading to Hadoop 0.19 (finally) this afternoon.
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
> region server 10.30.94.50:60020 for region
> content,c84bbfc94b2143e41ba119d159be2958,1231518442461, row
> 'c84bbfc94b2143e41ba119d159be2958', but failed after 10 attempts.
> Exceptions:
> java.io.IOException: java.io.IOException: HStoreScanner failed construction
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.(StoreFileScanner.java:70)
> at
> org.apache.hadoop.hbase.regionserver.HStoreScanner.(HStoreScanner.java:84)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2119)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.(HRegion.java:1878)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1162)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1673)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
> Caused by: java.io.FileNotFoundException: File does not exist:
> hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/1707725801/url/mapfiles/7039742044868774100/data
> at
> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:695)
> at
> org.apache.hadoop.hbase.io.SequenceFile$Reader.(SequenceFile.java:1431)
> at
> org.apache.hadoop.hbase.io.SequenceFile$Reader.(SequenceFile.java:1426)
> at
> org.apache.hadoop.hbase.io.MapFile$Reader.createDataFileReader(MapFile.java:310)
> at
> org.apache.hadoop.hbase.io.HBaseMapFile$HBaseReader.createDataFileReader(HBaseMapFile.java:96)
> at org.apache.hadoop.hbase.io.MapFile$Reader.open(MapFile.java:292)
> at
> org.apache.hadoop.hbase.io.HBaseMapFile$HBaseReader.(HBaseMapFile.java:79)
> at
> org.apache.hadoop.hbase.io.BloomFilterMapFile$Reader.(BloomFilterMapFile.java:65)
> at
> org.apache.hadoop.hbase.io.HalfMapFileReader.(HalfMapFileReader.java:86)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:438)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:96)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.(StoreFileScanner.java:67)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
