Hi,
I have noticed that my hbase client is complaining that
Trying to contact region server m0002036.ppops.net:60020 for region
HEARTBEAT_CLUSTER,,1279930490584.a84b7160318e91ef51c7efd215cd7e46., row
'time\x09weekly\x092010W30', but failed after 10 attempts.
I then go to the hbase master web page and it shows that this region is
there and served by one of the region server. however,
I clicked on the link for that regionserver and found that this region is
not listed on the served regions. I then searched the log on this region
server and I found the regionserver actually closed this region:
2010-09-13 13:16:58,602 INFO org.apache.hadoop.hbase.regionserver.Store:
Complet
ed major compaction of 4 file(s) in fields of
HEARTBEAT_CLUSTER,,1279930490584.a
84b7160318e91ef51c7efd215cd7e46.; new storefile is
hdfs://d-namenodes1.cloud.ppo
ps.net:8020/hbase/HEARTBEAT_CLUSTER/a84b7160318e91ef51c7efd215cd7e46/fields/8534
678064854849666; store size is 102.6m
2010-09-13 13:16:58,602 INFO org.apache.hadoop.hbase.regionserver.HRegion:
compa
ction completed on region
HEARTBEAT_CLUSTER,,1279930490584.a84b7160318e91ef51c7e
fd215cd7e46. in 46sec
2010-09-13 16:24:31,981 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionServer
: Closing region
HEARTBEAT_CLUSTER,,1279930490584.a84b7160318e91ef51c7efd215cd7e
46.
2010-09-13 16:24:31,982 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Clos
ing HEARTBEAT_CLUSTER,,1279930490584.a84b7160318e91ef51c7efd215cd7e46.:
disablin
g compactions & flushes
2010-09-13 16:24:32,028 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Upda
tes disabled for region, no outstanding scanners on
HEARTBEAT_CLUSTER,,127993049
0584.a84b7160318e91ef51c7efd215cd7e46.
2010-09-13 16:24:32,031 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
No m
ore row locks outstanding on region
HEARTBEAT_CLUSTER,,1279930490584.a84b7160318
e91ef51c7efd215cd7e46.
2010-09-13 16:24:32,057 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Close
d HEARTBEAT_CLUSTER,,1279930490584.a84b7160318e91ef51c7efd215cd7e46.
That means nobody can access this table any more. How could this happen ? I
checked HDFS and confirmed the files for this
table is there. I am using the CDH3 beta 2 hbase 0.89 version. I end up
having to restart all regionservers to get this region online.
Jimmy.
