Hello, I have started to test HBase and Hive-HBase integration in our test Hadoop cluster last week. Everything was working fine until both HDFS and HBase crashed because of a space issue. We fixed the space issue and brought to HDFS back up. But HBase was not happy. The HBase data was not important. So, I removed /hbase directory from HDFS to start from scratch. However things are not looking nice right now:
* "create" from shell errors with "NotAllMetaRegionsOnlineException" (but status shows 3 servers, 0 dead, 0.0000 average load) * "hbck" shows "ERROR: Root Region or some of its attributes is null. ERROR: Encountered fatal error. Exitting..." * Master web interface tries to load the page for a long time, shows nothing. * RegionServer web interface shows normal output with "Not serving regions" under Online Regions. We have 4 node test Hadoop cluster, 3 HBase servers, running CDH3.u0 (HBase version 0.90.1-cdh3u0). All three are running Zookeeper, HBase Master and RegionServer (I start them in that order). The logs don't show any error. Masters keep reporting this every 3-10 seconds: 2011-06-15 17:21:56,809 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: Lookedup root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@367c218e; hsa=null RegionServers keep reporting this every 5 minutes: 2011-06-15 17:15:01,194 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=6.68 MB, free=807.12 MB, max=813.8 MB, blocks=0, accesses=0, hits=0, hitRatio=�%, cachingAccesses=0, cachingHits=0, cachingHitsRatio=�%, evictions=0, evicted=0, evictedPerRun=NaN What else do I need to do to start from scratch other than just removing /hbase folder and restarting deamons? Thank you!
