I had about 30,000 rows in my table 'cached_parsedtext'. This morning when I checked, Hbase appeared to be down (master server web UI was not responding and the Shell crashed when I tried to count rows). I tried doing a nice shutdown via bin/stop-hbase, this hung for about 20 minutes though so I gave up and did a kill -9 on the hbase processes (what else was I supposed to do!?). Upon restarting I discovered that all of the rows were gone. I browsed the filesystem and saw that some of the metadata still existed in hadoop dfs. Is there a way to rebuild the table? (After the force kill I also did a nice restart of hbase and hadoop -- same results)
A few of the relevent looking log files are included below for those that speak the language. However, these don't really mean much to me. logs/hbase-pubget-master-carr.domain.com.log:2009-06-18 11:12:42,038 INFO org.apache.hadoop.hba se.master.ServerManager: Received MSG_REPORT_OPEN: cached_parsedtext,,1244838542607: safeMode=false fr om 10.0.16.91:60020 logs/hbase-pubget-master-carr.domain.com.log:2009-06-18 11:12:42,038 INFO org.apache.hadoop.hba se.master.ProcessRegionOpen$1: cached_parsedtext,,1244838542607 open on 10.0.16.91:60020 logs/hbase-pubget-master-carr.domain.com.log:2009-06-18 11:12:42,039 INFO org.apache.hadoop.hba se.master.ProcessRegionOpen$1: updating row cached_parsedtext,,1244838542607 in region .META.,,1 with startcode 1245337882941 and server 10.0.16.91:60020 logs/hbase-pubget-master-carr.domain.com.log:2009-06-18 11:31:31,595 INFO org.apache.hadoop.hba se.master.RegionManager: assigning region cached_parsedtext,,1244838542607 to the only server 10.0.16. 91:60020 logs/hbase-pubget-master-carr.domain.com.log:2009-06-18 11:31:34,823 INFO org.apache.hadoop.hba se.master.ServerManager: Received MSG_REPORT_PROCESS_OPEN: cached_parsedtext,,1244838542607: safeMode= false from 10.0.16.91:60020 Ideally I'd love to get my table back, but if not, learning how to avoid this in the future would be great. Thanks in advance, Mike
