Stack <stack@...> writes: > > ERROR: Region test,,1296067171940.0200bfe58a9e9fadf8ebfa523c47332f. found on > > server 10.101.45.82:60020 but is listed in META to be on server ip-10-117- 86- > > 81.ec2.internal:60020. > > Could this region have been deployed on this server before you ran > add_table? Or 'test' was a completely new addition.
The region had been deployed, but I dropped the table before I tried to re-add it. > If the latter, then this is a new spin. add_table.rb effectively just > edits .META. Somehow, the assignment of the just-added regions went > awry. Can you grep this region name in your master log? You might be > able to make some sense of what happened; was the region assigned two > places? > > > Now, this table is not at all important to me. However, if I drop it, these two > > regions end up surviving, and hbck gives me a different inconsistency message > > (... not listed in META...). > > > > Yeah. Disable and drop are going by the content in .META. Thats > supposed to be the authority. You could restart these individual > regionservers (or restart cluster)? That'd clean up the mess. When I would stop a single regionserver that was hosting the phantom region, it would just migrate to a different one. I managed to get the warnings to completely go away for a time this morning, by shutting down all of the regionservers. Hbck came back clean. Unfortunately, I then tried loading a large table that I had created as a set of HFiles, and when I ran "loadtable" on them, I was rewarded with 281 regions that are now on regionservers and not in META. > > Is there a way that I can just zot these regions out and make hbase hbck happy > > again? > > Not in 0.89.x. I believe you can send a direct close region to the > individual servers in 0.90 (I'd have to check) so you don't have to do > full regionserver restart. I did find the close_region tool in the shell, but I am not sure that it worked for me. > > How did you make the table? Was it with bulk loader? > The "test" table was made with "puts". My new disaster is an HTable.
