Stop and start hbase. Watch the master log as it starts up.
Try to figure why it is not judging regions that have the old server IPs as bad. Enable DEBUG before you restart. The extra info might help (see FAQ on wiki for how). St.Ack On Fri, Feb 27, 2009 at 8:00 PM, Liu Yan <[email protected]> wrote: > When I do "scan '.META.'", I see some interesting output: > > {{{ > 1002_profiles,7139226398444 column=info:server, timestamp=1235789710023, > value=10.254.51.127:60020 > 3021,1235657605714 > > 1002_profiles,7139226398444 column=info:serverstartcode, > timestamp=1235789710023, value=1235789499358 > 3021,1235657605714 > > 1002_profiles,7399192338534 column=historian:assignment, > timestamp=1235789558647, value=Region assigned to se > 9818,1235657605714 rver 10.254.51.127:60020 > > 1002_profiles,7399192338534 column=historian:open, > timestamp=1235789577850, > value=Region opened on server : h > 9818,1235657605714 master > }}} > > The IP address here is correct, pointing to the new master's IP. > > But I also see the following: > > {{{ > 1002_profiles,7399192338534 column=info:server, timestamp=1235789577848, > value=10.254.51.127:60020 > 9818,1235657605714 > > 1002_profiles,7399192338534 column=info:serverstartcode, > timestamp=1235789577848, value=1235789499358 > 9818,1235657605714 > > 1002_profiles,7572817158818 column=historian:assignment, > timestamp=1235297600858, value=Region assigned to se > 3981,1235242656324 rver 10.249.190.85:60020 > > 1002_profiles,7572817158818 column=historian:open, > timestamp=1235297623082, > value=Region opened on server : h > 3981,1235242656324 master > }}} > > This is the IP of our old master's. > > How to fix this? > > Regards, > Yan > > 2009/2/28 stack <[email protected]> > > > If scan is working, do 'scan ".META."'. > > > > There are three columns: info:regioninfo, info:serverstartcode, and > > info:server. > > > > What do you see for info:server? New addresses or the old? > > > > On startup, hbase should be judging the content of .META. as sour and > > reassigning regions to the servers that have just registered; i.e. those > of > > the new addresses. > > > > St.Ack > > > > > > On Fri, Feb 27, 2009 at 7:15 PM, Liu Yan <[email protected]> wrote: > > > > > hi, > > > > > > We have a 4-node cluster Hadoop 0.19.0 and HBase 0.19.0. We run > NameNode > > > and > > > RegionServer on the same server and created a bunch of tables on HBase. > > > > > > Now we want to use another (more powerful) machine to replace the old > > > master. Here is what we did: > > > > > > 1) Shutdown HBase and Hadoop > > > 2) Copy all the Hadoop related files from the old master to the new > > master. > > > 3) Re-configure the Hadoop and HBase so all (including the master and > > > clients) are now pointing to the new master. > > > 4) Start the Hadoop cluster. (This seems fine). > > > 5) Start the HBase cluster. (This seems fine too). > > > > > > Then when we try to do a "count" in HBase shell, (e.g. count > > 'table_name'), > > > we hit the following problem: > > > > > > 09/02/27 21:53:04 INFO ipc.HBaseClass: Retrying connect to server: / > > > 10.249.190.85:60020. Already tried 0 time(s). > > > 09/02/27 21:53:05 INFO ipc.HBaseClass: Retrying connect to server: / > > > 10.249.190.85:60020. Already tried 1 time(s). > > > 09/02/27 21:53:06 INFO ipc.HBaseClass: Retrying connect to server: / > > > 10.249.190.85:60020. Already tried 2 time(s). > > > 09/02/27 21:53:06 INFO ipc.HbaseRPC: Server at /10.249.190.85:60020not > > > available yet, Zzzzz... > > > 09/02/27 21:53:06 INFO ipc.HbaseRPC: Server at /10.249.190.85:60020 > could > > > not be reached after 1 tries, giving up. > > > 09/02/27 21:53:09 INFO ipc.HBaseClass: Retrying connect to server: / > > > 10.249.190.85:60020. Already tried 0 time(s). > > > 09/02/27 21:53:10 INFO ipc.HBaseClass: Retrying connect to server: / > > > 10.249.190.85:60020. Already tried 1 time(s). > > > 09/02/27 21:53:11 INFO ipc.HBaseClass: Retrying connect to server: / > > > 10.249.190.85:60020. Already tried 2 time(s). > > > > > > The IP address showing here is actually the old master's IP address > > instead > > > of the new one's. > > > > > > We tried "list" and "scan" commands in the HBase shell, both of them > are > > > working good. Just the "count" reported the above error. > > > > > > What's the problem here? > > > > > > Thanks, > > > Yan > > > > > >
