region t113,,1356955542345.5f083a03434e02cc8ec1be0dc095b01b. was stuck in PENDING_OPEN state on hadoop2,60020
Can you get log related to this region from hadoop2 ? You might be experiencing https://issues.apache.org/jira/browse/HBASE-6060 Cheers On Mon, Dec 31, 2012 at 9:17 PM, hua beatls <[email protected]> wrote: > Hi, > master server log below: > b2d218794c262a81641ba43ea4e. daughter > > a=BT_NET_LOG_200,0000189020759862012122618000104000070465,1357016716453.27cf99f6ff9f8a36bd2394ae42dcba98.daughter > > b=BT_NET_LOG_200,0000189021098542012121702200104000070465,1357016716453.70dbb96881d2f5be50d3a6fc4ff38efc. > 2013-01-01 13:05:20,435 DEBUG org.apache.hadoop.hbase.master.HMaster: Not > running balancer because 1 region(s) in transition: > > {5f083a03434e02cc8ec1be0dc095b01b=t113,,1356955542345.5f083a03434e02cc8ec1be0dc095b01b. > state=PENDING_OPEN, ts=1357016719988, server=hadoop2,60020,1357010035445} > 2013-01-01 13:05:20,435 DEBUG > org.apache.hadoop.hbase.client.HTable$ClientScanner: Creating scanner over > .META. starting at key > > i check there is a table 't113' is 'in transition' state, > > > i try to drop it, but failed, below: > > hbase(main):017:0> disable 't113' > 0 row(s) in 0.7790 seconds > > hbase(main):018:0> drop 't113' > > no response , and master server log,below: > > > 2013-01-01 13:09:50,548 DEBUG > org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Waiting on > region to clear regions in transition; > t113,,1356955542345.5f083a03434e02cc8ec1be0dc095b01b. state=PENDING_OPEN, > ts=1357016719988, server=hadoop2,60020,1357010035445 > 2013-01-01 13:09:51,550 DEBUG > org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Waiting on > region to clear regions in transition; > t113,,1356955542345.5f083a03434e02cc8ec1be0dc095b01b. state=PENDING_OPEN, > ts=1357016719988, server=hadoop2,60020,1357010035445 > > 2013-01-01 13:12:42,997 ERROR > org.apache.hadoop.hbase.master.handler.TableEventHandler: Error > manipulating table t113 > java.io.IOException: Waited hbase.master.wait.on.region (300000ms) for > region to leave region > t113,,1356955542345.5f083a03434e02cc8ec1be0dc095b01b. in transitions > at > > org.apache.hadoop.hbase.master.handler.DeleteTableHandler.handleTableOperation(DeleteTableHandler.java:62) > at > > org.apache.hadoop.hbase.master.handler.TableEventHandler.process(TableEventHandler.java:96) > at > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > any other way to delete this annoying table 't113'? > > Thanks! > > beatls > > On Tue, Jan 1, 2013 at 12:57 PM, Ted Yu <[email protected]> wrote: > > > hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40" > > > > You need to specify timestamp for the key value. > > > > After 'balancer' command returns, can you check master log ? You would > see > > why balancer doesn't run. > > > > Cheers > > > > On Mon, Dec 31, 2012 at 8:48 PM, hua beatls <[email protected]> wrote: > > > > > HI, > > > when issue 'balancer', 'false' returned. > > > below is error log: > > > > > > hbase> get 't1', "key\x03\x3f\xcd" > > > hbase> get 't1', "key\003\023\011" > > > hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40" > > > > > > The HBase shell is the (J)Ruby IRB with the above HBase-specific > commands > > > added. > > > For more on the HBase Shell, see > > > http://hbase.apache.org/docs/current/book.html > > > > > > ERROR: wrong number of arguments (1 for 0) > > > > > > Here is some help for this command: > > > Trigger the cluster balancer. Returns true if balancer ran and was able > > to > > > tell the region servers to unassign all the regions to balance (the > > > re-assignment itself is async). > > > Otherwise false (Will not run if regions in transition). > > > > > > > > > hbase(main):003:0> balance_switch true > > > true > > > > > > 0 row(s) in 0.0140 seconds > > > > > > hbase(main):004:0> balancer > > > false > > > > > > 0 row(s) in 0.0060 seconds > > > > > > maybe regions 'not run if regions in transition'. how to know > wheather > > > the region is 'in transition' status or not? any cli? > > > > > > Many Thanks! > > > > > > beatls > > > > > > > > > On Tue, Jan 1, 2013 at 2:39 AM, Ted Yu <[email protected]> wrote: > > > > > > > You need to issue 'balancer' command to trigger an immediate > balancing. > > > > > > > > Thanks > > > > > > > > On Monday, December 31, 2012, hua beatls wrote: > > > > > > > > > HI, > > > > > 1. below is the output : > > > > > > > > > > > > > > > > > > > > hbase(main):023:0> balance_switch true > > > > > true > > > > > > > > > > 0 row(s) in 0.0110 seconds > > > > > > > > > > hbase(main):024:0> [root@hadoop1 ~]# > > > > > > > > > > 2. we have 40 regions of the table; > > > > > 3. master log ok. > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > beatls > > > > > > > > > > On Tue, Jan 1, 2013 at 1:52 AM, Ted Yu <[email protected]> > wrote: > > > > > > > > > > > How many regions does the table have ? > > > > > > > > > > > > What was the output from "balance_switch true" command ? (there > was > > > > typo > > > > > in > > > > > > your command) > > > > > > > > > > > > Can you check master log around the time this command was issued > ? > > > > > > > > > > > > Cheers > > > > > > > > > > > > On Mon, Dec 31, 2012 at 9:48 AM, hua beatls <[email protected]> > > > wrote: > > > > > > > > > > > > > HI, > > > > > > > I am using hbase 0.92.1-cdh4.1.2, and i check it again, and > > > find i > > > > > can > > > > > > > create a table which has only regionser, happen to be on this > > > > > > regionserver, > > > > > > > but i cannot "balance_switch ture" to balance the regions over > > the > > > > > > cluster. > > > > > > > > > > > > > > Best R. > > > > > > > > > > > > > > beatls > > > > > > > > > > > > > > On Tue, Jan 1, 2013 at 1:32 AM, Ted Yu <[email protected]> > > > wrote: > > > > > > > > > > > > > > > The correct syntax is: > > > > > > > > bin/hbase-daemon.sh stop regionserver > > > > > > > > > > > > > > > > What HBase version are you using ? > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > On Mon, Dec 31, 2012 at 8:05 AM, hua beatls < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > HI all, > > > > > > > > > we stop and start regionserver with > > 'bin/hbase-daemon.sh > > > > > stop > > > > > > & > > > > > > > > > bin/hbase-daemon.sh start', the regionserver seems come > up, > > > > > > > > > but it no request was accecpted. > > > > > > > > > and we use 'banlande-switch true', not works. > > > > > > > > > > > > > > > > > > how to make this regionserver work normal? > > > > > > > > > seem there is some problem about zookeeper of this > > > server, > > > > > > below > > > > > > > > is > > > > > > > > > zookeeper.out: > > > > > > > > > > > > > > > > > > (ip addr for this server is 192.168.250.108, > while > > > > > > > > > 192.168.250.106 is another node in this cluster). > > > > > > > > > > > > > > > > > > 2012-12-31 23:24:15,402 [myid:3] - INFO > > [NIOServerCxn.Factory: > > > > > > > > > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1000] - Closed socket > > > > > connection > > > > > > > for > > > > > > > > > client /192.168.250.106:38950 which had sessionid > > > > > 0x33be7062e140389 > > > > > > > > > 2012-12-31 23:26:08,738 [myid:3] - INFO > > [NIOServerCxn.Factory: > > > > > > > > > 0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@202] - Accepted > > > socket > > > > > > > > > connection > > > > > > > > > from /192.168.250.106:39261 > > > > > > > > > 2012-12-31 23:26:08,739 [myid:3] - INFO > > [NIOServerCxn.Factory: > > > > > > > > > 0.0.0.0/0.0.0.0:2181:ZooKeeperServer@824] - Client > > attempting > > > to > > > > > > renew > > > > > > > > > session 0x23be99906db0342 at /192.168.250.106:39261 > > > > > > > > > 2012-12-31 23:26:08,739 [myid:3] - INFO > > [NIOServerCxn.Factory: > > > > > > > > > 0.0.0.0/0.0.0.0:2181:Learner@107] - Revalidating client: > > > > > > > > 0x23be99906db0342 > > > > > > > > > 2012-12-31 23:26:08,741 [myid:3] - INFO > [QuorumPeer[myid=3]/ > > > > > > > > 0.0.0.0:2181 > > > > > > > > > :ZooKeeperServer@595] - Established session > > 0x23be99906db0342 > > > > with > > > > > > > > > negotiated timeout 40000 for client /192.168.250.106:39261 > > > > > > > > > 2012-12-31 23:30:00,051 [myid:3] - INFO > > > > > > > > > [CommitProcessor:3:NIOServerCnxn@1000] - Closed socket > > > > connection > > > > > > for > > > > > > > > > client /192.168.250.106:39261 which had sessionid > > > > > 0x23be99906db0342 > > > > > > > > > 2012-12-31 23:3 > > > > > > > > > >
