Hi Gao That is because of the timeout.period that is set to default value of 30 mins.
But why was the initialization of the service slow ? May be any exception while trying to open the root region? If that is the case you see HBASE-4287 is one solution. Without HBASE-4287 surely it will take 30 mins. Regards Ram -----Original Message----- From: Gaojinchao [mailto:[email protected]] Sent: Tuesday, September 06, 2011 5:27 PM To: [email protected] Subject: A issue about region server startup. In my cluster , I found a issue that root was assigned after 30 minutes when the cluster start up, because region server startup slowly. Region server start up : Step 1: report to master it has started up. Step 2: init the service Hmaster start up: Step 1:Wait for the region server. Step 2:Assign the root and meta. Because the preceding section, if the region server initializes service slowly. Master may throw exception "Server is not running yet". The root can't be assigned and need wait for 30 minutes. Is there any good way to modify this bug? Who can give me some suggestions? The logs: 2011-08-26 07:53:26,065 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for -ROOT-,,0.70236052 so generated a random one; hri=-ROOT-,,0.70236052, src=, dest=linux-kqm6,20020,1314316432465; 1 (online=1, exclude=null) available servers 2011-08-26 07:53:26,065 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning region -ROOT-,,0.70236052 to linux-kqm6,20020,1314316432465 2011-08-26 07:53:26,065 DEBUG org.apache.hadoop.hbase.master.ServerManager: New connection to linux-kqm6,20020,1314316432465 2011-08-26 07:53:33,251 WARN org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: RemoteException connecting to RS org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningException: Server is not running yet at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771) at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257) at $Proxy6.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444) at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation. getHRegionConnection(HConnectionManager.java:969) at org.apache.hadoop.hbase.master.ServerManager.getServerConnection(ServerManag er.java:620) at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.ja va:555) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.ja va:1043) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.ja va:858) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.ja va:838) at org.apache.hadoop.hbase.master.AssignmentManager.assignRoot(AssignmentManage r.java:1304) at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:431) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:388 ) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:282) 2011-08-26 07:53:33,254 WARN org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of -ROOT-,,0.70236052 to serverName=linux-kqm6,20020,1314316432465, load=(requests=0, regions=0, usedHeap=0, maxHeap=0), trying to assign elsewhere instead; retry=0 org.apache.hadoop.hbase.ipc.ServerNotRunningException: org.apache.hadoop.hbase.ipc.ServerNotRunningException: Server is not running yet at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcces sorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc torAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteE xceptionHandler.java:96) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation. getHRegionConnection(HConnectionManager.java:977) at org.apache.hadoop.hbase.master.ServerManager.getServerConnection(ServerManag er.java:620) at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.ja va:555) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.ja va:1043) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.ja va:858) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.ja va:838) at org.apache.hadoop.hbase.master.AssignmentManager.assignRoot(AssignmentManage r.java:1304) at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:431) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:388 ) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:282) 2011-08-26 07:53:33,254 WARN org.apache.hadoop.hbase.master.AssignmentManager: Unable to find a viable location to assign region -ROOT-,,0.70236052 2011-08-26 07:55:41,226 INFO org.apache.hadoop.hbase.master.ServerManager: Registering server=linux-kqm5,20020,1314316540832, regionCount=0, userLoad=false 2011-08-26 07:56:13,037 INFO org.apache.hadoop.hbase.master.ServerManager: Registering server=linux-kqm4,20020,1314316645788, regionCount=0, userLoad=false 2011-08-26 08:23:36,528 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: -ROOT-,,0.70236052 state=OFFLINE, ts=1314316413254 2011-08-26 08:23:36,529 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been OFFLINE for too long, reassigning -ROOT-,,0.70236052 to a random server 2011-08-26 08:23:36,529 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=-ROOT-,,0.70236052 state=OFFLINE, ts=1314316413254
