You might do well to build a host file so that you can make the host names stable over time.
On Fri, Feb 18, 2011 at 2:29 AM, kushum sharma <[email protected]> wrote: > Hi, > I've deployed hbase on 5 nodes cluster of amazon ec2 successfully and was > working fine. > The next day when I logon I changed the configuration file(regionservers > list,slaves, masters,dfs rootdir etc) > in hadoop and hbase /conf directory for the new IP address which is dynamic > for ec2 > then the hbase master immediately stopped after startup and generated the > error message in log files, it tries to connect > to the old region server address and failed. Again I removed > both hadoop and hbase for fresh installation but it again generates the > error in > connection with an old ip address on port 60020. > I would be grateful if someone could getme out of it. > Thanks. > > log of hbase-root-master > .......... > > 2011-02-18 05:13:32,760 DEBUG > org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Handling OPENED > event for 70236052; deleting unassigned node > 2011-02-18 05:13:32,760 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: > master:60000-0x2e3842768e0003 Deleting existing unassigned node for > 70236052 > that is in expected state RS_ZK_REGION_OPENED > 2011-02-18 05:13:32,831 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: > master:60000-0x2e3842768e0003 Successfully deleted unassigned node for > region 70236052 in expected state RS_ZK_REGION_OPENED > 2011-02-18 05:13:32,831 DEBUG > org.apache.hadoop.hbase.master.handler.OpenedRegionHandler: Opened region > -ROOT-,,0.70236052 on ip-10-203-13-80.ec2.internal,60020,1298024003961 > 2011-02-18 05:13:52,794 INFO org.apache.hadoop.ipc.HbaseRPC: Problem > connecting to server: ip-10-114-29-91.ec2.internal/10.114.29.91:60020 > 2011-02-18 05:14:13,811 INFO org.apache.hadoop.ipc.HbaseRPC: Problem > connecting to server: ip-10-114-29-91.ec2.internal/10.114.29.91:60020 > 2011-02-18 05:14:34,817 INFO org.apache.hadoop.ipc.HbaseRPC: Problem > connecting to server: ip-10-114-29-91.ec2.internal/10.114.29.91:60020 > 2011-02-18 05:14:34,817 FATAL org.apache.hadoop.hbase.master.HMaster: > Unhandled exception. Starting shutdown. > java.net.SocketTimeoutException: 20000 millis timeout while waiting for > channel to be ready for connect. ch : > java.nio.channels.SocketChannel[connection-pending > remote=ip-10-114-29-91.ec2.internal/10.114.29.91:60020] > at > > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:213) > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404) > at > > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:311) > at > org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:865) > at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:732) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257) > at $Proxy6.getProtocolVersion(Unknown Source) > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419) > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393) > at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444) > at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349) > at > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:954) > at > > org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(CatalogTracker.java:384) > at > > org.apache.hadoop.hbase.catalog.CatalogTracker.getMetaServerConnection(CatalogTracker.java:283) > at > > org.apache.hadoop.hbase.catalog.CatalogTracker.verifyMetaRegionLocation(CatalogTracker.java:478) > at > org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:435) > at > > org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:382) > at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:277) > 2011-02-18 05:14:34,841 INFO org.apache.hadoop.hbase.master.HMaster: > Aborting > 2011-02-18 05:14:34,841 DEBUG org.apache.hadoop.hbase.master.HMaster: > Stopping service threads > 2011-02-18 05:14:34,841 INFO org.apache.hadoop.ipc.HBaseServer: Stopping > server on 60000 > 2011-02-18 05:14:34,841 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server > handler 2 on 60000: exiting > 2011-02-18 05:14:34,841 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server > handler 3 on 60000: exiting > .................................. >
