The last stack trace just looks like zookeeper isn't running. Can you do a netstat and see if their are any connections open on port 2181?
I don't know what would cause the original failure though, I haven't seen "Java.lang.RuntimeException: Master not initialized after 200 seconds" before, that would indicate some kind of startup problem, maybe the ROOT region couldn't get assigned? Also, did you make any changes to hbase-site.xml between restarts? On Wed, Sep 25, 2013 at 10:26 PM, Roy23 <[email protected]> wrote: > Tried running it again, now the error seems different: > > ava.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) > 2013-09-26 02:18:31,405 WARN > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: *Possibly transient > ZooKeeper exception: > *org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for > /hbase/rs/dev01.ec2.salami.pw,45258,1380161701409 > 2013-09-26 02:18:31,405 INFO org.apache.hadoop.hbase.util.RetryCounter: > Sleeping 8000ms before retry #3... > 2013-09-26 02:18:33,280 INFO org.apache.zookeeper.ClientCnxn: Opening > socket > connection to server localhost/127.0.0.1:2181. Will not attempt to > authenticate using SASL (Unable to locate a login configuration) > 2013-09-26 02:18:33,281 WARN org.apache.zookeeper.ClientCnxn: Session > 0x141580c419a0001 for server null, unexpected error, closing socket > connection and attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) > 2013-09-26 02:18:36,200 INFO org.apache.zookeeper.ClientCnxn: Opening > socket > connection to server localhost/127.0.0.1:2181. Will not attempt to > authenticate using SASL (Unable to locate a login configuration) > 2013-09-26 02:18:36,200 WARN org.apache.zookeeper.ClientCnxn: Session > 0x141580c419a0001 for server null, unexpected error, closing socket > connection and attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) > 2013-09-26 02:18:37,985 INFO org.apache.zookeeper.ClientCnxn: Opening > socket > connection to server localhost/127.0.0.1:2181. Will not attempt to > authenticate using SASL (Unable to locate a login configuration) > 2013-09-26 02:18:37,986 WARN org.apache.zookeeper.ClientCnxn: Session > 0x141580c419a0001 for server null, unexpected error, closing socket > connection and attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) > 2013-09-26 02:18:39,739 INFO org.apache.zookeeper.ClientCnxn: Opening > socket > connection to server localhost/127.0.0.1:2181. Will not attempt to > authenticate using SASL (Unable to locate a login configuration) > 2013-09-26 02:18:39,739 WARN org.apache.zookeeper.ClientCnxn: Session > 0x141580c419a0001 for server null, unexpected error, closing socket > connection and attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at > > org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) > 2013-09-26 02:18:39,839 WARN > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient > ZooKeeper exception: > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for > /hbase/rs/dev01.ec2.salami.pw,45258,1380161701409 > 2013-09-26 02:18:39,840 ERROR > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper delete > failed after 3 retries > 2013-09-26 02:18:39,840 WARN > org.apache.hadoop.hbase.regionserver.HRegionServer: Failed deleting my > ephemeral node > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for > /hbase/rs/dev01.ec2.salami.pw,45258,1380161701409 > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873) > at > > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:133) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1195) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1184) > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1134) > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899) > at java.lang.Thread.run(Thread.java:619) > 2013-09-26 02:18:40,520 INFO org.apache.zookeeper.ZooKeeper: Session: > 0x141580c419a0001 closed > 2013-09-26 02:18:40,520 INFO org.apache.zookeeper.ClientCnxn: EventThread > shut down > 2013-09-26 02:18:40,520 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: stopping server > dev01.ec2.salami.pw,45258,1380161701409; zookeeper connection closed. > 2013-09-26 02:18:40,520 INFO > org.apache.hadoop.hbase.regionserver.HRegionServer: > RegionServer:0;dev01.ec2.salami.pw,45258,1380161701409 exiting > 2013-09-26 02:18:40,520 INFO > org.apache.hadoop.hbase.regionserver.ShutdownHook: Starting fs shutdown > hook > thread. > 2013-09-26 02:18:40,521 INFO > org.apache.hadoop.hbase.regionserver.ShutdownHook: Shutdown hook finished. > > > > -- > View this message in context: > http://apache-hbase.679495.n3.nabble.com/Needed-to-restart-HBase-Now-Master-won-t-start-tp4051241p4051244.html > Sent from the HBase User mailing list archive at Nabble.com. > -- *Michael Webster*, Software Engineer Bronto Software [email protected] bronto.com <http://www.bronto.com/> Marketing solutions for commerce. Learn more.<http://www.bronto.com/platform>
