I am using the zookeeper that is provided by the hbase. Do I need to install zookeeper separately?
On Thu, Oct 4, 2012 at 8:48 PM, Jean-Marc Spaggiari <[email protected] > wrote: > I'm not 100% sure, but it looks like your "master" is not really your > master. > > ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node > /hbase/master already exists and this is not a retry > > > And it's starting as a backup master: > Adding ZNode for > /hbase/backup-masters/oc-PowerEdge-R610,60000,1349337115099 in backup > master directory > > And that might be why your regionservers can't connect to it since > it's not really the master. > > Can you shut everyting down (including zookeeper), start zookeeper > only and see what's into it? You might have some entries which might > be deleted. > > 2012/10/4, Venkateswara Rao Dokku <[email protected]>: > > Please find the attached logs. > > > > On Thu, Oct 4, 2012 at 7:37 PM, Jean-Marc Spaggiari > > <[email protected] > >> wrote: > > > >> Can you paste the entire logs from you zookeeper, master and > >> regionservers (the 2) on pastebin and provide the link to take a look? > >> > >> 2012/10/4, Venkateswara Rao Dokku <[email protected]>: > >> > I could only see this > >> > 2012-10-04 06:25:45,419 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:46,919 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:48,420 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:49,920 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:51,420 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:52,920 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > > >> > > >> > On Thu, Oct 4, 2012 at 6:50 PM, Jean-Marc Spaggiari > >> > <[email protected] > >> >> wrote: > >> > > >> >> Hi Venkateswara, > >> >> > >> >> What do you have on your master's logs? Do you have anything? > >> >> > >> >> JM > >> >> > >> >> 2012/10/4, Venkateswara Rao Dokku <[email protected]>: > >> >> > Hi, > >> >> > I configured 2 node hbase cluster with hadoop-0.20.2 & hbase > >> 0.92.1. > >> >> The > >> >> > installation went fine. One is the namenode & the other will act > asa > >> >> > datanode as well as region server, from the region server I > couldn't > >> >> access > >> >> > HMaster. I was getting the following error when I try to scan the > >> >> > tables > >> >> on > >> >> > the namenode > >> >> > > >> >> > hbase(main):001:0> list > >> >> > TABLE > >> >> > > >> >> > > >> >> > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 > >> >> > times > >> >> > > >> >> > Here is some help for this command: > >> >> > List all tables in hbase. Optional regular expression parameter > >> >> > could > >> >> > be used to filter the output. Examples: > >> >> > > >> >> > hbase> list > >> >> > hbase> list 'abc.*' > >> >> > > >> >> > But when I see the processes running with jps command on name-node( > >> >> master) > >> >> > & datanode(slave) I got the following list > >> >> > *namenode:* > >> >> > * > >> >> > * > >> >> > 6367 NameNode > >> >> > 6553 DataNode > >> >> > 7473 HMaster > >> >> > 7375 HQuorumPeer > >> >> > 6866 JobTracker > >> >> > 10391 Jps > >> >> > 6764 SecondaryNameNode > >> >> > 7041 TaskTracker > >> >> > > >> >> > *datanode:* > >> >> > * > >> >> > * > >> >> > 15710 TaskTracker > >> >> > 15556 DataNode > >> >> > 13126 -- process information unavailable > >> >> > 19259 Jps > >> >> > 13218 NC > >> >> > 15874 HRegionServer > >> >> > > >> >> > The following logs are logged into the hbase-region-server log file > >> >> > 2012-10-04 18:26:17,692 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Attempting > >> >> > connect > >> >> > to > >> >> > Master server at oc-PowerEdge-R610,60000,1349337115099 > >> >> > 2012-10-04 18:27:18,491 WARN > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to > >> >> > connect > >> >> > to > >> >> > master. Retrying. Error was: > >> >> > java.net.ConnectException: Connection refused > >> >> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > >> >> > at > >> >> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) > >> >> > at > org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045) > >> >> > at > >> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150) > >> >> > at $Proxy7.getProtocolVersion(Unknown Source) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183) > >> >> > at > >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) > >> >> > at > >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) > >> >> > at > >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) > >> >> > at > >> >> > > org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1643) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1680) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:643) > >> >> > at java.lang.Thread.run(Thread.java:662) > >> >> > I was struck here & couldn't get any help. Please help me out in > >> >> rectifying > >> >> > the error. > >> >> > -- > >> >> > Thanks & Regards, > >> >> > Venkateswara Rao Dokku, > >> >> > Software Engineer,One Convergence Devices Pvt Ltd., > >> >> > Jubille Hills,Hyderabad. > >> >> > > >> >> > >> > > >> > > >> > > >> > -- > >> > Thanks & Regards, > >> > Venkateswara Rao Dokku, > >> > Software Engineer,One Convergence Devices Pvt Ltd., > >> > Jubille Hills,Hyderabad. > >> > > >> > > > > > > > > -- > > Thanks & Regards, > > Venkateswara Rao Dokku, > > Software Engineer,One Convergence Devices Pvt Ltd., > > Jubille Hills,Hyderabad. > > > > > 2012/10/4, Venkateswara Rao Dokku <[email protected]>: > > Please find the attached logs. > > > > On Thu, Oct 4, 2012 at 7:37 PM, Jean-Marc Spaggiari > > <[email protected] > >> wrote: > > > >> Can you paste the entire logs from you zookeeper, master and > >> regionservers (the 2) on pastebin and provide the link to take a look? > >> > >> 2012/10/4, Venkateswara Rao Dokku <[email protected]>: > >> > I could only see this > >> > 2012-10-04 06:25:45,419 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:46,919 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:48,420 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:49,920 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:51,420 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > 2012-10-04 06:25:52,920 INFO > >> org.apache.hadoop.hbase.master.ServerManager: > >> > Waiting on regionserver(s) to checkin > >> > > >> > > >> > On Thu, Oct 4, 2012 at 6:50 PM, Jean-Marc Spaggiari > >> > <[email protected] > >> >> wrote: > >> > > >> >> Hi Venkateswara, > >> >> > >> >> What do you have on your master's logs? Do you have anything? > >> >> > >> >> JM > >> >> > >> >> 2012/10/4, Venkateswara Rao Dokku <[email protected]>: > >> >> > Hi, > >> >> > I configured 2 node hbase cluster with hadoop-0.20.2 & hbase > >> 0.92.1. > >> >> The > >> >> > installation went fine. One is the namenode & the other will act > asa > >> >> > datanode as well as region server, from the region server I > couldn't > >> >> access > >> >> > HMaster. I was getting the following error when I try to scan the > >> >> > tables > >> >> on > >> >> > the namenode > >> >> > > >> >> > hbase(main):001:0> list > >> >> > TABLE > >> >> > > >> >> > > >> >> > ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 > >> >> > times > >> >> > > >> >> > Here is some help for this command: > >> >> > List all tables in hbase. Optional regular expression parameter > >> >> > could > >> >> > be used to filter the output. Examples: > >> >> > > >> >> > hbase> list > >> >> > hbase> list 'abc.*' > >> >> > > >> >> > But when I see the processes running with jps command on name-node( > >> >> master) > >> >> > & datanode(slave) I got the following list > >> >> > *namenode:* > >> >> > * > >> >> > * > >> >> > 6367 NameNode > >> >> > 6553 DataNode > >> >> > 7473 HMaster > >> >> > 7375 HQuorumPeer > >> >> > 6866 JobTracker > >> >> > 10391 Jps > >> >> > 6764 SecondaryNameNode > >> >> > 7041 TaskTracker > >> >> > > >> >> > *datanode:* > >> >> > * > >> >> > * > >> >> > 15710 TaskTracker > >> >> > 15556 DataNode > >> >> > 13126 -- process information unavailable > >> >> > 19259 Jps > >> >> > 13218 NC > >> >> > 15874 HRegionServer > >> >> > > >> >> > The following logs are logged into the hbase-region-server log file > >> >> > 2012-10-04 18:26:17,692 INFO > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Attempting > >> >> > connect > >> >> > to > >> >> > Master server at oc-PowerEdge-R610,60000,1349337115099 > >> >> > 2012-10-04 18:27:18,491 WARN > >> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to > >> >> > connect > >> >> > to > >> >> > master. Retrying. Error was: > >> >> > java.net.ConnectException: Connection refused > >> >> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > >> >> > at > >> >> > > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) > >> >> > at > org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045) > >> >> > at > >> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150) > >> >> > at $Proxy7.getProtocolVersion(Unknown Source) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183) > >> >> > at > >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) > >> >> > at > >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) > >> >> > at > >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) > >> >> > at > >> >> > > org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1643) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1680) > >> >> > at > >> >> > > >> >> > >> > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:643) > >> >> > at java.lang.Thread.run(Thread.java:662) > >> >> > I was struck here & couldn't get any help. Please help me out in > >> >> rectifying > >> >> > the error. > >> >> > -- > >> >> > Thanks & Regards, > >> >> > Venkateswara Rao Dokku, > >> >> > Software Engineer,One Convergence Devices Pvt Ltd., > >> >> > Jubille Hills,Hyderabad. > >> >> > > >> >> > >> > > >> > > >> > > >> > -- > >> > Thanks & Regards, > >> > Venkateswara Rao Dokku, > >> > Software Engineer,One Convergence Devices Pvt Ltd., > >> > Jubille Hills,Hyderabad. > >> > > >> > > > > > > > > -- > > Thanks & Regards, > > Venkateswara Rao Dokku, > > Software Engineer,One Convergence Devices Pvt Ltd., > > Jubille Hills,Hyderabad. > > > -- Thanks & Regards, Venkateswara Rao Dokku, Software Engineer,One Convergence Devices Pvt Ltd., Jubille Hills,Hyderabad.
