Thank you very much. It's hard to remember these details. Yes, I uncommented HBASE_MANAGES_ZK. The following lines are uncommented in my hbase-env.sh:
export JAVA_HOME=/usr/lib/jvm/jdk1.6.0_37 export HBASE_OPTS="-XX:+UseConcMarkSweepGC" export HBASE_MANAGES_ZK=true On Wed, Jan 29, 2014 at 3:11 PM, Fernando Iwamoto - Plannej <[email protected]> wrote: > Iam new to HBASE too, but I had same problem long time ago and I dont > remember how i fixed, I will keep troubleshooting you... > How about zookeeper? have you uncommented the HBASE_MANAGE_ZK(something > like that) in hbase-env.sh and set to TRUE? > > > 2014-01-29 Guang Gao <[email protected]> > >> You mean the SSH key? Yes, any two nodes can ssh each other without >> password. >> >> On Wed, Jan 29, 2014 at 2:10 PM, Fernando Iwamoto - Plannej >> <[email protected]> wrote: >> > Did you tried to pass the key to the machines? >> > >> > >> > 2014-01-29 birdeeyore <[email protected]> >> > >> >> Thanks for your reply. Here's some additional info. Thanks. >> >> >> >> $ cat hbase-site.xml >> >> <configuration> >> >> <property> >> >> <name>hbase.cluster.distributed</name> >> >> <value>true</value> >> >> </property> >> >> <property> >> >> <name>hbase.rootdir</name> >> >> <value>hdfs://obelix8.local:9001/hbase</value> >> >> </property> >> >> <property> >> >> <name>hbase.zookeeper.quorum</name> >> >> <value>obelix105.local,obelix106.local,obelix107.local</value> >> >> </property> >> >> <property> >> >> <name>hbase.zookeeper.property.clientPort</name> >> >> <value>2183</value> >> >> </property> >> >> <property> >> >> <name>hbase.zookeeper.peerport</name> >> >> <value>2890</value> >> >> </property> >> >> <property> >> >> <name>hbase.zookeeper.leaderport</name> >> >> <value>3890</value> >> >> </property> >> >> <property> >> >> <name>hbase.zookeeper.property.dataDir</name> >> >> <value>/ssd/hbase/hbase-0.94.16/zookeeper</value> >> >> </property> >> >> <property> >> >> <name>hbase.master</name> >> >> <value>obelix8.local:60000</value> >> >> </property> >> >> <property> >> >> <name>hbase.master.info.port</name> >> >> <value>50070</value> >> >> </property> >> >> <property> >> >> <name>hbase.client.scanner.caching</name> >> >> <value>200</value> >> >> </property> >> >> </configuration> >> >> >> >> ========================== >> >> >> >> $ cat regionservers >> >> obelix105.local >> >> obelix106.local >> >> obelix107.local >> >> obelix108.local >> >> obelix109.local >> >> obelix110.local >> >> obelix111.local >> >> obelix112.local >> >> obelix113.local >> >> obelix114.local >> >> >> >> ========================= >> >> On my master node: >> >> >> >> $ cat /etc/hosts >> >> 127.0.0.1 localhost >> >> 192.168.245.8 obelix8.local xx.yy.net obelix8 >> >> >> >> # The following lines are desirable for IPv6 capable hosts >> >> ::1 ip6-localhost ip6-loopback >> >> fe00::0 ip6-localnet >> >> ff00::0 ip6-mcastprefix >> >> ff02::1 ip6-allnodes >> >> ff02::2 ip6-allrouters >> >> 192.168.245.1 obelix.local >> >> >> >> =============== >> >> >> >> On one of my slave nodes: >> >> >> >> $ cat /etc/hosts >> >> 127.0.0.1 localhost >> >> 127.0.1.1 obelix105.local xx.yy.net obelix105 >> >> >> >> # The following lines are desirable for IPv6 capable hosts >> >> ::1 ip6-localhost ip6-loopback >> >> fe00::0 ip6-localnet >> >> ff00::0 ip6-mcastprefix >> >> ff02::1 ip6-allnodes >> >> ff02::2 ip6-allrouters >> >> 192.168.245.1 obelix.local >> >> >> >> ========================== >> >> >> >> The error of HBase 0.94.16+Hadoop 1.2.1: >> >> >> >> 2014-01-29 12:58:30,922 INFO >> >> org.apache.hadoop.hbase.regionserver.HRegionServer: Attempting connect >> >> to Master server at obelix8.local,60000,1391018303918 >> >> 2014-01-29 12:58:40,960 WARN >> >> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to connect >> >> to master. Retrying. Error was: >> >> java.net.SocketException: Invalid argument >> >> at sun.nio.ch.Net.connect(Native Method) >> >> at >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532) >> >> at >> >> >> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) >> >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511) >> >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481) >> >> at >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:392) >> >> at >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:438) >> >> at >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1141) >> >> at >> >> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:988) >> >> at >> >> >> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87) >> >> at $Proxy9.getProtocolVersion(Unknown Source) >> >> at >> >> >> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:141) >> >> at >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208) >> >> at >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:2043) >> >> at >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2089) >> >> at >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:747) >> >> at java.lang.Thread.run(Thread.java:662) >> >> >> >> Best, >> >> >> >> Boduo >> >> >> >> >> >> On Wed, Jan 29, 2014 at 8:21 AM, Jean-Marc Spaggiari >> >> <[email protected]> wrote: >> >> > Hi, >> >> > >> >> > can you please share your config files and your host file? >> >> > >> >> > Thanks, >> >> > >> >> > JM >> >> > >> >> > >> >> > 2014-01-29 Guang Gao <[email protected]> >> >> > >> >> >> Hi all, >> >> >> >> >> >> This is my first time to try to setup HBase on a 10-node cluster. I >> >> tried >> >> >> two settings: HBase 0.94.16+Hadoop 1.2.1, and HBase 0.96.1.1+Hadoop >> >> 2.2.0. >> >> >> In both cases, the region servers can't connect to the master >> properly. >> >> >> >> >> >> [HBase 0.94.16+Hadoop 1.2.1]: >> >> >> >> >> >> 2014-01-29 00:55:06,875 WARN >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to >> connect to >> >> >> master. Retrying. Error was: >> >> >> java.net.SocketException: Invalid argument >> >> >> at sun.nio.ch.Net.connect(Native Method) >> >> >> at >> >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) >> >> >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511) >> >> >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:392) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:438) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1141) >> >> >> at >> >> >> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:988) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87) >> >> >> at $Proxy9.getProtocolVersion(Unknown Source) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:141) >> >> >> at >> >> >> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:2043) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2089) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:747) >> >> >> at java.lang.Thread.run(Thread.java:662) >> >> >> >> >> >> >> >> >> >> >> >> [HBase 0.96.1.1+Hadoop 2.2.0]: >> >> >> >> >> >> 2014-01-28 18:50:54,587 WARN [regionserver60020] >> >> >> regionserver.HRegionServer: error telling master we are up >> >> >> com.google.protobuf.ServiceException: java.net.SocketException: >> Invalid >> >> >> argument >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1670) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1711) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:5402) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1926) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:784) >> >> >> at java.lang.Thread.run(Thread.java:662) >> >> >> Caused by: java.net.SocketException: Invalid argument >> >> >> at sun.nio.ch.Net.connect(Native Method) >> >> >> at >> >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) >> >> >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529) >> >> >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupConnection(RpcClient.java:575) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:860) >> >> >> at >> >> >> >> org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1535) >> >> >> at >> >> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1424) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1653) >> >> >> ... 5 more >> >> >> >> >> >> >> >> >> >> >> >> I tried to replace lib/hadoop-core-xxx.jar of HBase 0.94.16 with the >> >> jar of >> >> >> Hadoop 1.2.1, and replace lib/hadoop-*.jar with those of Hadoop >> 2.2.0, >> >> but >> >> >> had no luck. Does anyone have any idea? >> >> >> >> >> >> Thanks. >> >> >> >> >> >>
