Guang, Try what Jean said, even though its work in that way, I've been told that it's better(normal?) to use 192.168 ip to connect btw machines. And i don't remember why.
2014-01-29 Kim Chew <[email protected]> > Not sure if this will help but it is worth to take a look the master > hostname /ip used by zk and make sure the same hostname/ip is in your > /etc/hosts. For example, > > hbase zkcli > get /hbase/master > > Kim > > > On Wed, Jan 29, 2014 at 12:11 PM, Fernando Iwamoto - Plannej < > [email protected]> wrote: > > > Iam new to HBASE too, but I had same problem long time ago and I dont > > remember how i fixed, I will keep troubleshooting you... > > How about zookeeper? have you uncommented the HBASE_MANAGE_ZK(something > > like that) in hbase-env.sh and set to TRUE? > > > > > > 2014-01-29 Guang Gao <[email protected]> > > > > > You mean the SSH key? Yes, any two nodes can ssh each other without > > > password. > > > > > > On Wed, Jan 29, 2014 at 2:10 PM, Fernando Iwamoto - Plannej > > > <[email protected]> wrote: > > > > Did you tried to pass the key to the machines? > > > > > > > > > > > > 2014-01-29 birdeeyore <[email protected]> > > > > > > > >> Thanks for your reply. Here's some additional info. Thanks. > > > >> > > > >> $ cat hbase-site.xml > > > >> <configuration> > > > >> <property> > > > >> <name>hbase.cluster.distributed</name> > > > >> <value>true</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.rootdir</name> > > > >> <value>hdfs://obelix8.local:9001/hbase</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.zookeeper.quorum</name> > > > >> <value>obelix105.local,obelix106.local,obelix107.local</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.zookeeper.property.clientPort</name> > > > >> <value>2183</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.zookeeper.peerport</name> > > > >> <value>2890</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.zookeeper.leaderport</name> > > > >> <value>3890</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.zookeeper.property.dataDir</name> > > > >> <value>/ssd/hbase/hbase-0.94.16/zookeeper</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.master</name> > > > >> <value>obelix8.local:60000</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.master.info.port</name> > > > >> <value>50070</value> > > > >> </property> > > > >> <property> > > > >> <name>hbase.client.scanner.caching</name> > > > >> <value>200</value> > > > >> </property> > > > >> </configuration> > > > >> > > > >> ========================== > > > >> > > > >> $ cat regionservers > > > >> obelix105.local > > > >> obelix106.local > > > >> obelix107.local > > > >> obelix108.local > > > >> obelix109.local > > > >> obelix110.local > > > >> obelix111.local > > > >> obelix112.local > > > >> obelix113.local > > > >> obelix114.local > > > >> > > > >> ========================= > > > >> On my master node: > > > >> > > > >> $ cat /etc/hosts > > > >> 127.0.0.1 localhost > > > >> 192.168.245.8 obelix8.local xx.yy.net obelix8 > > > >> > > > >> # The following lines are desirable for IPv6 capable hosts > > > >> ::1 ip6-localhost ip6-loopback > > > >> fe00::0 ip6-localnet > > > >> ff00::0 ip6-mcastprefix > > > >> ff02::1 ip6-allnodes > > > >> ff02::2 ip6-allrouters > > > >> 192.168.245.1 obelix.local > > > >> > > > >> =============== > > > >> > > > >> On one of my slave nodes: > > > >> > > > >> $ cat /etc/hosts > > > >> 127.0.0.1 localhost > > > >> 127.0.1.1 obelix105.local xx.yy.net obelix105 > > > >> > > > >> # The following lines are desirable for IPv6 capable hosts > > > >> ::1 ip6-localhost ip6-loopback > > > >> fe00::0 ip6-localnet > > > >> ff00::0 ip6-mcastprefix > > > >> ff02::1 ip6-allnodes > > > >> ff02::2 ip6-allrouters > > > >> 192.168.245.1 obelix.local > > > >> > > > >> ========================== > > > >> > > > >> The error of HBase 0.94.16+Hadoop 1.2.1: > > > >> > > > >> 2014-01-29 12:58:30,922 INFO > > > >> org.apache.hadoop.hbase.regionserver.HRegionServer: Attempting > connect > > > >> to Master server at obelix8.local,60000,1391018303918 > > > >> 2014-01-29 12:58:40,960 WARN > > > >> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to > connect > > > >> to master. Retrying. Error was: > > > >> java.net.SocketException: Invalid argument > > > >> at sun.nio.ch.Net.connect(Native Method) > > > >> at > > > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532) > > > >> at > > > >> > > > > > > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) > > > >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511) > > > >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:392) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:438) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1141) > > > >> at > > > >> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:988) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87) > > > >> at $Proxy9.getProtocolVersion(Unknown Source) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:141) > > > >> at > > > >> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:2043) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2089) > > > >> at > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:747) > > > >> at java.lang.Thread.run(Thread.java:662) > > > >> > > > >> Best, > > > >> > > > >> Boduo > > > >> > > > >> > > > >> On Wed, Jan 29, 2014 at 8:21 AM, Jean-Marc Spaggiari > > > >> <[email protected]> wrote: > > > >> > Hi, > > > >> > > > > >> > can you please share your config files and your host file? > > > >> > > > > >> > Thanks, > > > >> > > > > >> > JM > > > >> > > > > >> > > > > >> > 2014-01-29 Guang Gao <[email protected]> > > > >> > > > > >> >> Hi all, > > > >> >> > > > >> >> This is my first time to try to setup HBase on a 10-node > cluster. I > > > >> tried > > > >> >> two settings: HBase 0.94.16+Hadoop 1.2.1, and HBase > 0.96.1.1+Hadoop > > > >> 2.2.0. > > > >> >> In both cases, the region servers can't connect to the master > > > properly. > > > >> >> > > > >> >> [HBase 0.94.16+Hadoop 1.2.1]: > > > >> >> > > > >> >> 2014-01-29 00:55:06,875 WARN > > > >> >> org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to > > > connect to > > > >> >> master. Retrying. Error was: > > > >> >> java.net.SocketException: Invalid argument > > > >> >> at sun.nio.ch.Net.connect(Native Method) > > > >> >> at > > > >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) > > > >> >> at > > org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511) > > > >> >> at > > org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:392) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:438) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1141) > > > >> >> at > > > >> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:988) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:87) > > > >> >> at $Proxy9.getProtocolVersion(Unknown Source) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:141) > > > >> >> at > > > >> >> > > org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:2043) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2089) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:747) > > > >> >> at java.lang.Thread.run(Thread.java:662) > > > >> >> > > > >> >> > > > >> >> > > > >> >> [HBase 0.96.1.1+Hadoop 2.2.0]: > > > >> >> > > > >> >> 2014-01-28 18:50:54,587 WARN [regionserver60020] > > > >> >> regionserver.HRegionServer: error telling master we are up > > > >> >> com.google.protobuf.ServiceException: java.net.SocketException: > > > Invalid > > > >> >> argument > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1670) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1711) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:5402) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1926) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:784) > > > >> >> at java.lang.Thread.run(Thread.java:662) > > > >> >> Caused by: java.net.SocketException: Invalid argument > > > >> >> at sun.nio.ch.Net.connect(Native Method) > > > >> >> at > > > >> sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) > > > >> >> at > > org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529) > > > >> >> at > > org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupConnection(RpcClient.java:575) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:860) > > > >> >> at > > > >> >> > > > > org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1535) > > > >> >> at > > > >> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1424) > > > >> >> at > > > >> >> > > > >> >> > > > >> > > > > > > org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1653) > > > >> >> ... 5 more > > > >> >> > > > >> >> > > > >> >> > > > >> >> I tried to replace lib/hadoop-core-xxx.jar of HBase 0.94.16 with > > the > > > >> jar of > > > >> >> Hadoop 1.2.1, and replace lib/hadoop-*.jar with those of Hadoop > > > 2.2.0, > > > >> but > > > >> >> had no luck. Does anyone have any idea? > > > >> >> > > > >> >> Thanks. > > > >> >> > > > >> > > > > > >
