Hi
I have the following connection refused problem when a tserver is trying to
connect to master.
I checked the ip:port for the services as follows:
log from tserver: 22 17:56:51,289 [ipc.Client] INFO : Retrying connect to
server: ip-10-145-140-125.ec2.internal/10.145.140.125:9000. Already tried 9
time(s).22 17:56:51,292 [logger.LogService] FATAL: Failed to initialize log
service args=[--address,
ip-10-35-77-90.ec2.internal]java.lang.RuntimeException: Accumulo not
initialized, there is no instance id at /accumulo/instance_id at
org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(ZooKeeperInstance.java:272)
at
org.apache.accumulo.server.conf.ZooConfiguration.getInstance(ZooConfiguration.java:65)
at
org.apache.accumulo.server.conf.ServerConfiguration.getZooConfiguration(ServerConfiguration.java:49)
at
org.apache.accumulo.server.conf.ServerConfiguration.getSystemConfiguration(ServerConfiguration.java:58)
at
org.apache.accumulo.server.client.HdfsZooInstance.<init>(HdfsZooInstance.java:62)
at
org.apache.accumulo.server.client.HdfsZooInstance.getInstance(HdfsZooInstance.java:70)
at org.apache.accumulo.server.Accumulo.init(Accumulo.java:132)
at org.apache.accumulo.server.logger.LogService.<init>(LogService.java:142)
at org.apache.accumulo.server.logger.LogService.main(LogService.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616) at
org.apache.accumulo.start.Main$1.run(Main.java:89) at
java.lang.Thread.run(Thread.java:679)Caused by: java.net.ConnectException: Call
to ip-10-145-140-125.ec2.internal/10.145.140.125:9000 failed on connection
exception: java.net.ConnectException: Connection refused at
org.apache.hadoop.ipc.Client.wrapException(Client.java:1099) at
org.apache.hadoop.ipc.Client.call(Client.java:1075) at
org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) at
$Proxy1.getProtocolVersion(Unknown Source) at
org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396) at
org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379) at
org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238) at
org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203) at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123) at
org.apache.accumulo.core.file.FileUtil.getFileSystem(FileUtil.java:554)
at
org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(ZooKeeperInstance.java:258)
... 14 moreCaused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
The netstat listing is
2012-11-22 18:55:29,828 [myid:] - INFO
[main-EventThread:ClientCnxn$EventThread@511] - EventThread shut
downubuntu@ip-10-145-140-125:/opt/zookeeper$ netstat -pnl(Not all processes
could be identified, non-owned process info will not be shown, you would have
to be root to see it all.)Active Internet connections (only servers)Proto
Recv-Q Send-Q Local Address Foreign Address State
PID/Program nametcp 0 0 0.0.0.0:50060 0.0.0.0:*
LISTEN 1610/java tcp 0 0 127.0.0.1:59661
0.0.0.0:* LISTEN 1610/java tcp 0 0
0.0.0.0:50030 0.0.0.0:* LISTEN 1455/java tcp
0 0 0.0.0.0:39123 0.0.0.0:* LISTEN
1740/java tcp 0 0 0.0.0.0:50070 0.0.0.0:*
LISTEN 1050/java tcp 0 0 0.0.0.0:22
0.0.0.0:* LISTEN - tcp 0 0
0.0.0.0:35159 0.0.0.0:* LISTEN 1367/java tcp
0 0 0.0.0.0:50010 0.0.0.0:* LISTEN
1199/java tcp 0 0 0.0.0.0:33786 0.0.0.0:*
LISTEN 1050/java tcp 0 0 0.0.0.0:50075
0.0.0.0:* LISTEN 1199/java tcp 0 0
0.0.0.0:49500 0.0.0.0:* LISTEN 1199/java tcp
0 0 0.0.0.0:50020 0.0.0.0:* LISTEN
1199/java tcp 0 0 0.0.0.0:2181 0.0.0.0:*
LISTEN 1740/java tcp 0 0 0.0.0.0:53383
0.0.0.0:* LISTEN 1455/java tcp 0 0
10.145.140.125:9000 0.0.0.0:* LISTEN 1050/java tcp
0 0 10.145.140.125:9001 0.0.0.0:* LISTEN
1455/java tcp 0 0 0.0.0.0:50090 0.0.0.0:*
LISTEN 1367/java tcp6 0 0 :::22
:::* LISTEN - udp 0 0
0.0.0.0:68 0.0.0.0:* -
Active UNIX domain sockets (only servers)Proto RefCnt Flags Type
State I-Node PID/Program name Pathunix 2 [ ACC ] STREAM
LISTENING 5594 - @/com/ubuntu/upstartunix 2
[ ACC ] SEQPACKET LISTENING 1824 -
@/org/kernel/udev/udevdunix 2 [ ACC ] STREAM LISTENING 6318
-
/var/run/dbus/system_bus_socketubuntu@ip-10-145-140-125:/opt/zookeeper$
