Dear all,
I'm facing the following problem when I try to submit a storm command
(idem for a topology) on a remote AWS EC2 cluster from my development
machine (debian8):
(Note that for debugging purposes I've set up the MASTER/ZK/WORKER on
three different instances to have distinct IPs)
etienne@debian8:~$ storm list
Running: /opt/jdk/jdk1.8.0_121/bin/java -client -Ddaemon.name=
-Dstorm.options= -Dstorm.home=/opt/apache-storm-1.0.1
-Dstorm.log.dir=/opt/apache-storm-1.0.1/logs
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dstorm.conf.file= -cp
/opt/apache-storm-1.0.1/lib/reflectasm-1.10.1.jar:/opt/apache-storm-1.0.1/lib/log4j-slf4j-impl-2.1.jar:/opt/apache-storm-1.0.1/lib/storm-core-1.0.1.jar:/opt/apache-storm-1.0.1/lib/log4j-core-2.1.jar:/opt/apache-storm-1.0.1/lib/slf4j-api-1.7.7.jar:/opt/apache-storm-1.0.1/lib/kryo-3.0.3.jar:/opt/apache-storm-1.0.1/lib/storm-rename-hack-1.0.1.jar:/opt/apache-storm-1.0.1/lib/disruptor-3.3.2.jar:/opt/apache-storm-1.0.1/lib/clojure-1.7.0.jar:/opt/apache-storm-1.0.1/lib/servlet-api-2.5.jar:/opt/apache-storm-1.0.1/lib/objenesis-2.1.jar:/opt/apache-storm-1.0.1/lib/asm-5.0.3.jar:/opt/apache-storm-1.0.1/lib/minlog-1.3.0.jar:/opt/apache-storm-1.0.1/lib/log4j-api-2.1.jar:/opt/apache-storm-1.0.1/lib/log4j-over-slf4j-1.6.6.jar:/home/etienne/.storm:/opt/apache-storm-1.0.1/bin
org.apache.storm.command.list
Exception in thread "main" java.lang.RuntimeException:
org.apache.storm.thrift.transport.TTransportException:
java.net.UnknownHostException: ip-172-xx-xx-253.eu-west-1.compute.internal
at
org.apache.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:64)
at
org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:56)
at
org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:99)
at
org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69)
at
org.apache.storm.utils.NimbusClient.<init>(NimbusClient.java:106)
at
org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:78)
at org.apache.storm.command.list$_main.invoke(list.clj:22)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at org.apache.storm.command.list.main(Unknown Source)
Caused by: org.apache.storm.thrift.transport.TTransportException:
java.net.UnknownHostException: ip-172-xx-xx-253.eu-west-1.compute.internal
at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:226)
at
org.apache.storm.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
at
org.apache.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
at
org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:53)
... 8 more
Caused by: java.net.UnknownHostException:
ip-172-xx-xx-253.eu-west-1.compute.internal
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:221)
... 11 more
My .storm/storm.yaml is set like this:
storm.zookeeper.servers:
- "176.xx.xxx.50"
nimbus.seeds: ["54.xxx.xxx.91"]
ui.host: "54.xxx.xxx.91"
What I don't get is that the UnknownHostException:
ip-172-xx-xx-253.eu-west-1.compute.internal returned is on the nimbus
AWS EC2 private DNS.
Any body has a clue of what's wrong there?
Note1: The command "storm list" works fine from all 3 instances when I
ssh to them.
Note2: If I amend my /etc/hosts file with "public IP private IP" of
my AWS EC2 nimbus, all is fine. But as far as I understand it should
not be needed, and I don't want to edit any configuration file each time
a new instance is launched.
Thank you for your help!
Etienne
