Might be that nimbus.host is not correctly configured in supervisor machine or some firewall issue? It would be better to eliminate that possibility by doing a telnet from supervisor machine to nimbus port.
On Mon, Jul 11, 2016 at 8:55 PM, Walid Aljoby <[email protected]> wrote: > Hi, > > Supervisors have already connected with zookeeper, that is in the same > machine with nimbus. > > Thanks > > ------------------------------ > *From:* Abhishek Agarwal <[email protected]> > *To:* [email protected]; Walid Aljoby <[email protected]> > *Cc:* "[email protected]" <[email protected]> > *Sent:* Monday, July 11, 2016 11:19 PM > *Subject:* Re: Can't Run Topology in Production Cluster > > It looks like supervisor is not able to connect to nimbus server. You > should check the connectivity from supervisor to nimbus > > On Mon, Jul 11, 2016 at 5:09 PM, Walid Aljoby <[email protected]> > wrote: > > > > <http://stackoverflow.com/questions/36654714/error-running-topology-in-production-cluster-with-apache-storm-1-0-0-topology-d#> > Hi all, > > Please see that, > > I have a topology that runs well on a *Local cluster*. But when I try to > run it on a *production cluster* the following things happens: > > 1. The nimbus is up > 2. The storm UI is up > 3. The two workers I use are up > 4. Zookeper is up > 5. I run storm with > storm jar myjar.jar MyClass > 6. Nimbus submits the topology > 7. The topologies and the workers appears in the storm UI > > *BUT:* > > *The topology does not start despite the fact that its status is ACTIVE* > *The log file of the topology does not appear in the workers.* > I have the following log in the worker on the *supervisor.log*: > > > 2016-07-11 05:17:15.281 o.a.s.d.supervisor [INFO] Finished downloading > code for storm id WCT-5-1467890377 > 2016-07-11 05:17:15.288 o.a.s.d.supervisor [INFO] Missing topology storm > code, so can't launch worker with assignment {:storm-id "WCT-5-1467890377", > :executors [[7 7] [22 22] [1 1] [28 28] [16 16] [13 13] [19 19] [10 10] [4 > 4] [25 25]], :resources #object[org.apache.storm.generated.WorkerResources > 0x20f6adce "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} > for this supervisor 79d8581d-d3b8-47c5-8656-3a76a1873aac on port 6702 with > id 630303c9-04ab-4494-9c68-2ebf41a78b7a > 2016-07-11 05:17:15.288 o.a.s.d.supervisor [INFO] Missing topology storm > code, so can't launch worker with assignment {:storm-id "WCT-5-1467890377", > :executors [[8 8] [2 2] [20 20] [23 23] [11 11] [5 5] [26 26] [14 14] [17 > 17]], :resources #object[org.apache.storm.generated.WorkerResources > 0x230cc04a "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} > for this supervisor 79d8581d-d3b8-47c5-8656-3a76a1873aac on port 6701 with > id 0866c809-f152-4b00-9822-b849e2f90f92 > 2016-07-11 05:17:15.288 o.a.s.d.supervisor [INFO] Missing topology storm > code, so can't launch worker with assignment {:storm-id "WCT-5-1467890377", > :executors [[12 12] [3 3] [24 24] [18 18] [6 6] [9 9] [21 21] [27 27] [15 > 15]], :resources #object[org.apache.storm.generated.WorkerResources > 0x1a244c40 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} > for this supervisor 79d8581d-d3b8-47c5-8656-3a76a1873aac on port 6700 with > id aa296a04-622d-4975-b240-d618f49e2638 > 2016-07-11 05:17:15.749 o.a.s.d.supervisor [INFO] Downloading code for > storm id WCT-5-1467890377 > 2016-07-11 05:17:25.810 o.a.s.d.supervisor [WARN] There was a connection > problem with nimbus. #error { > :cause Connection refused > :via > [{:type java.lang.RuntimeException > :message org.apache.storm.thrift.transport.TTransportException: > java.net.ConnectException: Connection refused > :at [org.apache.storm.security.auth.TBackoffConnect retryNext > TBackoffConnect.java 64]} > {:type org.apache.storm.thrift.transport.TTransportException > :message java.net.ConnectException: Connection refused > :at [org.apache.storm.thrift.transport.TSocket open TSocket.java 226]} > {:type java.net.ConnectException > :message Connection refused > :at [java.net.PlainSocketImpl socketConnect PlainSocketImpl.java -2]}] > :trace > [[java.net.PlainSocketImpl socketConnect PlainSocketImpl.java -2] > [java.net.AbstractPlainSocketImpl doConnect AbstractPlainSocketImpl.java > 339] > [java.net.AbstractPlainSocketImpl connectToAddress > AbstractPlainSocketImpl.java 200] > [java.net.AbstractPlainSocketImpl connect AbstractPlainSocketImpl.java > 182] > [java.net.SocksSocketImpl connect SocksSocketImpl.java 392] > [java.net.Socket connect Socket.java 579] > [org.apache.storm.thrift.transport.TSocket open TSocket.java 221] > [org.apache.storm.thrift.transport.TFramedTransport open > TFramedTransport.java 81] > [org.apache.storm.security.auth.SimpleTransportPlugin connect > SimpleTransportPlugin.java 103] > [org.apache.storm.security.auth.TBackoffConnect doConnectWithRetry > TBackoffConnect.java 53] > [org.apache.storm.security.auth.ThriftClient reconnect ThriftClient.java > 99] > [org.apache.storm.security.auth.ThriftClient <init> ThriftClient.java 69] > 2016-07-11 05:17:25.810 o.a.s.d.supervisor [INFO] Finished downloading > code for storm id WCT-5-1467890377 > 2016-07-11 05:17:25.812 o.a.s.d.supervisor [INFO] Missing topology storm > code, so can't launch worker with assignment {:storm-id "WCT-5-1467890377", > :executors [[7 7] [22 22] [1 1] [28 28] [16 16] [13 13] [19 19] [10 10] [4 > 4] [25 25]], :resources #object[org.apache.storm.generated.WorkerResources > 0x450a0b28 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} > for this supervisor 79d8581d-d3b8-47c5-8656-3a76a1873aac on port 6702 with > id 37ca083b-412c-4c5c-af77-afa833d37a01 > 2016-07-11 05:17:25.812 o.a.s.d.supervisor [INFO] Missing topology storm > code, so can't launch worker with assignment {:storm-id "WCT-5-1467890377", > :executors [[8 8] [2 2] [20 20] [23 23] [11 11] [5 5] [26 26] [14 14] [17 > 17]], :resources #object[org.apache.storm.generated.WorkerResources > 0x16629281 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} > for this supervisor 79d8581d-d3b8-47c5-8656-3a76a1873aac on port 6701 with > id 441f1189-e054-44ad-a969-aa48a4e5811b > 2016-07-11 05:17:25.812 o.a.s.d.supervisor [INFO] Missing topology storm > code, so can't launch worker with assignment {:storm-id "WCT-5-1467890377", > :executors [[12 12] [3 3] [24 24] [18 18] [6 6] [9 9] [21 21] [27 27] [15 > 15]], :resources #object[org.apache.storm.generated.WorkerResources > 0x21805ea2 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} > for this supervisor 79d8581d-d3b8-47c5-8656-3a76a1873aac on port 6700 with > id c0e469d0-a62c-4596-80d3-1219ba9af028 > > > Thank You, > > -- > Regards > WA > > > > > -- > Regards, > Abhishek Agarwal > > > > -- Regards, Abhishek Agarwal
